Evaluating Collaboration in XR

Created by Dave Song, Spring 2023 

Evaluating Collaboration in VR 

As mentioned in the course description, CSCI1951T aims to explore, evaluate, and compare multi-user VR software and applications. While some students may also choose to explore existing projects or develop their own VR projects that align better with their interests, I wanted to focus on the evaluation of existing VR systems and software. Interested in User Experience research and UI designs, I wanted to study collaborative user experience in VR and the factors that influence the quality of the experience. For the second half of the semester, I conducted half-semester-long research to study and examine current studies on VR or XR collaboration evaluation. Additionally, I developed a standard evaluation tool for VR collaboration experience.



But, why do you care about VR collaboration quality evaluation?


Research Report and Key Insights(Related Work)

Developing a standardized evaluation tool for collaborative VR systems required basic knowledge and insights on current practice of evaluating VR systems and applications. First part of the project was to gather insights from related research papers. Some suggested having certain features while others suggested wholistic framework of evaluating vr systems. Listed below is a list of related work as well as key takeaways form each work related to evaluation of VR evaluation quality.


Click here to access Evaluating VR Systems page where more in-depth summaries and analysis are available.

From this, I could identify some of the key factors that influence the quality and performance of VR collaboration.

VR collaboration Evaluation Form


Collaboration Experience Evaluation Forms

: Collaboration and data handled inside the collaborative VR systems can be highly context based. Therefore, evaluating the quality of VR collaboration for a particular collaboration workflow of a particular group of researchers or experts is essential to provide development -feedback cycle. 

   



VR Collaborative Feature Evaluation Form

: It is also important to have a standard metric for evaluating collaborative features. Developing the evaluation form upon the comparison matrix developed previously, collaborative feature evaluation focus on identifying and displaying features that support collaborative experience in VR. 


Comparison Matrix