Evaluating Collaboration experience in Spatial VR
Created by Dave Song Spring 2023
VR SARS-CoV-2 Analysis
For Task Load Comparison, participants were asked to answer questions regarding task load. Used Likert Scale from 1 - 10 where 1 = Not at all and 10 = Very
*graphs on the right column represents answers regarding Spatial VR experience
On average, participants agreed that VR collaboration was more mentally demanding the one in-person.
On average, participants gave 4.43 for in-person and 6 for VR. Additionally, the distribution of the answers tells shows that the participants, overall, agree on the fact that VR collaboration requires similar or more physical effort.
As displayed in the two graphs, the participants felt more hurries or rushed when using Spatial VR to collaborate. This is largely due to time constraints the class had for in-class activity and the fact that VR collaboration took place near the end of the class time.
Performance survey results were interesting since more participants felt they were more successful in accomplishing the goal of the tasks. On average, participants gave 4 for in-person collaboration and 5.67 for VR collaboration. This can be explained by individual's expectations of VR collaboration, Spatial VR's collaborative tools, and the fact that VR collaboration took place after in-person collaboration.
However, participants felt VR collaboration as more demanding option.
Besides task load, participants also evaluated the VR system in terms of usability.
In general, users found the system somewhat complex and hard to use mainly due to:
lack of accurate full body motion detection
However, most users also highlighted that the system was easy to user with features:
Overall interaction design
Participants also provided individuals' definition of an "ideal collaboration experience." From the evaluation and feedback, key attributes are:
tools without bugs and glitches
Full body tracking
environment where people can measure and analyze data in real time
hand and finger tracking
Tools and Modes of Communication
Participants also identified some of the tools in Spatial VR that they utilized and enjoyed using:
Overall, SpatialVR and its collaborative features generally supported smooth collaboration when it comes to non-verbal communication and scientific rendering analysis. However, the major issue with the software was the lack of full body and finger motion tracking. 3D pens and sticky notes where users can share their ideas in a more conventional way in VR space were utilized. One of the key highlights from the activity and feedback analysis is the higher value of performance given to the VR system over real-life collaboration. While future work will be focusing on if the performance differences exist in controlled experiments.
Notes on body and motion tracking provided enough feedback and served as a great foundation when researching and developing evaluation rubrics for the VR collaboration experience. One thing I realized as I learn more about VR collaboration is that realism is at the center of collaboration experience quality. Knowing that human interaction includes more than just highly expressive rendering motion tracking, towards the end of the research phase, I became more interested in the potential of an AR collaborative system where collaborators are co-located in a traditional collaboration manner with assist or augmented visualization of the dataset of renderings. Future work will explore performance, task load, and social presence differences in AR and VR collaborative systems.