Goals

before after

---- ----

1 | 5 | Goal 1: articulate VR visualization software tool goals, requirements, and capabilities;

1 | 4 | Goal 2: construct meaningful evaluation strategies for software libraries, frameworks, and applications; strategies include surveys, interviews, comparative use, case studies, and web research;

1 | 5 | Goal 3: execute tool evaluation strategies;

0 | 3 | Goal 4: build visualization software packages;

1 | 5 | Goal 5: comparatively analyze software tools based on evaluation;

1 | 4 | Goal 6: be familiar with a number of VR software tools and hardware;

1 | 5 | Goal 7: think critically about software;

1 | 4 | Goal 8: communicate ideas more clearly;

0 | 4 | Goal 9: demonstrate VR tools to non-users;

PROPOSALS

**Google Drive Links are shared within Brown. If it does not work, please try Brown gmail. Thanks! 

Project 1 Proposal <Project 1 Plan>

Presentation for Project 1 Proposal <Project 1 Proposal Presentation>

In-class Activity <2D and 3D network>

In-class Activity Wiki Handout <Activity 1 Handout>

In-class Activity Feedback Form Results<Network Visualization Activity Feedback>

End Presentation for Project 1 <Project 1 Final Presentation: Visualizing Community Network Graphs in 2D vs. 3D>

Project 2 Proposal <Project 2 Proposal and Plan>

Presentation for Project 2 Proposal <Project2 Plan&Proposal>

In-class Activity Feedback Form Result 1<In-person Charades Feedback>

In-class Activity Feedback Form Result 2 <SpatialVR Collaboration Feedback>

In-class Activity<Activity 2 Handout>

Flash Talk Presentation<Link>

Poster <Link>

Public Demo <ADD LINK>

ALL OTHER WIKI CONTRIBUTIONS

CONTRIBUTION 1 [2023 Updates on Twitter API] <Twitter API Update Contribution>

CONTRIBUTION 2 [Redirected all the children pages under Student research page] <Student Research>

CONTRIBUTION 3 [wrote a tutorial/introduction page on 2D network visualization using Gephi] <2D Network Visualization>

CONTRIBUTION 4 [Wrote an informative wiki page on community network visualization in Unity] <Network Visualization in VR>

CONTRIBUTION 5[Wrote a project 1 report reviewing in-class activity feedback/VR application evaluation] <Project 1 Final Report & Software evaluation>

CONTRIBUTION 6[Wrote VR Collaboration page] <VR Collaboration: collaborative VR systems>

CONTRIBUTION 7[Combined and updated the list/evaluation of collaborative workspaces] <VR Collaboration>

CONTRIBUTION 8 [Under general VR collaboration page, added evaluating VR systems page] <VR collaboration evaluation , Link>

CONTRIBUTION 9 [Spatial VR evaluation report from in-class activity] <Evaluating Collaboration in SpatialVR>

CONTRIBUTION 10 [evaluating collaboration in XR: research and related work portion of Project 2] <Evaluating VR collaboration>

CONTRIBUTION 11[Journal Article Review 1] <Insight Based Method for Evaluating VR Visualization>

CONTRIBUTION 12[Journal Article Review 2] <Advanced Evaluation of VR Collaboration>

CONTRIBUTION 13[Journal Article Review 3] <Ontology of Evaluation>

CONTRIBUTION 14[Journal Article Review 4] <Fully Expressive Avatar in VR Collaboration>

CONTRIBUTION 15[Journal Article Review 5] <Evaluation of a Multi-user VR System for Collaborative Layout Planning Process>

CONTRIBUTION 16[Journal Article Review 6] <Evaluation of Mixed-Space Collaboration >

CONTRIBUTION 17[Updated Comparison Matrix from previous semesters + added collaborative feature evaluation] <Comparison Matrix[Google Sheets]>

CONTRIBUTION 18 [added evaluating vr systems page: Lists existing evaluation tools, collaboration evaluation forms developed as project 2] <Evaluating VR Systems>

CONTRIBUTION 19 [a short wiki page on NASA TLX] <NASA Task Load Index>

CONTRIBUTION 20 [a short wiki page on SUS] <SUS Evaluation Tool>

HOURS journal

1/29/23 - 4 Hours

• Set up student journal, joined course slack,  reviewed and explored the wiki as well as course homepage 

• 9 Changes to Wiki 

  • 10-minute changes:

1. Add recent VR hardware devices to the list of VR hardware on the wiki homepage 

2. On the VR development software page, there are #todo items that have not been executed. 

3. Since there is a dedicated page for VR development software in depth, the “VR development software” page under home> "I want to decide which VR tools to use for a specific user" can have a proper page title to avoid confusion regarding page hierarchy. [Completed] Changed the page title 

• • 1 hour to complete 

1. Under I want to decide which VR tools to use for a specific use case, Unreal Engine 4 is empty. 

2. Currently, if you access the VR Development Software comparison page, users only get access to the parts within the page about each software. It would be a better user workflow to also have links to each software page. 

3. On WebVR: Overview page, Wiki displays links to WebVR software. Some links lead to the software homepages while some lead to our wiki's review/analysis page. We can add links to Wiki pages as well as company websites to fix such inconsistencies in the links on the page. 

• • 10 hours to complete 

1. There is a page for Low budget VR headsets. However, it would be great to have a page dedicated to comparing low to high-budget VR headsets. It would be ideal to have each headset include a link back to pages already existing about each headset. This would take a while since doing comparisons of currently available headsets can take more than a couple of hours

2. It would be great to have a page on Meta’s Quest Pro under VR hardware. As the functionality of the quest pro increases, users can perform a variety of tasks. Adding critical research and potential usage of them on the wiki.  

3. Under the applications of the VR page, VR in Design/Art can be added. Immersive contemporary art grows more as VR and AR technologies become readily available to the public. It would be a great addition to the list of applications on our wiki. 

• Read an article on Meta Quest Pro's mixed reality system 

2/1/23 - 2 Hours

• Followed Quest 2 Setup Guide, Paperspace setup 

• Explored Oculus and VR earth

• Attended Jakobi's office hour to ask questions regarding Virtual Desktop 

2/2/23 - 1.5 hours 

• Pick 2-5 potential pieces of software to explore and evaluate for your research project 

  • Visualization Software 

    • Paraview, OpenSpace

  • Unreal engine, Unity3D, 

• Potential Project Ideas 

  • Collaborative 3D architecture model visualization&editing (Structure, environment)

  • Visualizing Twitter Community Detection results with different factors(that multiple users can view)

  • Immersive tour of our solar system 

  • Blue whale experience VR – their anatomy and behavior analysis

2/8/23 - 3.5 hrs 

• Finished DinoVR activities and survey 

• Tried DinoVR with different features enabled 

• Watched a course video on Community detection 

  • Girvan-Newman Algorithm, edge betweenness values, high edge betweenness means high possibility of holding edges together. 

  • CS224W: Machine Learning with Graphs 

  • how does information flow through the network? Social network.  How do people find out about new jobs? : Mark Grannovetter's work in 1960s.

  • Contacts were often acquaintances rather than closer friends 

    • Two perspectives on friendships: 1. structural -- spanning different parts of the network 2. Interpersonal -- can be strong or weak

  • Strength and structure of the friendships. Information flow: long-range edges allow you to gather information from different parts of the network and get a job.  -> access from different communities 

  • Social network evolve to naturally form communities -> triadic closure.  = high clustering coefficient 

  • Empirical study by Bearman and Moody: Teenage girls with low clustering coefficient are more likely to contemplate suicide 

    • proves the natural tendency and dependency of a person to be deeply embedded in the network/clusters 

  • However, Granovetter's theory was not tested. -> 2007 this was evaluated based on 20% of EU population 

  • Project plan

2/11/23 - 4hrs 

• read articles on Community detection algorithms and networks

• Watched tutorials on Gephi and Force Atlas 3d for 3d visualization of the network 

• Force Directed Graph 

• Gephi's clustering algorithms for Modularity seem to be the most widely used algorithm for the community detection for social media accounts and similar datasets 

• Found a step-by-step tutorial on Sample datasets(2)

• Have not found a way to get the resulting file from Gephi to Unity Yet. 

• Phonton seems to be the most reliable collaboration tool. 

2/12/23 - 3.5hrs 

• Was stuck finding ways to make Gephi graphs into 3D and turn that into VR experience in Unity 

• Jakobi redirected me to check out the force-directed approach which is another way of displaying closeness among nodes 

• Unity has a physics engine as well as tools to develop force-directed graph representation in 3D 

• Downloaded unity hub 

• Found some other Github repo for 3D force-directed graphs

Project Plan(Updated)

• Changed from Twitter social network community detection to general community  detection 

  • Twitter just shut down its public API (now for full access, we need to pay?)

  •  Other datasets were also available 

  • Focusing on compare/contrast between Gephi's 2D community detection software vs. Unity Force-directed graph in VR

• In-class activity: compare community visualization produced by Gephi(2D) vs. force-directed graph of the network(VR)

  • Which method visualized the separation of communities(or clusters) better and why do you think that method did a better job visualizing the individual group of nodes? 

  • What are the differences between the two methods? 

  • Which method allowed better visualization and understanding of the links between groups or nodes? What aspect of the method do you think enabled such visualization? 

• Wiki Contributions/deliverables: Tutorials on Gephi and Force Directed Graphs 

  • How to use Gephi and upload data(node and edge files) to develop a community detection network. 

    • Introduction to Modularity and 4 main algorithms in Gephi 

    • This page will go under the Related Technologies Section of the wiki 

  • Tutorial and background summary: what is a force-directed graph and how to make one using Unity. 

• Collaborative Functionalities:

  • Users(2) can touch each node and apply force to the graph 

  • Users(2) can be in a synchronized 3D virtual space in which they can interact with, discuss, and analyze patterns of rendered community networks.

• Worked on 3 min presentation 

2/13/2023 - 3 hrs 

• Found wiki tutorial for Unity Photon 

• HW for 2/14 

• Evaluation of Project Plan 

Here is an evaluation rubric for projects:

• Project Evaluation

o The proposed project clearly identifies deliverable additions to our VR Software Wiki:4

o The proposed project involves collaboration in VR:4

o The proposed project involves large scientific data visualization along the lines of the "Scientific Data" wiki page and identifies the specific data type and software that it will use:5

o The proposed project has a realistic schedule with explicit and measurable milestones at least each week and mostly every class:3

o The proposed project explicitly evaluates VR software, preferably in comparison to related software:4

o The proposed project includes an in-class activity, which can be formative (early in the project) or evaluative (later in the project):3

o The proposed project has resources available with sufficient documentation:4

• First Milestone for Feb. 14th

  • Generate one small network graph and community network using Gephi 

  • Followed this tutorial

  • An example dataset is ppl sending mails in the EU

  • Data: 1000 nodes, 14,116 edges (1-mode, directed)

2/17/2023 - 4.5 hours

• Lost the original Gephi file for some reason

• Found some bugs regarding workspace and data management for Mac 

• Reinstalled Gephi and recreated the project. 

• Recreated the community detection from dataset one(1000 nodes and 14,116 edges) 

• Also explored Gephi functionalities. Created a 2 mode nework from a dataset containing 110 nodes and 142 edges

2/22/2023 - 2.5 hrs 

• Completed installation steps for both Paraview and Ben's Volume Rendering applications

• Played with different functionalities of Paraview(section, contour, color edits, etc.)

2/23/2023 - 2hr 

• Read articles and documentation for Twitter API v2 

• Researched claimed 'Pros' of the new API v2 with fees 

• Added a summary of the 3 main access levels On the Science Data page 

2/27/2023 - 2.5 hrs

• worked on project1 update presentation for Tuesday. 

• Found a professional VR network visualization tool Aviar Graph

• Reached out the CEO and got a permission for one-off software demo 

• Also researched about the current state of net work visualization software and technologies (both VR and desktop) and found Neo4j graph database which Aviar Graph utilizes in order to generate it's 3D visualization of networks. 

3/1/2023 - 1 hrs 

• finished Melvin's in-class activity

• class in class activities prep

• had some issues with sidequest working with my personal computer.  

3/3/2023 - 4hrs

• Debugging the graph problem.  At the moment, the graph get's generated. However, it disappears into the void. 

• Tried to work with the collider as well as mass and gravity setting for the node prefab. 

3/6/2023 - 2hrs 

• worked on the in-class activities handout.

• Tried out HackMD for the first time (originally tried to make a handout with a google doc. However, I saw hackMD handout and decided to learn how to use it and make one for my in-class activity). 

• My in-class activity handout can be found here.

• The dataset and the apk file I used for the in-class activity can be found here.  

3/11/2023 - 2.5hrs

• Worked on deliverables - Gephi Wiki page as well as Unity Page 

• Tried to make the nodes responsive to the number of nodes connected. Encountered some errors. making progress.

3/15/2023 - 2hrs 

• worked on the Project 1 Reflection wiki page content 

• made project 1 presentation 

• Wrote self evaluation of my progress 

3/19, 20 / 2023 - 2hrs 

• Updated Presentation 

• fixed some visualizations for FDG network visualization. Played with gravity and mass since I could not prevent it from flying away. 

• Enabling gravity and light mass ended up working well 

• worked on revised self-evaluation

3/20/2023 - 4hrs 

• Updated Project Final Report page

• Read HW paper 

• worked on project 2 ideation

3/21/2023 - 2hrs 

• Seven Scenarios
  
  

  • Understanding environments and work practices (UWP)

    • Understanding the work, analysis, or information processing practices by a given group of people with or without software in use.

  • Evaluating visual data analysis and reasoning (VDAR)

    • assess a visualization tool’s ability to support visual analysis and reasoning about data.

  • Evaluating communication through visualization (CTV)

    • the goal or purpose to convey a message to one or more persons, in contrast to targeting focused data exploration or discovery. Their effectiveness is usually measured in terms of how effectively such a message is delivered and acquired.

  • Evaluating collaborative data analysis (CDA)

    • Evaluations in this group study how an information visualization tool supports collaborative analysis and/or collaborative decision-making processes. Collaborative sys- tems should support both taskwork, the actions required to complete the task, and teamwork, the actions required to complete the task as a group

  • Evaluating user performance (UP)

    • Evaluations in the UP group study if and how specific features affect objectively measurable user performance.

  • Evaluating user experience (UE)

    • Evaluation of user experience seeks to understand how people react to a visualization either in a short or a long time span.

  • Evaluating visualization algorithms (VA)

    • Evaluations in the VA group study the performance and quality of visualization algorithms by judging the generated output quantitatively. A visualization algorithm is broadly defined as a procedure that optimizes the visual display of information according to a given visualization goal.

• Started Ideation for second/ final project
  
  

• Idea 1 

  • Evaluating Collaborative workspace for architecture 3d space modeling

  • Seven Scenarios in terms of Project 2 idea 1

    • From UWP:

      • What is the context of use of visualizations?

        • 3D rendering of an architecture is sometimes not enough to give a sense of scale as well as perspectives.

    • From VDAR:

      • How does it support processes aimed at seeking infromation, searcing, filtering, and reading and extracting information?

        • The VR visualization of the architecture project enables extracting spacial information easier with realistic analysis of light, reflection, as well as potential material choices.

    • From CTV:

      • Do people learn better and/or faster using the visualization tool?

        • Having a VR space where people/stakeholders can experience the space instead of 3d rendering/floor plans will make the process of understanding the space faster.

    • From CDA

      • Does the tool support effective and efficient collaborative data analysis?

        • If some of the existing softwares provide mutiplayer feature, yes.

    • From UP

      • How does one visualization or interaction technique compare to another as measured by human performance?

        • Comparison of overall structure understanding using floor plan vs. the vr experience can be done

    • From UE

      • Is the tool understandable and can it be learned?

        • intuitive architecture rendering VR software should be able to provide an intuitive way to nagivate through the space.

    • From VA

      • N/A

Project 2 idea 2

• Visualizing public transportation volume before after lock down (major cities).

  • More research needed.

3/22/2023 - 3.5hr

• Improved Wiki contribution -- fixed links, page locations, page title. Fixed based on feedback from midterm review.

• Worked on Ideation for Project 2. Looked at serveral project pages as well as Online resources for potential APIs, softwares, and others.

Project 2 Idea: Collaborative VR Workspace User Experience Analysis

• Project Name: Collaborative VR Workspace User Experience Analysis

  • Motivation: Although there are pages on Collaborative VR software and Collaborative workspaces in the wiki, there is no analysis of VR collaboration user experience, nor is there any comparison of collaboration using the Seven Scenario Analysis.

  • Project Description: Evaluate the current state of Collaborative VR spaces for science and other applications. Potentially conduct in-depth research on collaborative communication and interaction experiences in VR space and propose new techniques to improve the collaborative VR experience.

  • Similar format to Aakansha Mathur's page.

• Wiki Results

  • Collaboration User Experience Wiki Page

    • Analysis of VR collaboration and factors that influence the quality of collaborative workflows.

  • Collaborative VR Visualization Page

    • Collection of tutorials for developing collaborative VR projects.

  • Collaborative VR App and Software Hub Page

    • Analysis of existing pages and reorganization to highlight collaboration in VR.

  • Collaborative User Experience Analysis for Art/Other Areas

  • Additional Section for VR Collaboration Course Homepage.

• Milestones:

  • April 4: Reorganize previous collaborative VR projects and learn how they implemented collaborative aspects of their projects. Read research papers on collaboration in VR and the factors that affect their quality. Apply Seven Scenarios to the projects.

  • April 6: Read at least two research papers on collaboration in VR and research collaborative environment applications in VR.

  • April 11: Buffer and analyze collaborations in scientific data visualization.

  • April 13: Work on the Collaborative VR Visualization Page and scientific visualization software.

  • April 18: Buffer and compare and analyze. Finish the Collaborative VR Visualization Page and Collaborative VR App/Software Hub Page.

  • April 20: Start working on the Collaboration User Experience Analysis.

  • April 25: Incorporate Quest Pro experience if possible, and research VR vs. AR.

  • April 27: Buffer and finish setting up analysis metrics and user experience research.

  • May 2: Try out and analyze collaborative software for art and workspaces.

  • May 4: Finish up the wiki contributions and finalize the analysis report.

4/2/2023 - 2.5hrs 

• read multiple wiki pages related to UX evaluations or software evaluation metrics 

• revisited the seven scenarios paper in order to get relevant questions + general direction to develop my VR collaboration evaluation project. 

4/3/2023 - 2hrs 

• Looked through more wiki pages 

• fixed a link under A-frame section of "webVR collaboration tutorial"  of "WebVR: overview" page. 

• Evaluated Current Wiki Page "Collaborative Workspaces" and their metrics 

• Cleared some of ambiguity regarding the goal of project 2

• HW for 4/4 

  • Project Description: Develop evaluation metrics for Collaborative User Experience in VR Data Storytelling and evaluate three software: Glue, MeetingVR, and FlowImmersive

• • conduct more research on factors that influence Collaborative UX in VR, with a heavy focus on contributing to the wiki and conducting research studies 

  • Potential In class activity description: split into pairs. Each group will encounter challenges (details TBD: solving a puzzle, constructing a given object from parts, etc) and they will time the duration of the activity for the three selected software

Question1 : Data Exploration: how does it support processes aimed at seeking information, searching, filtering, and reading and extracting information collaboratively?

Question 2: Decision Making: How does it support the communication and application of analysis results?

Question 3: How are certain system features used during collaborative work? What are patterns of system use?

4/9/2023 - 3.5 hours 

• Read "evaluation of a multi-user VR system for collaborative layout planning processes" 

• Read "evaluation of Mixed Space Collaboration" 

• Reviewed NASA Task Load Index 

• Read and learned about Nielsen's Heuristics 

• Read and learned about System Usability Scale. 

4/10/2023 - 0.5 hours 

• Prepared Lexi's in-class activity 

• Read two articles on Hyundai and Kia's new VR/AR design evaluation system 

4/12/2023 - 6.5 hrs 

• journal articles, research papers on evaluation of XR systems 

  • "Evaluation of a multi-user VR system for collaborative layout planning processes "

  • "Using Fully Expressive Avatar to Collaborate in Virtual Reality: evaluation of Task performance, presence, and attraction"

  • "Evaluating the effect of collaboration and competition in navigation tasks and spatial knowledge acquisition within virtual reality environments"

  • "An Ontology for Evaluation of remote collaboration using AR"

  • Towards Advanced Evaluation of Collaborative XR Spaces

• Read wiki pages on collaborative workspaces and created collaboration in XR page under Applications of VR 

• Here's the project timeline (pivoted and updated version)

4/13/2023 - 2.5hrs 

• VR workspace software to use and review for the project 

  • candidates 

    • Flow Immersive 

    • Glue 

    • MeetinVR 

    • Vizible 

    • Immersed 

    • Meta Horizon Workrooms 

    • Immersed 

    • Spatial 

-----------------------

MeetinVR 

MozillaHub 

Vizible 

• Worked on project 2 progress presentation 

4/16/2023 - 1.5 hrs 

• explored Meta Horizon Workrooms 

• MeetinVR 

• learned about Flow Immersive 

• revisited Engage VR features 

• Brainstorm/ideation for in class activity 

4/17/2023 - 5 hrs

• Finished Reading journal articles and drafted Journal Article reviews on Notion

  •  Ontology of Evaluation

  • Fully Expressive avatar in VR collaboration

  • Evaluation of a multi-user VR system for collaborative layout planning process

4/18/2023 - 2.5 hrs

• In-class activity design 

• Designed in person and vr activities based charades. With difficulty controlled. 

• Worked on standard collaboration XR rubric (added more contextual guide to the evaluation form). 

4/19/2023 - 4hrs 

• changed the in-class activity 

• Decided to use SpatialVR.

• Initially tried to use MeetinVR to perform Charade rounds in-person and in VR. However, I edited the activity a bit to include scientific data analysis activity where people are told to collaboratively analyze different protein structures of Sars-Cov-2 Virus.

• Encountered some bugs uploading existing obj file to MeetinVR environments.

• Changed the software for in-class activity from MeetinVR to SpatialVR. Although MeetinVR provides wide range of interpersonal communication tools compared ot SpatialVR, SpatialVR seemed to be a right fit(having both highly expressive avatars and option to upload 3d models).

• Worked on HackMD documentation for the in-class activity

• Worked on the evaluation survey for the activity.

4/22/2023 - 3hrs

• finished reading journal articles and wrote a review wiki page draft on Notion 

  • advanced evaluation of VR collaboration 

  • Evaluation of Mixed-Space Collaboration

4/24/2023 - 1.5 hrs 

• prepared for Melvin and Yefei’s inclass activities

• followed tutorials for Horizon Worlds and Horizon Workrooms

• Followed Immersed VR tutorial and set up hand tracking mode

• Tried using multi-screen feature of ImmersedVR and did some work in the VR environment for 15 minutes for testing

4/26/2023 - 1 hrs 

• Liza and Yuanbo’s in-class activities

4/28/2023 - 2hr

• encountered problems with paperspace machine during in-class activity. finished Liza's in-class activity. 

• Finished Melvin's in-class activity

5/1/2023 - 4.5hrs

• finished reading "Insight based method for evaluating VR visualization" and made wiki draft on Notion 

• finished drafting Evaluation tools 

• Added insight based evaluation portion to collaborative vr data visualization

5/4/2023 - 1hr 

• Worked on Project 2 End Presentation 

5/5/2023 - 3hrs 

• worked on wiki contributions 

• moved journal article review pages to Course Wiki from notion drafts 

• worked on In-class activity & SpatialVR evaluation Analysis page

5/8/2023 - 1.5 hrs

• flash talk presentation 

• Poster Draft 1 for class on Tuesday

5/13/2023 - 1hr 

• fixed contribution links and completed adding wiki contributions.

5/14/2023 -3hrs 

• additional wiki pages on SUS, NASA Task Load Index 

• Added google form templates to previous wiki contributions

• total of 133.5 hours so far 

5/15/2023 - 6hrs 

• Evaluated collaboration features of Horizon Worlds and Workrooms (different from collaboration evaluation. It is for updated comparison matrix)<link>

• Interested in AR collaboration and tools, I looked up some AR devices including NReal AR Glasses

• Revised Flash Talk Presentation 

• Final Version of project poster

5/16 - 17 / 2023 - 3.5 hrs

• Printed poster for the final presentation 

• practiced flash talk 

• finished and shared my self-evaluation with Prof. Laidlaw and Jakobi 

• Checked my demo 

• Finalized Wiki contributions