Course Objectives

1. articulate AR/VR visualization software tool goals, requirements, and capabilities;

2. construct meaningful evaluation strategies for software libraries, frameworks, and applications; strategies include surveys, interviews, comparative use, case studies, and web research;

3. execute tool evaluation strategies;

4. build visualization software packages;

5. comparatively analyze software tools based on evaluation;

6. be familiar with a number of AR/VR software tools and hardware;

7. think critically about software;

8. communicate ideas more clearly;

My Scores

before after
----  ----
  1  | 4 | Goal 1: I'm starting class only with knowledge on the potential use cases of AR and VR in science and other fields. By the end of the semester I will be able to comfortably reach this goal.
  2  | 5 | Goal 2: Currently have some background from conducting research, will ideally feel super comfortable knowing how to evaluate software that I haven't necessarily used before.
  1  | 4 | Goal 3: I would like to feel very comfortable in analyzing the pros and cons of different software tools.

  1  | 4 | Goal 4: In order to feel comfortable with this goal I aim to have enough knowledge on what are the existing libraries, etc and their uses.
  2  | 5 | Goal 5: I would like to be able to comfortably be able to compare software tools and be able to confidently suggest which ones to use in what situations.

  1  | 5 | Goal 6: Like many of the goals before, I haven't had experience working with any type of AR/VR tools, I would like to use this class to explore the available software tools and hardware in depth. 

  3  | 5 | Goal 7: I try to think critically of software in general, trying to understand the broader context it exists in and the implications of that. I think I can be better at doing this more systematically.

  3  | 5 | Goal 8: I think I do this pretty well, but I want to keep practicing it, specially in the context of new technologies.

  2  | 5 | Goal 9: With so many technologies out there, scoping becomes a super important aspect of any project. My goal is to effectively scope the knowledge needed to conduct a project, while also explicitly noting of things I might have excluded.

Projects

ALL OTHER WIKI CONTRIBUTIONS

1. Updated Hardware page to include AR and VR hardware (previously it was only the latter) here

2. Spectacles by Snapchat (AR Hardware):  here

3. Monocle by Brilliant Labs (AR Hardware): here

4. VR in MRI for Neuroimaging (Made a one-stop page for all things MRI / Neuroimaging. Moved some of the existing pages around and linked them. Included file types, softwares, evaluation methods, adjacent tutorials):  here

5. Virtual Brain Segmenter (Software): here

6. BodyScope (Software): here

7. EmbodyXR (Service / Software): here

8. fMRI (Data Analysis, Uses): here

9. Building Unity Apps for Collocated multiplayer on Meta Quest (Development): here

10. Networking Frameworks in Unity for Collocated Multiplayer (Development): here

HOURS journal

04/07/2025 - PROJECT + HW (5 hours)

• Created progress update presentation for class

• Updated timeline

• Started planning in class activity and writing the instructions

• Continued Unity development (trying to get collaboration to work)

04/03/2025 - Project (2 hours)

• Starting setting up Unity with dependencies to try and get basic collaboration + surface projected passthrough + poke interaction with meta blocks 

04/03/2025 - HW + Project (5 Hours)

• Revise Bloom's Taxonomy

• (timeline deliverable) Understanding how fMRI data is encoded and how to decode it + adding it to the VR in MRI wiki page.

• (timeline deliverable) Software exploration (sp fMRI in Paraview)

• (deliverable) started document with development information for technical aspect of project: here

• (late) mid semester evaluation

04/01/2025 - HW + Project Work (3 hours)

• Schedule time preferences for project 2 in-class activities: Choosing Wed 22nd (changed course timeline + adjusted project timeline to reflect this)

• Review seven scenarios paper + choose 3 evaluation questions that apply to project 2:

1. Understanding Work Practices (UWP): "How do neuroscience researchers currently interact with and analyze MRI data, and what specific visualization challenges could be addressed through AR/VR technology?"

2. Collaborative Data Analysis (CDA): "How does the accuracy of identifying anomalies in MRI data differ when researchers work collaboratively in the AR/VR environment versus individually?"

• More applicable / potentially evaluated for the project as: How do observations differ between individual and collaborative modes?

3. User Experience (UE): "What is the difference in spatial understanding and recall of 3D brain structures when users interact with fMRI data in AR/VR compared to conventional display methods?"

Project Work:

• Doing background research on the current use of fMRI today, trying to understand what features it's best to try and develop as well as identifying gaps in the field.

• Add wiki page on fMRI from research paper that I read before as task for point above. Also update VR in MRI page

3/20/2025 - HW (30 minutes)

• Project Self Evaluation:

1. clearly identifies deliverable additions to our VR Software Wiki

5: Yes, all on the spreadsheet.

2. involves passthrough or “augmented” in VR

4: Yes, not quite clear on what exactly will be implemented, but has ideas.

3. involves large scientific data visualization along the lines of the "Scientific Data" wiki page and identifies the specific data type and software that it will use

4: Yes, fMRI data. But same as point above.

4. has a realistic schedule with explicit and measurable milestones at least each week and mostly every class

  5: Yes, very explicit.

5. explicitly evaluates VR software, preferably in comparison to related software

  3: Mentions on software that will be used as well as necessary exploration. Not enough progress for proper evaluation.

6. includes an in-class activity, which can be formative (early in the project) or evaluative (later in the project)

5: Yes, a more evaluative activity.

7. has resources available with sufficient documentation

Not enough progress on project timeline to make an evaluation on this?

3/18/2025 - Extra Wiki Contributions (1.5 hours)

• Added some of the extra wiki contributions as described above, all mostly related to AR Hardware

3/11/2025 -  HW (4 hours) - not added to summary

• Project Updates:

  • Worked on tutorial for demo

  • Worked on wiki additions (software, etc)

  • Worked on putting findings from in class activity together

• Others:

  • finish in class acitivities (ben's volume rendering)

3/06/2025 -  HW (5 hours)

• Project Updates:

  • Finish and present in class experiment.

    • Instructions:  here! 

    • Post-Activity Questionnaire: here

3/4/2025 -  HW (8 hours)

• Prepped for SideQuest activity

• Project Updates:

  • Figuring out Meta Building blocks in Unity

  • Figuring out Meta XR Simulator for MacOS use

  • Finishing building AR and VR scenes

• Updated journal with progress

Steps I followed for Unity Development + Testing:

• https://developers.meta.com/horizon/documentation/unity/unity-project-setup/

• https://developers.meta.com/horizon/documentation/unity/unity-passthrough-tutorial-with-blocks/

• https://developers.meta.com/horizon/documentation/unity/unity-passthrough-tutorial/

  • Realized I was developing on my Mac (needed to be a windows device or another type of troubleshooting)

  • Troubleshooting:

    • There is a new integration of Meta XR development for MacOS, had to update some command lines and run on the terminal:

      • brew tap Oculus-VR/tap

      • brew install meta-xr-simulator

        • https://github.com/Oculus-VR/homebrew-repo/blob/main/meta-xr-simulator.md?utm_source=chatgpt.com 

    • Meta XR Simulator not appearing on menu bar

      • https://www.reddit.com/r/OculusQuest/comments/1ixoyxb/meta_xr_simulator_version_72_not_showing_up_on/ 

      • Added package by url: com.meta.xr.simulator

    • FASTER Iteration With Meta XR Simulator - MacOS Support Available!

2/27/2025 -  HW (4 hours)

• Project updates:

  • Got 3D model from MRI images

  • Got it ready in 3D Slicer (.stl) and MeshLab (.obj) for use in Unity.

  • Mainly followed this tutorial: https://www.youtube.com/watch?v=k1WIpwV-8lE 

2/20/2025 -  HW (5 hours)

• Project Updates:

  • Discarded paraview option (too complex)

  • Started exploring 3D Slicer ( + MeshLab)

    • Found data set, explored 3D Slicer offerings and followed some tutorials

      • https://www.andreasjakl.com/visualising-mri-ct-scans-in-mixed-reality-vr-ar-part-1-importing-data/ 

• https://www.andreasjakl.com/visualizing-mri-ct-scans-mixed-reality-vr-ar-part-2-3d-volume-rendering/ 

• • • • https://www.andreasjakl.com/visualizing-mri-ct-scans-mixed-reality-vr-ar-part-3-3d-model-maker/ 

2/18/2025 -  HW (3 hours)

• Project Updates:

  • Decided on getting 3D Mesh of Brain from MRI (instead of using existing model)

  • Started exploring Paraview

    • Got quite stuck

  • Also started thinking of what the AR vs VR scenes would look like

2/13/2025 -  HW (.5 hours)

• Journal Self Evaluation:

  • Journal activities are explicitly and clearly related to course deliverables:  5

  • Deliverables are described and attributed in wiki:  3? Haven't made many additions to the wiki (yet)

  • Report states total amount of time:  5, It's there

  • Total time is appropriate: 5, I've reported the time that it's taken me to work on these projects.

2/11/2025 -  HW (4.5 hours)

• Completed Project 1 Proposal Page 

• Completed  Project 1 Presentation

• Updated/Finalized Project 1 Schedule

• Inserted DinoVR screenshots (unsuccessful simulation)

2/6/2025 -  HW (4 hours)

• Gave feedback to pre-project plans in activity board (Mia,  Eunjin)

• Started solidiying project proposal + created project timeline

• Kept looking on wiki for fMRI related software / past work

  • Found a software NeuroCave, add to wiki: http://brain.uic.edu/software 

  • https://www.vrwiki.cs.brown.edu/vr-visualization-software/medicalholodeck

    • Could update this page (certain links and information is outdated) 

  • https://www.vrwiki.cs.brown.edu/vr-visualization-software/syglass 

  • Mainly, Liza Kolev's Project 2 included working with Paraview and Medical Holodeck

  • https://www.vrwiki.cs.brown.edu/course-years/spring-2023/journals-2023/lizas-journal/project-2-in-class-activity-handout

  • https://www.vrwiki.cs.brown.edu/course-years/spring-2023/journals-2023/lizas-journal/project-2-proposal-plan-lk 

  • https://www.vrwiki.cs.brown.edu/course-years/spring-2023/journals-2023/lizas-journal/paraview-vs-medicalholodeck#h.gz11bpotes2k 

2/4/2025 -  In Class Activity

• DinoVR - Submitted feedback form

2/4/2025 -  HW (5 Hours)

• DinoVR: installing it, reading DinoVR paper to prepare for in-class activity running the app and reproducing images.  

• Solidify  3 project ideas:

1) fMRI VR Visualization

Three things to do during the project:

• Explore different VR tools for fMRI visualization (e.g., SyGlass, Medical Holodeck, 3D Slicer), comparing their capabilities and ease of use. 

• Document how fMRI data is integrated into VR, considering real-time interaction and accessibility.  

• Investigate the current landscape of fMRI visualization tools, comparing open-source and proprietary options in terms of cost, features, and adoption in research.

Class activity idea:

• VR Data Walkthrough

  • Goal: Is this the best visualization mode? Is it one of the cases where 2D > 3D?

  • Have students take turns exploring a defined VR environment and note usability aspects like navigation, interactivity, and clarity.

  • Follow-up discussion on whether the VR approach enhances understanding compared to traditional 2D imaging.

Potential deliverables:

• A comparative analysis of the multiple fMRI VR tools, focusing on accuracy, user-friendliness, and visualization capabilities (Wiki: MRI section).  

• A step-by-step tutorial on how to use the USC VR Tool for fMRI correction (Wiki: MRI section).  

• A table evaluating different software on parameters like accuracy, real-time interaction, ease of integration, and cost.  (Wiki: MRI section)

Software evaluation metrics to brainstorm:

• Error correction accuracy for before vs. after using VR correction tools), Ease of use for neuroscientists vs. general users, Real-time interactivity

2) Population Distribution in Argentina (Urban vs. Rural)

Three things to do during the project:

• Collect population data from Argentina and categorize it by urban vs. rural areas. 

  • Would have to manually do it from INDEC’s census.

• Use VR to create an interactive heatmap or 3D visualization of population distribution.

• Compare different tools (GIS software, VR mapping tools) for their effectiveness in presenting demographic data.  

Class activity idea:

• Group activity where students visualize population data using different tools (e.g., Unity, Cesium for geospatial VR) and compare outputs.  

Potential deliverables:

• A tutorial on creating population heatmaps in VR using GIS tools (Wiki: Data Visualization section).  

• A comparison of different tools for visualizing demographic data in VR (Wiki: Data Visualization section).  

• An evaluation table ranking VR software based on usability, rendering quality, and scalability.  

Software evaluation metrics to brainstorm:

• Data accuracy, Scalability, Visualization clarity, User interaction

3) Interactive Menstrual Cycle Visualization -- More education oriented / Science communication

Three things to do during the project:

• Design an interactive 3D model of the menstrual cycle, mapping hormonal fluctuations over time.  

• Explore tools like WebXR, Unity, or Unreal Engine to create an engaging and educational experience.  

• Test different ways to present data (e.g., VR charts, immersive animations) for clarity and engagement.  

Class activity idea:

• Group testing: students interact with different versions of the model and provide feedback on clarity, engagement, and scientific accuracy.  

Potential deliverables:

• A tutorial on building an interactive hormonal cycle visualization in VR (Wiki: Medical Visualization section).  

• A user study on how interactive models improve understanding of the menstrual cycle compared to static charts (Wiki: Medical Visualization section).  

• A comparison of VR tools for educational health simulations, including usability and effectiveness rankings.  

Software evaluation metrics to brainstorm:

• • Scientific accuracy, User engagement, Educational effectiveness? VR tool flexibility so how easily can the model be updated or expanded?

• Continue adding potential software / data to the wiki

• • Under MRI in Scientific Data, add Virtual Brain Segmenter (USC’s software) https://sites.usc.edu/duncanlab/virtual-brain-segmenter/

1/30/2025 -  In Class Activity (50 minutes)

Google Earth VR, Google Earth Web, Completed form comparing them

Google Earth VR

Google Earth Web