Brian's Journal
before after
---- ----
1 | 5 | Goal 0: example goal showing "novice" score before and "expert mentor" after
1 | 4 | Goal 1: articulate VR visualization software tool goals, requirements, and capabilities;
1 | 3 | Goal 2: construct meaningful evaluation strategies for software libraries, frameworks, and applications; strategies include surveys, interviews, comparative use, case studies, and web research;
1 | 3 | Goal 3: execute tool evaluation strategies;
1 | 2 | Goal 4: build visualization software packages;
1 | 4 | Goal 5: comparatively analyze software tools based on evaluation;
1 | 4 | Goal 6: be familiar with a number of VR software tools and hardware;
1 | 5 | Goal 7: think critically about software;
1 | 5 | Goal 8: communicate ideas more clearly;
1 | 3 | Goal 9: demonstrate VR tools to non-users;
Google Earth VR
Above, left: the house in California I lived in during high school
Above, middle: my current off-campus housing in Providence
Above, right: the airport at Indianapolis where I met my girlfriend during Covid
Left: my video
Above, left: the house in California I lived in during high school
Above, right: my current off-campus housing in Providence
Below, left: the airport at Indianapolis where I met my girlfriend during Covid
Below, right: my video
ALL OTHER WIKI CONTRIBUTIONS
CONTRIBUTION 1 [Adding links to the prerequisite courses on the main course page would be useful for students who cannot immediately remember if they took a specific class or for students who are considering taking this class in the future and want to complete one of the recommended courses] <Link>
CONTRIBUTION 2 [Added a page going over differences in SteamVR after the release of SteamVR 2.0, and the new technical limitations that this added to developer. Recommended page for developers to read before working with SteamVR in Unity] <Link>
CONTRIBUTION 3 [Created a new page going over two different websites for finding and downloading 3D models, and basic introductions on using the pages] <Link>
CONTRIBUTION 4 [Created a new page going over the different major 3D file types that a student or a developer using this wiki is likely to run into on the internet, listing pros and cons] <Link>
CONTRIBUTION 5 [Created a new page discussing the two major plug-ins for UnityVR that support a VR application: SteamVR and OpenXR] <Link>
CONTRIBUTION 6 [Modified the page somewhat to mark out areas where the tutorial has changed due to the release of SteamVR 2.0 and provide some context for the changes] <Link>
CONTRIBUTION 7 [Added links to the new pages on SteamVR 2.0 and the comparison with OpenXR] <Link>
CONTRIBUTION 8 [Added a new tutorial for working with custom actions in Unity3D using the OpenXR plugin] <Link>
CONTRIBUTION 9 [Added a tip to the Google Woe's page for dealing with slow loading/saving times and possible crashes] <Link>
CONTRIBUTION 10 [Added a comparison page going over some of the different technologies that I compared for building collaboration into WebVR, particularly with A-Frame] <Link>
CONTRIBUTION 11 [Added a tutorial page on setting up collaborative spaces in WebVR using A-Frame and networked-aframe] <Link>
CONTRIBUTION 12 [Updated information on Aframe City builder that might be useful for future students] <Link>
CONTRIBUTION 13 [Tutorial for using Python in Blender] <Link>
HOURS SUMMARY
Total: 154 hours
HOURS journal
1/26/21 - 2 Hours
After class, researched AirwayVR some more
Followed up on published document
Considered exploring medical training-based VR as a potential project
1/27/21 - 2 Hours
Explored the course website, looked at the weekly schedule to get a better idea of the upcoming assignments and workload
Searched "Project 1" on the website to find past examples
Was specifically interested in Project 1: Visualizing Mars Data in OpenSpace
1/29/23 - 6 Hours
Proposed changes for HW1
~10 minutes changes
Under the Collaborative VR Research page, I wasn't sure exactly how the research papers listed were connected to the class. I think that they're just papers from our classes associated lab, but are students involved with them? This could be made clearer.
Adding links to the prerequisite courses on the main course page would be useful for students who cannot immediately remember if they took a specific class or for students who are considering taking this class in the future and want to complete one of the recommended courses (COMPLETED)
Including some examples of the homework and deliverables, or links to such examples, would be useful for prospective students exploring the course.
~1 hours changes
A sample page of recommended changes that past students have made would have been useful for this assignment, as I was a little uncertain on how to approach these changes.
A staff page introducing the professor and the TA could be helpful for welcoming new students into the course, since I feel that getting a sense of the staff is an important part of the shopping period.
Since some of the technologies, like VTK and VTK.js, are reliant on coding, it could be useful to have a section for relevant coding languages that might be useful for outside observers to explore.
~10 hour changes
It could be helpful to have a video alongside the Quest 2 Practice Tutorial for students who have never worked with VR headsets.
A section of the home page that includes recent news articles about new uses or innovations involving VR might help interest prospective students in VR as a technology.
I think a page where non-student visitors can make suggested contributions and see them pinned to a page could be useful at both allowing for interaction with outside users and getting a sense of what outside users are interested in.
2/1/23 - 4 Hours
Set up Oculus Quest 2
Reviewed three previous projects:
Explored three potential pieces of software
Potential Project Ideas
Using Muscletech as a way to explore personal training systems
Explore molecular data using 4sight
Use Infinite Kitchen as a way to study how people can collaborate on 3D-space projects
Copied over course goals
2/2/23 - 4 Hours
Potential Project Ideas:
Using Musletech as a way to explore personal training systems
clearly identifies deliverable additions to our VR Software Wiki (5)
involves collaboration in VR (3)
involves large scientific data visualization along the lines of the "Scientific Data" wiki page and identifies the specific data type and software that it will use (3)
has a realistic schedule with explicit and measurable milestones at least each week and mostly every class (2)
explicitly evaluates VR software, preferably in comparison to related software (5)
includes an in-class activity, which can be formative (early in the project) or evaluative (later in the project) (5)
has resources available with sufficient documentation (3)
Explore molecular data using 4sight (or building a comparable Unity plugin)
clearly identifies deliverable additions to our VR Software Wiki (5)
involves collaboration in VR (4)
involves large scientific data visualization along the lines of the "Scientific Data" wiki page and identifies the specific data type and software that it will use (5)
has a realistic schedule with explicit and measurable milestones at least each week and mostly every class (3)
explicitly evaluates VR software, preferably in comparison to related software (4)
includes an in-class activity, which can be formative (early in the project) or evaluative (later in the project) (4)
has resources available with sufficient documentation (5)
Use Infinite Kitchen as a way to study how people can collaborate on 3D-space projects (or build a comparable plugin, maybe creating a museum or biolab to display scientific data, allow researchers to create such display spaces) (planetary exploration)
clearly identifies deliverable additions to our VR Software Wiki (4)
involves collaboration in VR (5)
involves large scientific data visualization along the lines of the "Scientific Data" wiki page and identifies the specific data type and software that it will use (2)
has a realistic schedule with explicit and measurable milestones at least each week and mostly every class (3)
explicitly evaluates VR software, preferably in comparison to related software (4)
includes an in-class activity, which can be formative (early in the project) or evaluative (later in the project) (4)
has resources available with sufficient documentation (3)
Google Earth VR:
Didn't get to in class, will finish at home
2/5/23 - 2 Hours
Google Earth VR:
Worked on it, setting up SteamVR
Ran into technical problems, the main one being a serious lag issue, I tried restarting the headset and my laptop a few times
Ultimately got a headache from the lag, put a question on Slack
2/6/23 - 5 Hours
Google Earth VR:
Finished in the CIT
Made relevant edits to journal
Installed DinoVR
Project Ideas
VRWorkout
Alternative to MusleTech, this is open-source and should be easier to work with
Three things to do
Install VRWorkout and test it out
Research scientific studies on exercises and how those apply to different muscle groups
Build out a VR human model to represent the results and impact of training using that research (probably with Unity)
One in-class activity
Test out VRWorkout in class before building (the app is free)
Test out my project at the end
Potential deliverables
The new VR system for exploring muscle growth after a workout
Screen captures and recordings from using the system
Any relevant and centralized research on different exercises and workout results
Tracing the lifecycle of a star
Three things to do
Research the lifecycle of stars, the different layers they have, and the effect this has on solar systems
Build out a model of star and a possibly a surrounding solar system in a system that can be ported into VR, probably Unity
Let this system be altered in its passage through time to see the star grow or shrink, consuming the solar system around it
One in-class activity
Try out my solar system/sun model once it is completed
Potential deliverables
The solar system/sun model, including the changes at different points in the life cycle and the different layers of the star
Relevant screen captures and recordings
Depicting weather changes in one location due to climate change
Three things to do
Research predicted climate change patterns (probably building off of research like this)
Recreate (based on how much time it takes) either one location or several (maybe just Providence) with weather and scenery today
Offer a competing simulation of that same location after predicted climate change (make use of open-source software for weather and geography, such as for rain, being underwater, and different possible landscapes or levels of greenery)
One in-class activity
Discuss climate change and how VR can be used to make this problem more visible and apparent to users
Explore the contrast between the present world and the projected future world
Potential deliverables
Any relevant research on climate environments that can be written out
Possibly demonstrated to non-students and then interview students afterwards to see how/whether this has changed how they think about climate change
The VR model itself
Exploring explosions in VR
Three things to do
Explore a possible source of fluid dynamics simulation in VR, preferably open source (such as this) to portray physics
Research how explosions of different types affect their environments and how this can be simulated with particle motion to represent fluids of different densitites (such as air or water)
Build a simulation out in VR (probably in Unity) that allows for explosions of different types to be experimented with to see how they affect their fluid environments
One in-class activity
Explore this simulation once I've built it out
Potential deliverables
Any screenshots or recordings
The explosion simulator itself
The environment for particles that can be moved by dynamic forces should also be usable by future students
Software Evaluation Metrics
Speed
Code readability
User accessibility
2/7/23 - 1 Hour
Worked on project outline
2/8/23 - 2 Hours
Finished DinoVR
Finished project outline for class
VRWorkout
Three things to do
Install VRWorkout and test it out
Research scientific studies on exercises and how those apply to different muscle groups
Build out a VR human model to represent the results and impact of training using that research (probably with Unity)
One in-class activity
Test out VRWorkout in class before building (the app is free)
Test out my project at the end
Potential deliverables
The new VR system for exploring muscle growth after a workout
Screen captures and recordings from using the system
Any relevant and centralized research on different exercises and workout results
2/14
Install VRWorkout, test it out on my own
Pull the codebase
2/16
Figure out exporting workout data from VRWorkout
2/23
Build a human model in Unity that is connected to controllers, and a mirrored VR room
2/28
Modify the model so that muscle groups can be represented on it
3/02
Put data together linking different exercises to an emphasized section of the VR human model
3/07
Build a pipeline from exported VRWorkout data to a VR representation of the results in the VR human model
3/09
Present the final result in class (might move this around if the ending is too crowded)
2/12/23 - 4 Hours
Wrote my project 1 presentation
Tried to respond to all low scores in teacher/TA feedback
Got started on the VR Wiki's Unity tutorial
Researched human body meshes that would be compatible with Unity, found a possible candidate
2/15/23 - 2 Hours
Successfully installed VRWorkout
Tested using Quest 2
Set up online account, figured out how to export workout data in a readable format from the online site
2/17/23 - 4 Hours
Pulled the sample human models
Ran into an issue with not being able to easily divide them into muscle groups in Unity
Ended up installing Blender and re-introducing myself to the software
Opened the .obj models in Blender, highlighted and separated out specific groups for the different body parts
2/18/23 - 4 Hours
Finished breaking up the human body model, output the result as a .fbx file
Pulled the new file into Unity, with clearer subdivisions
Finished the Unity tutorial on my machine, began altering the code to apply to my human body model
Got started on outlining a possible wiki contribution going over different file types that might be commonly used in both Blender and Unity (probably, .obj, .fbx, and .stl)
2/19/23 - 6 Hours
Tried to use UnityVR with SteamVR, per the wiki recommended setup
Ran into problems with changes to UnityVR2.0 and SteamVR making it difficult to implement the wiki setup
Need to have the headset connected to SteamVR, which comes with hardware requirements
Tried with both my Mac and my PC, both did not have the required GPU capacity
Potential ideas for wiki modificaiton on changes from UnityVR to UnityVR2.0
2/20/23 - 1 Hours
Did HW, prepared for class
Continued trying to make Paperspace work
2/22/23 - 1 Hours
Did HW, prepared for class
Looked through the wiki again for information on Unity
2/25/23 - 6 Hours
Tried again to figure out how to use UnityVR2.0 and SteamVR
Tried to use modelled headset/controllers instead of real
Tried to connect the headset to the local machine using wireless link
Bought a new USB-to-USB-C cable to create a physical link between machine and headset
Tried to set up work over Paperspace
Ended up unable to figure out how to make this work efficiently
Would have had to install UnityHub on Paperspace, transfer existing work from local machine to virtual machine, keep Virtual Desktop connection open, and ultimately unable to create necessary SteamVR link between headset and Paperspace machine
Ended up reaching out to Jakobi and Lexi for help
Decided to give up on the SteamVR approach and focus on OpenXR
2/26/23 - 2 Hours
Finished Thursday assignment
Researched documentation on working with OpenXR in Unity
Possible wiki contribution
2/27/23 - 1 Hour
Prepared for class and in-class assignment
3/1/23 - 1 Hour
Prepared for class and in-class assignment
3/4/23 - 5 Hours
Worked on implementing a working scene in UnityVR with the human body, the hands with ray interactions, and separate body parts
Several points that made this take a while (hence the long hours logged over the weekend and Monday):
Unable to test changes in code in Unity due to being unable to link Quest to machine
Had to make changes in the code without being able to see the immediate results
Unable to debug without a soft reload in Unity
Had to build a .apk package, move to SideQuest, and move to the Quest every time I wanted to test the scene
3/5/23 - 9 Hours
Focused on getting the right interactions between the controllers, ray extensions, and body model parts
Lack of existing Unity/OpenXR abstract support for non-grabbable interactions with objects
Lots of time spent debugging
Ended up constructing a new set of actions and Unity/OpenXR components in C#, attaching them to the controller objects in Unity
Ultimately able to create a hovering action, with objects responding as the controller points at them
3/6/23 - 11 Hours
Continuous from the last entry (long night)
Focused primarily on getting trigger actions to work properly
Tried to use Unity's built-in trigger functions, OpenXR's built-in trigger functions, unable to get either to work with my code for ray interactions
Had to turn off existing trigger functions and override trigger functions in the code, creating a new response for triggers
Unable to log responses to console to debug when trigger functions and ray interactions did not work, ended up learning how to use text objects in Unity, then just putting a text object in the middle of the scene to post debug comments towards to get printed statements on variable values to debug with
Finally got the scene to respond to both ray hovering and clicking satisfactorily
Removed the text objects
Ate breakfast, went to sleep
Worked on the in-class activity
Got feedback from Jakobi, made edits
Prepared for the next in-class activities
3/8/23 - 2 hours
Made some last-minute changes to the in-class activity based on Jakobi's feedback
Created and linked the in-class activity page
Prepared for other in-class activities
3/9/2023 - 1 hour
Collected some data from the in-class activity, summarized below:
For comparisons between 2D and 3D
Somewhat difficult to directly compare 2D and 3D visualization at this stage
However:
Broad agreement among students that it was easier to visualize active muscle sites using the 3D visualization
Broad agreement that the model benefited from direct, tangible feedback
Users suggested several improvement:
Automated muscle coloring and direct connection with numerical data (most popular)
2D chart alongside 3D model
More detail and more muscle groups
3/11/2023 - 3 hours
Worked on creating pages for finding and understanding 3D file types online
Two pages as main outputs:
3/12/2023 - 2 hours
Did a deeper dive into differences between SteamVR and SteamVR 2.0
Found existing resources on the Wiki related to this change
Got started on wiki pages related to explaining the technical requirements of SteamVR 2.0 and the difference between SteamVR and OpenXR as Unity plugins
3/13/2023 - 2 hours
Finished the following wiki pages on SteamVR and OpenXR
Unity3D (minor edits)
Unity Tutorial #4.1 (minor edits)
Prepared for Tuesday's in-class activities
3/14/2023 - 5 hours
Wrote a tutorial for creating custom functions in Unity using the OpenXR plugin, with a broader focus on introducing the user to OpenXR, installing the plugin and related plugins into a Unity project, and covering some of the basic concepts of OpenXR
Made my presentation
3/16/2023 - 0.5 hours
Added a small contribution to Google Sites Woes to work more safely with slow loading/saving
3/20/2023 - 5 hours
Read the Seven Scenarios paper
Started working on an idea for the second/final project
Researching famous historical examples of data visualization
Found John Snow's 1854 chart of cholera outbreaks in London
Create two possible 3D VR visualizations:
A 3D color map that can be viewed in VR, with heights, to help users see the relevant infection data
A 3D map of the area to travel through and see how diseases would have looked
Could even be used to show how 3D, ground-level situations are harder to understand
Researched frameworks for WebVR
Already some wiki pages on WebVR, hoping to expand with some more frameworks
Done some research on multiplayer and collaboration
My source uses deepstream.io, there is currently a tutorial for collaboration in WebVR using socket.io
Possibly developing a comparison between the technologies
Applying Seven Scenarios
Scenarios for understanding data analysis
Understanding environments and work practices (UWP)
Description: To derive design advice through developing a better understanding of the work, analysis, or information processing practices by a given group of people with or without software use.
Evaluating visual data analysis and reasoning (VDAR)
Description: To assess how an information visualization tool supports analysis and reasoning about data and helps to derive relevant knowledge in a given domain.
Evaluating communication through visualization (CTV)
Description: To assess the communicative value of a visualization or visual representation in regards to goals such as teaching/learning, idea presentation, or casual use.
Evaluating collaborative data analysis (CDA)
Description: To understand to what extent an information visualization tool supports collaborative data analysis by groups of people.
Scenarios for understanding visualizations
Evaluating user performance (UP)
Description: To objectively measure how specific features affect the performance of people with a system.
Evaluating user experience (UE)
Description: To elicit subjective feedback and opinions on a visualization tool.
Evaluating visualization algorithms (VA)
Description: To capture and measure characteristics of a visualization algorithm.
Relevant suggestions and ideas
From UWP:
What kinds of visualizations are currently in use? How do they help to solve current tasks?
There are existing 2D visualizations that help people determine the origin points of diseases and epidemics
From VDAR:
Decision making? How does it support the communication and application of analysis results?
These tools might make it easier to convey either information about the spread of diseases or it might make it easier to understand how certain situations can be difficult to understand on a ground level
From CTV:
Do people learn better and/or faster using the visualization tool?
Could show if learning is faster or slower
Can useful information be extracted from a casual information visualization?
Could be useful to get survey info on this
From CDA
Does the tool support effective and efficient collaborative data analysis?
With multiplayer, hopefully
How is the collaborative visualization system used?
Ideally, people can just enter multiplayer at the same time
From UP
How does one visualization or interaction technique compare to another as measured by human performance?
We'll find out by comparing the results
From UE
Are there limitations of the current system which would hinder its adoption?
I am hoping that the use of WebVR will make it more accessible than it would be otherwise
From VA
Not relevant, as far as I can tell, since this isn't evaluating an algorithm
3/22/2023 - 4 hours
Project 2 Presentation
Evaluation
Using VR to revisualize cholera deaths in 1854
clearly identifies deliverable additions to our VR Software Wiki (5)
involves collaboration in VR (5)
involves large scientific data visualization along the lines of the "Scientific Data" wiki page and identifies the specific data type and software that it will use (3)
has a realistic schedule with explicit and measurable milestones at least each week and mostly every class (3)
explicitly evaluates VR software, preferably in comparison to related software (5)
includes an in-class activity, which can be formative (early in the project) or evaluative (later in the project) (5)
has resources available with sufficient documentation (4)
4/3/2023 - 2 hours
Read up on Seven Scenarios, thought up 3 questions
Worked on project
Focused on getting scientific epidemic data to show up clearly on the John Snow map as an overlay
Used Jupyter Notebook
Went through several possible data sources:
learn.arcgis.com/en/projects/map-a-historic-cholera-outbreak/ (data unclear)
www.nationalgeographic.org/activity/mapping-london-epidemic/ (data too difficult to parse)
www.randomservices.org/random/data/Snow.html (data too difficult to map)
http://www.ncgia.ucsb.edu/pubs/snow/snow.html (data too old, links broken)
Finally settled on:
4/7/2023 - 5 hours
Focused on building the heat map using the data from Monday in Blender with Python
Learned about using Python in Blender
Laid out outline for Wiki contribution on how to use Python in Blender, hopefully make this in the future
Applied to John Snow cholera epidemic data
Created output .glb file of the heat map using John Snow data
Got started with A-Frame/Glitch, placed the heat map model in A-Frame hosted on Glitch
4/10/2023 - 5 hours
Worked on trying to get Deepstream.io to work to enable multiplayer in A-Frame
Used tutorial from before
Ran into difficulties
Hosted on deepstreamhub.com/
Tried to set up an account
Unable to acces dashboard
Looked for support online, unable to find
Ended up emailing their customer support, waiting for a response
EDIT (4/28): Never got a response
Prep for Lexi's activity
4/11/2023 - 1 hour
Worked on Project 2 Update presentation
4/12/2023 - 6 hours
Made a last-ditch effort to get Deepstream.io to work
Ultimately abandoned due to impossibility of using their servers
Looked into Jakobi's tutorial on collaboration in WebVR
Experimented with Socket.io
Researched using my laptop as an Internet server
Found some instructions
Found lots of advice against
Found some other tutorials (here and here)
All used Deepstream
Seems the website went down recently, as several Glitch projects used Deepstream but were no longer running
Finally found a non-Deepstream, non-hosting possibility in networked-aframe
Finalized and sent Project 2 presentation
4/16/2023 - 4 hours
Got started with networked-aframe on Glitch
Very happy with the results, seems to successfully host a server on Glitch itself
Initially tested with just laptop browser tabs, due to working off-campus without headset
Remixed from sample Glitch library
Explored the code, focusing on JavaScript code running the server and and A-Frame code for objects, particularly user objects.
4/17/2023 - 2 hours
Continued working with networked-aframe on Glitch
Continued remixing networked-aframe project on Glitch
Focused on integrating previous work with heat map into networked-aframe project
Experimented with different bodies and movements for the user body, still using keyboard controls
Prep for Camilo's activity
4/18/2023 - 4 hours
Continued working with networked-aframe on Glitch
Started testing with headset controls
Found that keyboard controls for A-Frame are completely incompatible with headset controls
Spent time rewriting object code to enable movement in headset without Glitch crashing or freezing
Jakobi helped me test the Glitch site so that I could make sure that the system worked on different machines
Redid the heat maps
Went back to Blender/Python and redid to include a heat map with no rising markers, a heat map with only rising markers for deaths per location, a heat map with only rising markers for contagion locations (wells), and the original heat map
Integrated with Glitch app
Prep for Dave's activity
4/24/2023 - 2 hours
Prep for Yifei's activity
Tutorials for Horizon Apps
Prep for Melvin's activity
Setup of Immersed
4/26/2023 - 0.5 hours
Prep for Mohammed's activity
Prep for Liza's activity
Prep for Yuanbo's activity
4/28/2023 - 1 hour
Worked on a page for comparing WebVR collaborative technologies
4/29/2023 - 3 hours
Started a page as a tutorial for working with networked-aframe
Mainly focused on recreating the project through Glitch
Looked through feedback on the in-class activity and saw that users had been hoping for interactive and visible avatars
Tried to implement that, ran into road blocks due to issues with getting the "position" of the headset
Unable to find code solution to the problem
4/30/2023 - 2 hours
Continued working on trying to get the avatar to move with the model
Tried switching to an avatar connected to the hand controllers instead of the headset
Did not work
Tried switching to a teleportation method, with the location of the avatar manually updated with the landing position of the headset
Did not work
Putting project on hold for now, unable to make headway
5/3/2023 - 3 hours
Created a tutorial for working with Python in Blender
Created and sent my final project presentation