Lexi's Journal
before after
---- ----
2| 4 | Goal 1: articulate VR visualization software tool goals, requirements, and capabilities
1| 3 | Goal 2: construct meaningful evaluation strategies for software libraries, frameworks, and applications; strategies include surveys, interviews, comparative use, case studies, and web research
1| 4 | Goal 3: execute tool evaluation strategies
1| 4 | Goal 4: build visualization software packages
2| 3 | Goal 5: comparatively analyze software tools based on evaluation
2| 4 | Goal 6: be familiar with a number of VR software tools and hardware
3| 5 | Goal 7: think critically about software
3| 5 | Goal 8: communicate ideas more clearly
2| 5 | Goal 9: create an experience in the context of data!
HOURS SUMMARY
Total: 65.5 hours
HOURS journal
1/26/21 - 3 Hours
Example entry.
Be sure to always include the amount of hours you've worked.
Descriptions of what you did.
Feel free to add pictures as well!
1/27/23 - 1.5 Hours
Set up journal page, joined Slack, researched Kenny Gruchalla and NREL, explored the course page and brainstormed ideas.
Interested in creating a mod for an existing video game (“ To what extent can VR games be used for collaboration and scientific visualization?”)
1/30/23- 6 Hours
Nine proposed changes to the wiki (1.5 hr):
Quick changes (~10 minute)
The documentation on Unreal Engine is missing some information on compatibility with C++ code.
Introduction to ARCore in Unity and Introduction to ARCore using Sceneform links are broken/out of date. [COMPLETE]
Why is the TA page public? Can we put that somewhere else?
Longer changes (1 hour)
Documentation on Lumberyard is very incomplete. The whole VR Development Software page could use a combing for incomplete and missing info, actually.
Add links to the VR Development Software page to helpful beginner tutorials on said software at the end of each blurb. As-is, it leaves off abruptly after installation. Figure out how this interacts with the tutorials page.
Extend/update and pretty the comparison table at the bottom of this page.
Longest changes (10 hours)
Wiki is a bit intimidating for someone who has never worked with AR/VR and thus does not know the terminology. On hub pages, like the one for "VR Modeling Software," short blurbs on what exactly the type of software is and should be able to accomplish should be included, hopefully with fun pictures of it in action.
Related to the above--Many links to resources are labelled simply with the name of the software/hardware/etc. so a user without outside knowledge has to click into every single link in order to find what they're looking for. A blurb like the one for Google Sketchup here would be very helpful on any page that acts as a hub for links
Not sure how long this would take (hopefully not 10 hours!), but search bar should really bring up possible matching results before actually hitting enter so it doesn't feel like you're typing blindly. It would be cool if you could also search software by what you want to do, so blurbs would be helpful in searching (related to above).
Research (4.5 hr)
Read paper on CoVar, the VR Remote Collaboration tool
Researched Sandbox VR (Would it be possible to extend this to support remote capability?)
Possible with TESLASuit? (Expensive!)
Researched AR/VR Training for emergency situations (STRIVR)
Read some Vogue business articles on the rise of AR Fashion. Considered how this could be used to analyze trends and experiment with difficult or expensive clothing manufacturing before creation of a physical object.
"Gamers succeed where scientists fail," by Leila Grey -- Human intuition's benefits over automated methods
Youtube Videos
Modbox AR for MagicLeap
2/2/23 - 2.5 Hours
Read about and brainstormed previous projects
Setup for Quest 2
Project Ideas
Coral Reef Preservation
Pollution datasets from data.world overlayed on google earth so you can see air quality
Multiplayer google earth
multiplayer 3D art gallery/fashion museum/famous building/art pieces
3D graph data
visualize ocean depth/ocean flow patterns
2/6/23 - 4 hours
Installed DinoVR (technical difficulties, solved now)
Reviewed wiki / web to solidify 3-5 project ideas
3D graph of connections between different twitter political figures, using natural language processing to display the tone of interactions through color coding. Can be displayed, discussed, and viewed collaboratively.
TODO: Use the Twitter API to render interactive 3D graphs, use natural language processing to analyze tonal content of tweets sent between users, implement multi-user functionality
Activity: Explore and discuss what we can learn from this "3D political landscape" in VR. Users can point to specific parts of the graph.
Guide on how to render 3D graphs in VR
VR data tool to assist artists in developing skills through displaying 3D models of famous sculptures, figures, and objects, where they can draw on a virtual canvas and share their work with others in the "room."
TODO: Render in a set of loadable 3D sculptures to the space, implement "drawing" capability either through Quill or own creation (can be primitive), preserve canvas data and implement multiplayer.
Activity: Drawing objects in VR from different angles on a "2D" surface.
Guide on building off of Quill (if applicable), if not, guide on spacial user input, lighting simulation, research benefits of VR as compared to sketching from 2D reference
Topographical Data VR Visualization for mountains, so climbers can safely simulate and chart new routes. Should be able to travel with others, and draw potential routes on the 3d map.
TODO: Use topographical data (probably point-cloud or polygonal) to create an accurate visualization of the ground, implement user movement, setting of landmarks/path, implement multi-user functionality
Activity: Mark a potentially viable trail up a portion of a mountain
Guide on user movement in VR, user placement of objects in VR
Ocean floor topography in VR to examine tectonic plates through the land formation.
TODO: Use topographical data (probably point-cloud or polygonal) to create an accurate visualization of the ocean floor (water flow patterns would be cool here), implement user movement, setting of landmarks/path, implement multi-user functionality
Activity: Find an odd land form on the ocean floor and discuss what could have created it.
Guide on user movement in VR, user placement of objects in VR
Google Earth VR
From left to right, my neighborhood in Belfast, Maine, my freshman year dorm (Andrews), and where I went to high school. Unfortunately, this final location did not have street view available.
Video: https://drive.google.com/file/d/1ACDkT2MAXuOYNF92qIoCeCT4eEQQcoPV/view?usp=sharing
Notes: My video could only record for a limited amount of time, so I had to teleport around a bit.
Google Earth Web
From left to right, my neighborhood in Belfast, Maine, my freshman year dorm (Andrews), and where I went to high school.
Video: https://drive.google.com/file/d/1Zx4eWk46t6FsqnxnhRXgmk0rQmubY4Xd/view?usp=sharing
Notes: Navigation over long distances was much easier for me on the web, as well as I found being able to place down markers very helpful, though this might come down to being a skill issue. (Much) less immersive, but I felt more able to be precise.
2/8/23 - 2 Hours
Created a timeline and plan for the drawing development VR project here
Unfortunately did not have enough time to nail down exactly what tools I'll need, which means I have no idea what deliverables will be
2/12/23-2/13/23 - 5 Hours
Research on tools for the project
Read lots of tutorials
Options on how to use Unity on Mac OSX, Android studio, APK files and sideloading on Quest 2
Consulted other devs on pros and cons
Decided on avoiding a VM setup
Setting up "whiteboards" in unity VR tutorial
Development Environment setup
Revised Project Plan/Schedule (Found here)
Presentation creation
2/15/23-2/16/23 - 4.5 Hours
Awful awful time trying to get developer mode to work on Quest 2, but finally got the app to compile and load (nothing inside yet). Now realizing I should have built in time to get hand motions and button controls working too. Fundamental stuff missing.
Next Steps:
Load in objects
HANDS
MOVEMENT (Did not factor these things properly into timeline)
Canvas input to be pushed back -- must get these done before.
Started a stream-of-consciousness journal about this project here, embedded with resources I use, which I will later turn into a proper deliverable.
Deliverable Idea: Guide on how to set up Android Developer mode, because it is NOT clear on the website :(
Starting to learn Unity!! Messing around
2/19/23 - 3 Hours
Followed tutorial here on coding whiteboard scripts in Unity and set up a very basic scene, but got stuck on a bug where Unity was not recognizing names and instance variables
Later resolved: had to change Whiteboard to MyWhiteboard and add some import statements for enumerable (Thank you Ayman!)
2/19/23 - 2.5 Hours
Debugging and finishing the whiteboard scripts, tested them (works!)
Pen tool creates lines between frames for continuous movement
Can change line weight
Theoretically can change color, but only from developer end for now
Setting up XR Interaction Toolkit for VR user input compatability (unfinished)
New challenges: How to clear whiteboard while scene still running?
Maybe should make an eraser tool with similar mechanics, except very heavy line weight and color of background of the board...
2/22/23 - 0.5 Hours
Installation of Paraview and Ben's Volume Renderer
2/25/23 - 5.5 Hours
Loaded in 3D statue GLTF model
Set textures, created podium
Rebuilt and ran app
Resources:
Building Unity to Oculus Guide: https://developer.oculus.com/documentation/unity/unity-build/
Model library: https://sketchfab.com/3d-models/figure-of-a-dancer-63212acff8474095ba2626d9366df852
Importing custom assets (GLTF Models) to Unity: https://www.youtube.com/watch?v=FbwZAyzo6Kc
General Oculus mac dev guide with some missing info: https://neerajjaiswal.com/how-to-setup-for-quest-2-development-on-mac-using-unity-without-bootcamp
Video Series on VR Development in Unity for beginners with a focus on gaming: https://www.youtube.com/watch?v=gGYtahQjmWQ
Decided to use Action-Based XR Controllers, since tutorial is out of date
This was great for getting the camera to track with the headset, understand floor level and dynamically adjust, and set up initial user input using the XRI Toolkit
HANDS SETUP:
This didn't work. The hands do not track. Debugging, but no luck so far.
^FIXED by adding Tracked Pose Driver component with appropriate settings to each controller. Animations still do not work.
TODO:
Get controller animations working
Implement teleportation
Make marker grabable and test drawing with controller input
Resource: https://www.youtube.com/watch?v=QiKzU2Z0KiI
Scale everything up for greater detail
2/26/23 - 7 Hours
Continued from above
Resources:
Hand animation: https://fistfullofshrimp.com/unity-vr-inputs-and-hand-animation/
Troubleshot Unity Project Already Open error (sol'n here: https://hackmd.io/@fdengr/BJajQRf2_)
Discovery of the OVR Grabber Component
Remade left and right controllers from scratch --now animate but do not track and super offset from actual hand position
Grabbing is impossible.
2/27/23 - 4 Hours
Unity randomly deleted some of my progress :|
Tutorials that don't work:
All of the previously mentioned
SUCCESSFULLY DEBUGGED GRABBING! (Unity docs are trash, tutorials are all missing info. Got this done out of sheer luck and time.)
The issue: In Unity, 3D objects come with a collidable component and a rigidbody componant. If you are working with a parent object to child objects, you MUST remove these from all children, and add a rigidbody and collidable componant to the empty parent class gameobject
ALSO, on your controllers, you must have sphere collider component attached, and set to a low size like 0.05
You must use XR Direct Interactor Component on your controllers, NOT Grab Interactor, as many tutorials will instruct you.
3/1/23 - 5 Hours
Locomotion added, rotation added
9 more drawing tools added
Extensive testing (drawing in app)
Hands start to hurt, should adjust controls
Creation of in-class activity handout, download, and google form (Link to handout here)
TODO: Fix game graphics quality. Is abysmal
3/7/23 - 2.5 Hours
Preparation for in class activities
Created Troubleshooting Unity wiki page
Created Online Resources Link Hub wiki page
TODO: Create "Understanding the Unity UI" page, maybe, create "Findings" page, test existing VR drawing apps, create "Journey" page, VR in Art.. do one of these
3/10/23 - 3 Hours
Researched & downloaded Gesture VR
Started on VR in art article deliverable
Tried to add eraser to app, but script does not play nice for some reason --seems to color pixels white regardless of the shadow that should be on it, making it look like a highlighter instead of an eraser
GestureVR.mp4
3/11/23 - 7 Hours
Played with Gesture VR
Very impressed overall, will use again. Myriad of tools like pressure sensitivity and variable size charcoal and eraser. Can adjust lighting-- GREAT! No color or different medium capability. I would say relatively easy to adjust to, though pressure is difficult to measure and i was often dissatisfied. Arms and head get fatigued more quickly, but the experience of depth is realistic. Some issues with measuring form as you can see, due to not being able to measure length and angles by marking pencil with thumb while holding straight out. Eraser unrealistically effective, and no smudging which is a shame. This is basically what I wanted to build, but their pose models are somewhat limited.
Limitations of even very advanced VR drawing: Without a physical surface, hands are shaky and pressure is unreliable. This means detail is overall a no-go. Large sweeping strokes OK. This means unfortunately VR is probably not best for scientific drawings despite its potential in getting rare specifies documented.
Many complaints on my in class activity alleviated here-- I found it very engaging, so I assume the results of the activity were somewhat biased becuase people found it hard to feel engaged with an app that wasn't behaving smoothly/obeying what they wanted.
Wrote VR in Visual Art
A joint reflection on VRStudio, research piece, and comparison/exploration of Gesture VR
3/12/23 - 1 Hour
Tried to make canvas movable, realize that it may require a complete overhaul of the program because adding a rigidbody to the plane representing the canvas causes the whiteboard script to break. May have to figure out how to draw on an actual shape like a thin rectangle rather than a plane, though I don't like this...
3/15/23 - 6 Hours
Another attempt to make canvas movable, this time making the whiteboard a child of a movable empty gameobject. Unfortunatly, while this moves the canvas, the rigidbody on the parent still causes the whiteboard script to break
Graphics: Researched antialiasing added + messed around with quality settings
looking a little jittery - though maybe just more noticable in higher quality
Woah, goes insane when I put more than 2 sculptures in at a time, to the point where I couldn't even screen record what was going on because it got so jittery I couldn't click record
Huge improvements in Graphics
Implemented custom skybox
Implemented spotlight lighting (smoother, better lighting)
Enabled anti-aliasing 4x Multi
Unbelievable improvement to line quality on canvas, though control is still an issue. Lines are clean and vary thickness to an extent based on angle
Set Android build quality to Very High
Realized "Figure of a dancer" model is buggy after much pain and suffering trying to figure out why the shadows were patchy and tested other formats of 3D models, inc. Spartacus, who I have switched her out for now
Created recordings, photos, and final project presentation
VRSTUDIO DEMO FIN.mp4
3/20/23 - 2 Hours
Read Developing Virtual Reality Visualizations for (DINOVR) paper
Looked into EngageVR
Read Seven Scenarios Paper
3/23/23 - 2 Hours