before after (goal)

----  ----

    2| 4 | Goal 1: articulate VR visualization software tool goals, requirements, and capabilities

    1| 3 | Goal 2: construct meaningful evaluation strategies for software libraries, frameworks, and applications; strategies include surveys, interviews, comparative use, case studies, and web research

    1| 4 | Goal 3: execute tool evaluation strategies

    1| 4 | Goal 4: build visualization software packages

    2| 3 | Goal 5: comparatively analyze software tools based on evaluation

    2| 4 | Goal 6: be familiar with a number of VR software tools and hardware

    3| 5 | Goal 7: think critically about software

    3| 5 | Goal 8: communicate ideas more clearly

    2| 5 | Goal 9: create an experience in the context of data!

before after (end-of-semester self-evaluation)

----  ----

    2| 4 | Goal 1: articulate VR visualization software tool goals, requirements, and capabilities

    1| 3 | Goal 2: construct meaningful evaluation strategies for software libraries, frameworks, and applications; strategies include surveys, interviews, comparative use, case studies, and web research

    1| 3 | Goal 3: execute tool evaluation strategies

    1| 4 | Goal 4: build visualization software packages

    2| 4 | Goal 5: comparatively analyze software tools based on evaluation

    2| 4 | Goal 6: be familiar with a number of VR software tools and hardware

    3| 5 | Goal 7: think critically about software

    3| 4 | Goal 8: communicate ideas more clearly

    2| 5 | Goal 9: create an experience in the context of data!

PROPOSALS, PRESENTATIONS, EVALUATIONS

• Project 1 Proposal HERE (Revised HERE) 

• Presentation for Project 1 Proposal HERE

• In-class Activity HERE (Results HERE) 

• End Presentation for Project 1 HERE (GITHUB LINK)

• Project 2 Proposal HERE (Revised HERE) 

• Presentation for Project 2 Proposal HERE

• End-of-Development Presentation HERE

• Poster HERE

• Flash Talk HERE

• In-class Activity HERE (Results HERE)

• Public Demo HERE

ALL OTHER WIKI CONTRIBUTIONS

CONTRIBUTION 1: Unity Troubleshooting here 

CONTRIBUTION 2: Online Resource Hub (for Videos, Tutorials and Assets) here

CONTRIBUTION 3:  VR in Visual Arts (Demo reflection, buildable project, and expansion) here

CONTRIBUTION 4:  VR Tools for Art and Science (ENGAGE, Gesture, Tilt Brush) here

CONTRIBUTION  5: Debugging C++ Unity Code with Oculus Guide  (for Mac, via Andriod Logcat) here

CONTRIBUTION  6: The Multidisciplinary, Exciting Merits of Scientific Sketching in VR here

HOURS journal

1/26/21 - 3 Hours (Example, do not include in count)

• Example entry.

• Be sure to always include the amount of hours you've worked. 

• Descriptions of what you did.

• Feel free to add pictures as well!

1/27/23 - 1.5 Hours

• Set up journal page, joined Slack, researched Kenny Gruchalla and NREL, explored the course page and brainstormed ideas.

  • Interested in creating a mod for an existing video game (“ To what extent can VR games be used for collaboration and scientific visualization?”)

1/30/23- 6 Hours

• Nine proposed changes to the wiki (1.5 hr):

  • Quick changes (~10 minute)

    • The documentation on Unreal Engine is missing some information on compatibility with C++ code.

    • Introduction to ARCore in Unity and Introduction to ARCore using Sceneform links are broken/out of date. [COMPLETE]

    • Why is the TA page public? Can we put that somewhere else?

  • Longer changes (1 hour)

    • Documentation on Lumberyard is very incomplete. The whole VR Development Software page could use a combing for incomplete and missing info, actually.

    • Add links to the VR Development Software page to helpful beginner tutorials on said software at the end of each blurb. As-is, it leaves off abruptly after installation. Figure out how this interacts with the tutorials page. 

    • Extend/update and pretty the comparison table at the bottom of this page.

  • Longest changes (10 hours)

    • Wiki is a bit intimidating for someone who has never worked with AR/VR and thus does not know the terminology. On hub pages, like the one for "VR Modeling Software," short blurbs on what exactly the type of software is and should be able to accomplish should be included, hopefully with fun pictures of it in action.

    • Related to the above--Many links to resources are labelled simply with the name of the software/hardware/etc. so a user without outside knowledge has to click into every single link in order to find what they're looking for. A blurb like the one for Google Sketchup here would be very helpful on any page that acts as a hub for links

    • Not sure how long this would take (hopefully not 10 hours!), but search bar should really bring up possible matching results before actually hitting enter so it doesn't feel like you're typing blindly. It would be cool if you could also search software by what you want to do, so blurbs would be helpful in searching (related to above). 

• Research (4.5 hr)

  • Read paper on CoVar, the VR Remote Collaboration tool

  • Researched Sandbox VR (Would it be possible to extend this to support remote capability?)

    • Possible with TESLASuit? (Expensive!)

  • Researched AR/VR Training for emergency situations (STRIVR)

  • Read some Vogue business articles on the rise of AR Fashion. Considered how this could be used to analyze trends and experiment with difficult or expensive clothing manufacturing before creation of a physical object. 

  • "Gamers succeed where scientists fail," by Leila Grey -- Human intuition's benefits over automated methods

  • Youtube Videos

    • Modbox AR for MagicLeap

2/2/23 - 2.5 Hours

• Read about and brainstormed previous projects

• Setup for Quest 2

• Project Ideas

  • Coral Reef Preservation 

  • Pollution datasets from data.world overlayed on google earth so you can see air quality 

  • Multiplayer google earth

  • multiplayer 3D art gallery/fashion museum/famous building/art pieces

  • 3D graph data

  • visualize ocean depth/ocean flow patterns 

2/6/23 - 4 hours 

• Installed DinoVR (technical difficulties, solved now)

• Reviewed wiki / web to solidify  3-5 project ideas

• 3D graph of connections between different twitter political figures, using natural language processing to display the tone of interactions through color coding. Can be displayed, discussed, and viewed collaboratively.

  • TODO: Use the Twitter API to render interactive 3D graphs, use natural language processing to analyze tonal content of tweets sent between users, implement multi-user functionality

  • Activity: Explore and discuss what we can learn from this "3D political landscape" in VR. Users can point to specific parts of the graph.

  • Guide on how to render 3D graphs in VR

• VR data tool to assist artists in developing skills through displaying 3D models of famous sculptures, figures, and objects, where they can draw on a virtual canvas and share their work with others in the "room."

  • TODO: Render in a set of loadable 3D sculptures to the space, implement "drawing" capability either through Quill or own creation (can be primitive), preserve canvas data and implement multiplayer.

  • Activity: Drawing objects in VR from different angles on a "2D" surface.

  • Guide on building off of Quill (if applicable), if not, guide on spacial user input,  lighting simulation, research benefits of VR as compared to sketching from 2D reference 

• Topographical Data VR Visualization for mountains, so climbers can safely simulate and chart new routes. Should be able to travel with others, and draw potential routes on the 3d map.

  • TODO: Use topographical data (probably point-cloud or polygonal) to create an accurate visualization of the ground, implement user movement, setting of landmarks/path, implement multi-user functionality

  • Activity: Mark a potentially viable trail up a portion of a mountain

  • Guide on user movement in VR, user placement of objects in VR

• Ocean floor topography in VR to examine tectonic plates through the land formation.

  • TODO: Use topographical data (probably point-cloud or polygonal) to create an accurate visualization of the ocean floor (water flow patterns would be cool here), implement user movement, setting of landmarks/path, implement multi-user functionality

  • Activity:  Find an odd land form on the ocean floor and discuss what could have created it.

  • Guide on user movement in VR, user placement of objects in VR

Google Earth VR

From left to right, my neighborhood in Belfast, Maine, my freshman year dorm (Andrews), and where I went to high school. Unfortunately, this final location did not have street view available.

Video: https://drive.google.com/file/d/1ACDkT2MAXuOYNF92qIoCeCT4eEQQcoPV/view?usp=sharing

Notes: My video could only record for a limited amount of time, so I had to teleport around a bit.

Google Earth Web

From left to right, my neighborhood in Belfast, Maine, my freshman year dorm (Andrews), and where I went to high school. 

Video: https://drive.google.com/file/d/1Zx4eWk46t6FsqnxnhRXgmk0rQmubY4Xd/view?usp=sharing

Notes: Navigation over long distances was much easier for me on the web, as well as I found being able to place down markers very helpful, though this might come down to being a skill issue. (Much) less immersive, but I felt more able to be precise.

2/8/23 - 2 Hours

• Created a timeline and plan for the drawing development VR project here 

• Unfortunately did not have enough time to nail down exactly what tools I'll need, which means I have no idea what deliverables will be

2/12/23-2/13/23 - 5 Hours

• Research on tools for the project

• Read lots of tutorials

  • Options on how to use Unity on Mac OSX, Android studio, APK files and sideloading on Quest 2

  • Consulted other devs on pros and cons

  • Decided on avoiding a VM setup 

  • Setting up "whiteboards" in unity VR tutorial 

• Development Environment setup

• Revised Project Plan/Schedule (Found here)

• Presentation creation

2/15/23-2/16/23 - 4.5 Hours

• Awful awful time trying to get developer mode to work on Quest 2, but finally got the app to compile and load (nothing inside yet). Now realizing I should have built in time to get hand motions and button controls working too. Fundamental stuff missing.

  • Next Steps:

    • Load in objects 

    • HANDS

    • MOVEMENT (Did not factor these things properly into timeline)

    • Canvas input to be pushed back -- must get these done before. 

• Started a stream-of-consciousness journal about this project here, embedded with resources I use, which I will later turn into a proper deliverable. 

• Deliverable Idea: Guide on how to set up Android Developer mode, because it is NOT clear on the website :(

• Starting to learn Unity!! Messing around

2/19/23 - 3 Hours

• Followed tutorial here on coding whiteboard scripts in Unity and set up a very basic scene, but got stuck on a bug where Unity was not recognizing names and instance variables

  • Later resolved: had to change Whiteboard to MyWhiteboard and add some import statements for enumerable (Thank you Ayman!)

2/19/23 - 3 Hours

• Debugging and finishing the whiteboard scripts, tested them (works!)

  • Pen tool creates lines between frames for continuous movement

  • Can change line weight

  • Theoretically can change color, but only from developer end for now

• Setting up XR Interaction Toolkit for VR user input compatability (unfinished)

• New challenges: How to clear whiteboard while scene still running?

  • Maybe should make an eraser tool with similar mechanics, except very heavy line weight and color of background of the board...

2/22/23 - 0.5 Hours

• Installation of Paraview and Ben's Volume Renderer

2/25/23 - 5.5 Hours

• Loaded in 3D statue GLTF model

• Set textures, created podium

• Rebuilt and ran app

• Resources: 

  • Building Unity to Oculus Guide: https://developer.oculus.com/documentation/unity/unity-build/

  • Model library: https://sketchfab.com/3d-models/figure-of-a-dancer-63212acff8474095ba2626d9366df852

  • Importing custom assets (GLTF Models) to Unity: https://www.youtube.com/watch?v=FbwZAyzo6Kc

  • General Oculus mac dev guide with some missing info: https://neerajjaiswal.com/how-to-setup-for-quest-2-development-on-mac-using-unity-without-bootcamp

  • Video Series on VR Development in Unity for beginners with a focus on gaming: https://www.youtube.com/watch?v=gGYtahQjmWQ

    • Decided to use Action-Based XR Controllers, since tutorial is out of date 

    • This was great for getting the camera to track with the headset, understand floor level and dynamically adjust, and set up initial user input using the XRI Toolkit 

  • HANDS SETUP:

    • https://www.youtube.com/watch?v=ijcn-mIJL5s

    • This didn't work. The hands do not track. Debugging, but no luck so far.

    • ^FIXED by adding Tracked Pose Driver component with appropriate settings to each controller. Animations still do not work. 

    • https://www.youtube.com/watch?v=d2gEEQeu1b4

  • TODO:

    • Get controller animations working

    • Implement teleportation

    • Make marker grabable and test drawing with controller input 

      • Resource: https://www.youtube.com/watch?v=QiKzU2Z0KiI

    • Scale everything up for greater detail

2/26/23 - 7 Hours

• Continued from above

• Resources:

  • Hand animation: https://fistfullofshrimp.com/unity-vr-inputs-and-hand-animation/

• Troubleshot Unity Project Already Open error (sol'n here: https://hackmd.io/@fdengr/BJajQRf2_)

• Discovery of the OVR Grabber Component

• Remade left and right controllers from scratch --now animate but do not track and super offset from actual hand position 

• Grabbing is impossible.

2/27/23 - 4 Hours

• Unity randomly deleted some of my progress :|

• Tutorials that don't work:

  • https://www.youtube.com/watch?v=sif7BLMoydI

  • All of the previously mentioned

• SUCCESSFULLY DEBUGGED GRABBING! (Unity docs are trash, tutorials are all missing info. Got this done out of sheer luck and time.)

  • The issue: In Unity, 3D objects come with a collidable component and a rigidbody componant. If you are working with a parent object to child objects, you MUST remove these from all children, and add a rigidbody and collidable componant to the empty parent class gameobject

  • ALSO,  on your controllers, you must have sphere collider component attached, and set to a low size like 0.05

  • You must use XR Direct Interactor Component on your controllers, NOT Grab Interactor, as many tutorials will instruct you. 

3/1/23 - 7 Hours

• Locomotion added, rotation added

• 9 more drawing tools added

• Extensive testing (drawing in app)

  • Hands start to hurt, should adjust controls

• Creation of in-class activity handout, download, and google form (Link to handout here)

• TODO: Fix game graphics quality. Is abysmal

3/7/23 - 2.5 Hours

• Preparation for in class activities 

• Created Troubleshooting Unity wiki page 

• Created Online Resources Link Hub wiki page

• TODO: Create "Understanding the Unity UI" page, maybe, create "Findings" page, test existing VR drawing apps, create "Journey" page, VR in Art.. do one of these 

3/10/23 - 3 Hours

• Researched & downloaded Gesture VR 

• Started on VR in art article deliverable

• Tried to add eraser to app, but script does not play nice for some reason --seems to color pixels white regardless of the shadow that should be on it, making it look like a highlighter instead of an eraser 

3/11/23 - 7 Hours

• Played with Gesture VR

  • Very impressed overall, will use again. Myriad of tools like pressure sensitivity and variable size charcoal and eraser. Can adjust lighting-- GREAT! No color or different medium capability. I would say relatively easy to adjust to, though pressure is difficult to measure and i was often dissatisfied. Arms and head get fatigued more quickly, but the experience of depth is realistic. Some issues with measuring form as you can see, due to not being able to measure length and angles by marking pencil with thumb while holding straight out. Eraser unrealistically effective, and no smudging which is a shame. This is basically what I wanted to build, but their pose models are somewhat limited.

  • Limitations of even very advanced VR drawing: Without a physical surface, hands are shaky and pressure is unreliable. This means detail is overall a no-go. Large sweeping strokes OK. This means unfortunately VR is probably not best for scientific drawings despite its potential in getting rare specifies documented.

  • Many complaints on my in class activity alleviated here-- I found it very engaging, so I assume the results of the activity were somewhat biased becuase people found it hard to feel engaged with an app that wasn't behaving smoothly/obeying what they wanted. 

• Wrote VR in Visual Art

  • A joint reflection on VRStudio, research piece, and comparison/exploration of Gesture VR 

3/12/23 - 1 Hour

• Tried to make canvas movable, realize that it may require a complete overhaul of the program because adding a rigidbody to the plane representing the canvas causes the whiteboard script to break. May have to figure out how to draw on an actual shape like a thin rectangle rather than a plane, though I don't like this...

3/15/23 - 6 Hours

• Another attempt to make canvas movable, this time making the whiteboard a child of a movable empty gameobject. Unfortunatly, while this moves the canvas, the rigidbody on the parent still causes the whiteboard script to break 

• Graphics: Researched antialiasing added + messed around with quality settings

  • looking a little jittery - though maybe just more noticable in higher quality 

  • Woah, goes insane when I put more than 2 sculptures in at a time, to the point where I couldn't even screen record what was going on because it got so jittery I couldn't click record

• Huge improvements in Graphics

  • Implemented custom skybox

  • Implemented spotlight lighting (smoother, better lighting)

  • Enabled anti-aliasing 4x Multi

    • Unbelievable improvement to line quality on canvas, though control is still an issue. Lines are clean and vary thickness to an extent based on angle

  • Set Android build quality to Very High

  • Realized "Figure of a dancer" model is buggy after much pain and suffering trying to figure out why the shadows were patchy and tested other formats of 3D models, inc. Spartacus, who I have switched her out for now

• Created recordings, photos, and final project presentation 

3/20/23 - 2.5 Hours

• Drafted plan for project 2 

• Read Developing Virtual Reality Visualizations  for (DINOVR) paper

• Looked into EngageVR 

• Read Seven Scenarios Paper

3/23/23 - 2.5 Hours

• Revised plan for project 2 

• Project 2 Presentation 

3/23/23 - 1 Hours

• Exploring  EnagageVR/Setup class activity 

  • Engage does not have capability to import meshes without payment :(

• Created handout and google form 

3/23/23 - 2.5 Hours

• In class activity result analysis

• Prep for others' in class activities 

• Project 2 Progress Report Presentation 

3/23/23 - 3 Hours

• Bought and set up Tilt Brush by google, explored capabilities and discovered a few things

  • This software has practically been abandoned by its maker. It's library of 3d models went obsolete and is no longer available, and it does not allow imports of your own models. Tilt Brush is seemingly open source, so many projects have been created off of it, such as OpenBrush and MultiBrush, but these do not seem to have reliable model imports either, though it seemingly is (was?) possible using a PC. Overall not accessible to the average person who wants to use that type of thing casually. Docu

  • No 2D capability :(

4/24/23 - 4 Hours

• More exploration of Tilt Brush/Research how to implement 3d drawing 

  • Open source code for OpenBrush: https://github.com/icosa-gallery/open-brush

• Creation of wiki deliverable on Comparing ENGAGE VR, Gesture VR, and Tiltbrush (optimizing VRStudio for flexibility and effectiveness) (unfinished)

4/25/23 - 4 Hours

• Melvin and Yifei's In Class activity setup 

• Deliverable on optimizing VRStudio based on research of other apps + comparison and review (HERE) 

4/26/23 - 5 Hours

• Adapting Device-Based scripts to be Input Based, in order to implement 3D brush functionality

  • Learning how to code without tutorial guidance

  • Works off this base: Creating Unity VR Drawing App

• Getting heavy into coding now. Not looking promising unless I can find a real person to help offer guidance. While I can get everything to compile, nothing actually shows up when the trigger is pressed, and there seem to be serious issues with the controllers being compatible/recognized.

  • Posted a long post on the unity forum. Hopefully help comes my way!

• Researched implementing save states

  • https://www.youtube.com/watch?v=DCD8Pt1zf4c

4/27/23 - 3 Hours

• Posted a long post on the unity forum. Hopefully help comes my way!

  • No luck. Learning how to debug in c#

4/28/23 - 0.5 Hours

• In-class activity setup 

4/28/23 - 8 Hours

• A painful decision, but I have decided to abandon the base that I was trying to work off of. I was originally trying to work off of this former student's code, but the code is very outdated and works with the abandoned device-based features of the XR Interaction Toolkit. I was hoping to preserve the logic while updating this (quite detailed) code to work on the modern system. After many, many hours, I simply do not have enough experience and debugging prowess in C++ to work with all of this stuff that I do not understand. I think it is smarter at this point to get a more primitive model working, even if I am working from scratch. 

• Learned how to install a debugging console for Unity which is compatible with Mac + Build and Run to Oculus so I can live debug while running the app. Will make a deliverable for this, because information on Unity development/debugging on Mac is sparse. 

4/30/23 - 6 Hours

• Starting coding 3D drawing scripts from scratch, beginning from spheres and then creating a long, connected tube with cylinders based on the rotation + direction of the trigger hand 

  • Pauses tube creation  between when the trigger is pressed 

4/31/23 - 4 Hours

• Added adjustable 3D line width via clicking right joystick 

• Created Debugging C++ for Unity deliverable  for Mac (mentioned earlier)

5/2/23 - 5 Hours

• Tested compatibility with 2D drawing

• Attempt to implement 3D eraser functionality 

  • Somehow deleting my hand upon contact with the eraser???

  • SUCCESS!! It was a bracketing error !! ERASER WORKS! And you can sort of sculpt with it!

• Researched importing models by users at runtime 

  • Zoe possibly. Exciting news: New Sketchfab Plugin for Unity Allows Model Download at Runtime - Sketchfab Community Blog - Sketchfab Community Blog

  • I am going to try to get this to work. I think it is higher priority than multiplayer functionality, and will make a deliverable if i succeed

  • Package Manager -> plus icon + ->Import from Git URL -> https://github.com/Zoe-Immersive/SketchfabCSharp.git

  • also add https://github.com/atteneder/glTFast.git

5/3/23 - 3.5 Hours

• Work with Zoe

  • Sketchfab API Tutorial - YouTube

  • Seems the installation of these packages causes a compile error of the player. I am not sure I am willing to deal with this. It's a small company and very experimental, and I do not think I can debug this with the given docs. 

• Playing around with imports to backend from sketchfab

• Researched Multiplayer

• Created Final Presentation

• Logged hours

5/7/23 - 3 Hours

• Researching the merits of Scientific Sketching in VR, read the following articles, amongst skimming others:

  • http://www.sci.utah.edu/publications/Ush2017c/vr-neuron-tracing-2.pdf

  • https://livrepository.liverpool.ac.uk/2036703/

  • https://www.researchgate.net/publication/366223181_A_Virtual_Reality_Whiteboard_System_for_Remote_Collaboration_Using_Natural_Handwriting

  • https://onlinelibrary.wiley.com/doi/full/10.1002/cae.21935

  • https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8520331/

• Creation of Wiki Deliverable "The Multidisciplinary, Exciting Merits of Scientific Sketching in VR"

5/8-9/23 - 6 Hours

• Lexi's final valiant attempt to implement multiplayer

  • Originally implemented with Alteruna, a new multiplayer plugin as follows: https://www.youtube.com/watch?v=l8TFvqeeC2U, however I ran into the known bug of Unity infinitely loading when launched using the plugin and had to roll back all changes

  • This tutorial must have depreciated, because upon running gave compile errors (in code I had not touched) https://www.youtube.com/watch?v=stJ4SESQwJQ

  • Ended up going back to the basics and using the Unity-supported Netcode for Game Objects (NGO), tutorial here: https://docs-multiplayer.unity3d.com/netcode/current/tutorials/get-started-ngo

    • Better tutorial (Hello world) HERE

    • This may or may not work. I implemented and ran it, but I can't really test it on my own. I am going to blindly assume it works for presentation purposes and will debug it if it doesn't. One thing that is not implemented is for multiple users to see one user's drawings, I think. I am not 100% sure how to approach that, and I need to test that people are actually in the same space/avatars working before I go too deep on it so I avoid wasting my own time. Netcode should support it for future development, since I made sure that expansion was possible.

• Creation of Flash Talk slides/script 

• Creation of Final Project Poster 

5/16/23 - 5 Hours

• Getting ducks in a line :)

• Attempt to fix Github version control (unsuccessful, uploaded relevant files to drive HERE as temp soln)