Loudon Cohen journal

Activity Log (2020)

1/24, reinstalled Paraview, reviewed YURT software and status (2 hours)

1/26, investigated/downloaded UniCAVE and viability--looks very promising (5 hours)

1/28, installed linux vm on local machine and Unity (3 hours)

1/29, investigated potential projects, talked with Brandon about Unicave in the Yurt, read MinVR source code (5 hours)

1/31, Yurt access, investigated Quadifier, reviewed Unicave source code (5 hours)

2/3, vm research (4 hours)

2/5, Setup remote access, installed Unity 2019 on Oscar, fixed errors, helped Ben and Camilo reorganize Yurt room (6 hours)


Project Proposals:

  • Unity v1: Port Unity using Unicave in the Yurt only with Linux vms.

  • Unity v2: Dual boot machines in the Yurt to remove the vm constraint, possibly allowing more options to run in the Yurt including Unity.

    • Both of the above would likely use polygonal model data for testing.

  • Vulkan: Improve performance in the Yurt by creating a Vulkan version of MinVR. This would be tested on point cloud data as datasets such as from the Kellner lab have already had well documented point limits which would hopefully be improved.

Solidified Project ideas:

Unity Porting v1/2:

  • Activities:

    • First try the 2019 version of Unicave to evaluate all the issues. From talking with those who have worked on Unity porting in the past, it is getting stereo to work.

    • If any other element doesn't work, consult with class. If only stereo doesn't work, first try cheap solutions such as Ethan's library fix or Quadifier.

    • If the above don't work, try a minimal version using vms (Docker was suggested)

    • If vms provably don't work, then get a minimal version working on a dual-booted system

    • If either of the above work, facilitate porting to the Yurt

  • Potential deliverables:

    • Vm (Docker Guide)

    • A write-up why [software quad-buffer stereo, vms, dual-booting] does/doesn't work for the Yurt. Essentially, a compiled list of things tried w/explanation

    • A set of guidelines and extra software for working with Unity and then porting to the Yurt

    • A Unity display configuration scene. Potentially with added features such as software calibration

    • A repository with a default Unity configuration that works in the Yurt along with scripts for running it

  • Potential class activity:

    • Guide on porting simple Unity app to the Yurt


Vulkan MinVR:

  • Activities:

    • Dig out old Vulkan code I wrote and test viability on minimal setup. MinVR only loosely depends on OpenGL and swapping out the rendering API should be the easy part

    • Upgrade the stencil for a minimal working version in the Yurt (and ensure the Yurt has proper drivers)

    • Build a similar MinVR app in Vulkan

  • Potential Deliverables:

    • A MinVR Github fork with a Vulkan backend (Using my Vulkan stencil code)

    • At least one Vulkan example working in the Yurt

    • A guide for Vulkan development for the Yurt

  • Potential class activity:

    • Simple Vulkan app tutorial



Unity Porting:

  • Activities (milestones are results):

    • Unity working on CCV (2/11)

    • First try the 2019 version of Unicave to evaluate all the issues. From talking with those who have worked on Unity porting in the past, it is getting stereo to work. (2/13)

    • If any other element doesn't work, consult with class. If only stereo doesn't work, first try cheap solutions such as Ethan's library fix or Quadifier. Report on results. (2/20)

    • If the above don't work, try a minimal version using vms (Docker was suggested) (2/25)

    • If vms provably don't work, then get a minimal version working on a dual-booted system (2/27)

    • If either of the above work, facilitate porting to the Yurt (3/03)

  • Potential deliverables:

    • Vm Guide (for Unity)

    • A write-up why [software quad-buffer stereo, vms, dual-booting] does/doesn't work for the Yurt. Essentially, a compiled list of things tried w/explanation

    • A set of guidelines and extra software for working with Unity and then porting to the Yurt

    • A Unity display configuration scene. Potentially with added features such as software calibration

    • A repository with a default Unity configuration that works in the Yurt along with scripts for running it

  • Potential class activity:

    • Guide on porting simple Unity app to the Yurt

    • VM for Unity tutorial

    • Show off iatk

  • Resources:

  • Data:

    • iatk

    • Polygonal model data

      • Treak cavern


Project Evaluation by Carlos

The proposed project clearly identifies deliverable additions to our VR Software Wiki: 5

The proposed project involves previously unavailable Yurt data visualization functionality: 5

The proposed project involves large data visualization along the lines of the "Data Types" wiki page and identifies the specific data and software that it will use: 3

The proposed project has a realistic schedule with explicit and measurable milestones at least each week and mostly every class: 3

The proposed project includes an in-class activity: 5

The proposed project has resources available with sufficient documentation: 5

Activity Log (2019)

1/27, installed Paraview and investigated software, setup Journal, read last year's wiki (2 hours)

1/28, read Experiments in Immersive Virtual Reality for Scientific Visualization, wrote a wiki page for NVIDIA's VRWorks (2.5 hours)

1/29, reviewed project specs. and old projects, thought about project ideas and investigated relevant s/w (4 hours)

1/30, assisted with VR Lab setup (3 hours)

2/3, installed VRWorks Unity plugin and explored environment (3 hours)

Potential Project Titles:

  • Comprehensive VRWorks Investigation (deep dive into Turing advances, compatibility, etc.)

    • API vs. plugin comparison/evaluation

    • Testing of all application features (Variable Rate Shading, Lens Matched Shading, VR SLI, Single Pass Stereo, Multi-View Rendering, Multi-Res Shading)

    • Project demo incorporating all (toggleable) features

    • Class testing/study for impact of said features

    • Possibly investigate application to the YURT: "VRWorks includes SDKs that accelerate 360 video capture and processing as well as enhance professional VR environments like CAVES, Immersive Displays & Cluster Solutions "

  • Real-time Pathtracing Visualization using RTX (for content creators)

    • Port of RTX-accelerated path tracer to VR

    • Denoising layer such as SVGF (for interactive VR framerate reqs. and image quality)

    • Class activity unknown currently

  • Audial Propagation Visualization

    • Compare Google's Spatial Audio API with VRWorks Audio

    • Evaluation of Binaural playback in VR with above APIs

    • Demo scene for audio "visualization" such as a maze with audio cues (perhaps with a class user study)

    • Could be of interest to the Evolutionary Morphology Lab (Bat hearing)


Pre-project Draft:

  • Working Title: VR Spatial Audio Framework Comparison

  • Equipment: Vive and VR desktop in the VR Lab, software to evaluate

  • Goal: Compare and contrast various VR audio frameworks (VRWorks, Google's Spatial Audio API, Steam Audio)

  • Deliverable(s): audio framework tutorials, software comparison/evaluation results

  • Activities:

    • Go through tutorials for each framework

    • Create tutorials for each framework

    • Design an experiment for objective evaluation

  • Milestones:

    • 2/12 compile all APIs/libraries to be evaluated and install them to ensure s/w compatibility

    • 2/14 project proposal done, use unreal engine with VRWorks

    • 2/21 personally complete tutorials for each framework, figure out data type


2/9, installed Google Resonance and Steam Audio on middle VR machine, attempted using unofficial Unity port of VRWorks--successfully installed, no documentation on how to use (6 hours)

2/10, data type for project research--standard audio formats and some research on environments that would be enhanced with spatial audio (i.e. caves) (2.5 hours)

2/11, pre-project proposal finalization and resource compilation (1 hour)

2/13, installed VRWorks with Unreal plugin (2 hours)


Time running total: 26 hours


Project Proposal:

Title: Spatial VR Audio Comparison

In-class activity: Have everyone import an environment and an audio type into Unity using the best evaluated API (ex. cave geometry and bat audio)

Milestones:

    • 2/21 personally complete tutorials for each framework, figure out data type for experiment later on

    • 2/26 complete tutorial of Steam Audio API on the wiki

    • 2/28 complete tutorial of Resonance Audio API on the wiki

    • 3/5 complete tutorial of VRWorks Audio API on the wiki

    • 3/7 personal evaluation/comparison of each API logged

    • 3/12 experiment design to evaluate each library objectively

    • 3/14 create portable testing environment in Unity

      • (Pre-idea): VR Maze with and without audio cues guiding to the exit. Test time for users to find the exit. Ask users for their evaluation of each API implementation

    • 3/19 port environment to Unreal Engine to ensure it works with VRWorks audio

    • 3/21 conduct experiment (results linked from journal--not compiled on wiki yet)

    • 4/2 compiled experimental results and software comparison on wiki in entirety

Resources:

https://valvesoftware.github.io/steam-audio/

https://resonance-audio.github.io/resonance-audio/

https://developer.nvidia.com/vrworks/vrworks-audio


Project evaluation:

  • The proposed project clearly identifies deliverable additions to our VR Software Wiki (5)

  • The proposed project will inform future research, ie, advancing human knowledge (4)

  • The proposed project involves large data visualization along the lines of the "Data Types" wiki page and identifies the specific data and software that it will use (3)--needs to still be decided

  • The proposed project has a realistic schedule with explicit and measurable milestones at least each week and mostly every class

  • The proposed project includes an in-class activity (5)

  • The proposed project has resources available with sufficient documentation (4)


Journal Evaluation:

  • Journal activities are explicitly and clearly related to course deliverables (4)

  • deliverables are described and attributed in wiki (3)

  • report states total amount of time (5)

  • total time is appropriate (4)


2/18-2/21, HRTF research (4 hours), Steam Audio Tutorial (2.5 hours), Resonance Tutorial (2.5 hours), VRWorks Audio Tutorial (2.5 hours)

2/24-2/25, Steam Audio VR Wiki Tutorial (3.5 hours)

2/26-2/27, Resonance Audio Wiki Tutorial (3.5 hours)

3/2-3/3, VRWorks Audio Wiki Tutorial (4 hours)

Physical DIY HRTF research (6 hours)

https://www.earfish.eu/sites/default/files/2018-01/DIY_earfish_iPhone_0.pdf

https://www.audiosciencereview.com/forum/index.php?threads/measuring-hrtf-for-headphone-use.3962/

3/7: Audio tutorial in class, compiled feedback and edited tutorials (2 hours)

3/10-3/11: Experiment design--started underground cave environment (2 hours)

3/12: Treak cavern environment in the YURT. Also ported Andrej's meteorite data in to YURT (4 hours)

3/13: Unity environment for cave (6 hours)

3/16: Environment edits and conduction of an experiment (4 hours)

3/17: More experiment conduction (2 hours)

3/23: Experimental results started to be compiled (2 hours)

4/4: Journal updating and finishing project results (2 hours)

4/9: Compiled, ran, and investigated Variable Rate Shading (api level as well as Unity and Unreal) (5 hours)

4/11: Compiled, ran, and investigated Lens Matched Shading (4 hours)

4/16: Compiled, ran, and investigated Single Pass Stereo (4 hours)

4/18: Compiled, ran, and investigated Multi-view Rendering (4 hours)

4/23: Compiled, ran, and investigated Multi-res Shading (5 hours)

4/25: Compiled, ran, and investigated VR SLI (7 hours)

4/30: VRWorks overview page completed (2 hours)

5/2: Feature evaluations put on the Wiki (4 hours)

5/7: Demo day preparation (poster) (4 hours)

5/9-5/10: Demo day preparation (4 hours)

5/13: Reviewing Zak's journal and Andrew's journal (2 hours)

Cleaned up Journal and contributions (3 hours)


(Across semester):

PPT creation and editing (6 hours)

Lab assistance [class only] (Brandon for class setup, Guise on VR machine 3 for IATK, Ross/Zak for tutorials, Andrew for VR laptop Glyph) (7 hours)

Total time: 138.5 hours


Wiki contributions:


VR Spatial Audio Software Comparison and VRWorks Investigation Poster:

VR Spatial Audio


Audio Tutorial Feedback

Audio Tutorial/Software Feedback (Responses)

Reviewed by: Andrew Buccellato

"Loudon compiled wiki contribtuions on VRWorks, Steam Audio, and Resonance Audio,outlining tutorials in all of them and offering a comprehensive software comparison on the cons and benefits of all of the spatial sound offerings. These pages seem to offer a great base for establishing a VR audio component to the wiki, and offers setup solutions for whatever software users settle on. Loudon not only added an entirely subset to the wiki, but successfully outfitted it with a well curated set of pages. I think these contributions go above and beyond the expectations and will really benefit the wiki's flexibility. (They really make me feel bad about my own wiki contributions by comparison)- but it's great to see the wiki so well outfitted"