Loudon Cohen journal

1/27,  installed Paraview and investigated software,  setup Journal, read last year's wiki (2 hours)

1/28,  read Experiments in Immersive Virtual Reality for Scientific Visualization, wrote a wiki page for NVIDIA's VRWorks (2.5 hours)

1/29, reviewed project specs. and old projects, thought about project ideas and investigated relevant s/w (4 hours)

1/30, assisted with VR Lab setup (3 hours)

2/3, installed VRWorks Unity plugin and explored environment (3 hours)

Potential Project Titles:

• Comprehensive VRWorks Investigation (deep dive into Turing advances, compatibility, etc.)

  • API vs. plugin comparison/evaluation

  • Testing of all application features (Variable Rate Shading, Lens Matched Shading, VR SLI, Single Pass Stereo, Multi-View Rendering, Multi-Res Shading)

  • Project demo incorporating all (toggleable) features

  • Class testing/study for impact of said features 

  • Possibly investigate application to the YURT: "VRWorks includes SDKs that accelerate 360 video capture and processing as well as enhance professional VR environments like CAVES, Immersive Displays & Cluster Solutions "

• Real-time Pathtracing Visualization using RTX (for content creators)

  • Port of RTX-accelerated path tracer to VR

  • Denoising layer such as SVGF (for interactive VR framerate reqs. and image quality)

  • Class activity unknown currently

• Audial Propagation Visualization

  • Compare Google's Spatial Audio API with VRWorks Audio

  • Evaluation of Binaural playback in VR with above APIs

  • Demo scene for audio "visualization" such as a maze with audio cues (perhaps with a class user study) 

  • Could be of interest to the Evolutionary Morphology Lab (Bat hearing)

Pre-project Draft:

• Working Title: VR Spatial Audio Framework Comparison

• Equipment: Vive and VR desktop in the VR Lab, software to evaluate

• Goal: Compare and contrast various VR audio frameworks (VRWorks, Google's Spatial Audio API, Steam Audio)

• Deliverable(s): audio framework tutorials, software comparison/evaluation results 

• Activities:

  • Go through tutorials for each framework

  • Create tutorials for each framework

  • Design an experiment for objective evaluation

• Milestones:

  • 2/12 compile all APIs/libraries to be evaluated and install them to ensure s/w compatibility

  • 2/14 project proposal done, use unreal engine with VRWorks

  • 2/21 personally complete tutorials for each framework, figure out data type

2/9, installed Google Resonance and Steam Audio on middle VR machine, attempted using unofficial Unity port of VRWorks--successfully installed, no documentation on how to use (6 hours)

2/10, data type for project research--standard audio formats and some research on environments that would be enhanced with spatial audio (i.e. caves) (2.5 hours)

2/11, pre-project proposal finalization and resource compilation (1 hour)

2/13, installed VRWorks with Unreal plugin (2 hours)

Time running total: 26 hours

Project Proposal:

Title: Spatial VR Audio Comparison

In-class activity: Have everyone import an environment and an audio type into Unity using the best evaluated API (ex. cave geometry and bat audio)

Milestones:

• • 2/21 personally complete tutorials for each framework, figure out data type for experiment later on

  • 2/26 complete tutorial of Steam Audio API on the wiki

  • 2/28 complete tutorial of Resonance Audio API on the wiki

  • 3/5 complete tutorial of VRWorks Audio API on the wiki

  • 3/7 personal evaluation/comparison of each API logged

  • 3/12 experiment design to evaluate each library objectively

  • 3/14 create portable testing environment in Unity

    • (Pre-idea): VR Maze with and without audio cues guiding to the exit. Test time for users to find the exit. Ask users for their evaluation of each API implementation

  • 3/19 port environment to Unreal Engine to ensure it works with VRWorks audio

  • 3/21 conduct experiment (results linked from journal--not compiled on wiki yet)

  • 4/2 compiled experimental results and software comparison on wiki in entirety

Resources:

https://valvesoftware.github.io/steam-audio/

https://resonance-audio.github.io/resonance-audio/

https://developer.nvidia.com/vrworks/vrworks-audio

Project evaluation:

• The proposed project clearly identifies deliverable additions to our VR Software Wiki (5)

•  The proposed project will inform future research, ie, advancing human knowledge (4)

• The proposed project involves large data visualization along the lines of the "Data Types" wiki page and identifies the specific data and software that it will use (3)--needs to still be decided

• The proposed project has a realistic schedule with explicit and measurable milestones at least each week and mostly every class

• The proposed project includes an in-class activity (5)

• The proposed project has resources available with sufficient documentation (4)

Journal Evaluation:

• Journal activities are explicitly and clearly related to course deliverables (4)

• deliverables are described and attributed in wiki (3)

• report states total amount of time (5)

• total time is appropriate (4)

2/18-2/21, HRTF research (4 hours), Steam Audio Tutorial (2.5 hours), Resonance Tutorial (2.5 hours), VRWorks Audio Tutorial (2.5 hours)

2/24-2/25, Steam Audio VR Wiki Tutorial (3.5 hours)

2/26-2/27, Resonance Audio Wiki Tutorial (3.5 hours)

3/2-3/3, VRWorks Audio Wiki Tutorial (4 hours)

Physical DIY HRTF research (6 hours)

https://www.earfish.eu/sites/default/files/2018-01/DIY_earfish_iPhone_0.pdf 

https://www.audiosciencereview.com/forum/index.php?threads/measuring-hrtf-for-headphone-use.3962/ 

3/7: Audio tutorial in class, compiled feedback and edited tutorials (2 hours)

3/10-3/11: Experiment design--started underground cave environment (2 hours)

3/12: Treak cavern environment in the YURT. Also ported Andrej's meteorite data in to YURT (4 hours)

3/13: Unity environment for cave (6 hours)

3/16: Environment edits and conduction of an experiment (4 hours)

3/17: More experiment conduction (2 hours)

3/23: Experimental results started to be compiled (2 hours)

4/4: Journal updating and finishing project results (2 hours)

4/9: Compiled, ran, and investigated Variable Rate Shading  (api level as well as Unity and Unreal) (5 hours)

4/11: Compiled, ran, and investigated Lens Matched Shading  (4 hours)

4/16: Compiled, ran, and investigated Single Pass Stereo  (4 hours)

4/18: Compiled, ran, and investigated Multi-view Rendering  (4 hours)

4/23: Compiled, ran, and investigated Multi-res Shading  (5 hours)

4/25: Compiled, ran, and investigated VR SLI  (7 hours)

4/30: VRWorks overview page completed (2 hours)

5/2: Feature evaluations put on the Wiki (4 hours)

5/7: Demo day preparation (poster) (4 hours)

5/9-5/10: Demo day preparation (4 hours)

5/13: Reviewing Zak's journal and Andrew's journal (2 hours) 

Cleaned up Journal and contributions (3 hours)

(Across semester):

PPT creation and editing (6 hours)

Lab assistance [class only] (Brandon for class setup, Guise on VR machine 3 for IATK, Ross/Zak for tutorials, Andrew for VR laptop Glyph) (7 hours)

Total time: 138.5 hours

Wiki contributions:

• VRWorks Page (100%)

• VRWorks Features (100%)

• Steam Audio tutorial (100%)

• Resonance Audio tutorial (100%)

• VRWorks Audio setup tutorial (100%)

• VR Audio Evaluation (100%)

VR Spatial Audio Software Comparison and VRWorks Investigation Poster: