PROPOSALS

Project 1 Proposal

Presentation for Project 1 Proposal

End Presentation for Project 1

Project 2 Proposal

Presentation for Project 2 Proposal <ADD LINK>

Poster <ADD LINK>

In-class Activity <ADD LINK>

Public Demo <ADD LINK>

ALL OTHER WIKI CONTRIBUTIONS

CONTRIBUTION 1 [Table of Contents for Software Comparison] Software Comparison Page

CONTRIBUTION 2 [Wiki setup and Template VR Project for Unreal Engine 5] Unreal Engine 5

CONTRIBUTION 3 [Terrain Generation with Blender] Terrain Visualization In Blender

CONTRIBUTION 4 [Exporting Materials From Blender] Baking Blender 

CONTRIBUTION 5 [Unreal Summary Page On General VR Development] VR Development Software

.....

CONTRIBUTION N [short description] <ADD LINK>

HOURS journal

Course Learning Goals:

before after

----  ----

 1   | 4 | Goal 1: articulate VR visualization software tool goals, requirements, and capabilities

 2   | 4 | Goal 2: construct meaningful evaluation strategies for software libraries, frameworks, and applications; strategies include surveys, interviews, comparative use, case studies, and web research;

 2   | 5 | Goal 3: execute tool evaluation strategies;

 1   | 4 | Goal 4: build visualization software packages;

 2   | 4 | Goal 5: comparatively analyze software tools based on evaluation;

 1   | 5 | Goal 6: be familiar with a number of VR software tools and hardware;

 3   | 5 | Goal 7: think critically about software;

 2   | 4 | Goal 8: communicate ideas more clearly;

 1   | 3 | Goal 9: effectively program useful VR-related features;

1/26/23 - 1.5 Hour

• Began ticking off to-do's on HW1, setup page and introduction, joined slack channel.

• Explored website for class context.

1/27/23 - 2.5 Hours

• Continuation of HW1

• Continued to explore site for class content.

• I was reading through the project ideas page and Gaia sky seems really interesting to me! Finding a way to work multiplayer capabilities in sounds like an interesting project to explore. Unity seems to have a way to do multiplayer VR (see Meta example), so maybe there's a way to link that together with Gaia to create guided tours of the solar system of sorts? Unsure, requires more research.

• Three changes should each require ~10 minutes to complete.

  • Move all tutorials relating to the Meta Quest 2 into one area, so it's easier to access.

  • The VR Development Software comparison page is quite long - adding a way to jump to certain sections would be helpful.

  • The "Oculus Website" Link under Oculus tutorials in the Unreal Engine page is broken, determine what page it was originally linked to and re-link it accordingly.

• Three changes should each require ~1 hour to complete.

  • Add a page about the newly released mixed-reality Meta Quest Pro Headset, with general facts and information about it. Include what makes it better than other headsets.

  • A bunch of the "VR in..." would benefit from a short description of applications before presenting users with academic papers or projects. A more general overview would help users with limited understanding of possible uses in the field before delving into an extremely detailed example.

  • The Meshlab VR Modeling Software page is a bit sparse. Adding more information about it would be helpful to the user to understanding the complexity, as well as compatibility with systems.

• The final three changes should each require ~10 hours to complete.

  • Add a page that denotes the differences between the types of Reality (Augmented, Virtual, Mixed, etc). Compare and contrast the pros and cons of each type. Include applications of each and possibly list a few different devices that use each.

  • There's a lot of content in the sidebar, consider restructuring and consolidating information in a more digestible format. Can take a while, as there are a lot of things to be reshuffled.

  • Add a FAQ's page (similar to What Is VR, but with other questions as well), with the most asked questions when it comes to the world of VR, collaboration, and data visualization.

• Complete one of the 10-minute changes.

  • Added a table of contents to the comparison page for easy access to specific sections, if a user is looking for a specific comparison.

1/29/23 - 1 Hour

• Reviewed Kenny Gruchalla's bio and reviewed some of featured work.

• Added questions to class activity board here.

1/31/23 - 1.5 Hours

• Setup Quest 2 Headset for use.

• Setup host system (local) with SteamVR and Oculus Link,

• Tested Quest 2 wired mode with Google Earth VR.

2/1/23 - 2.5 Hours

• Read through previous student projects.

• Pick 2-5 potential pieces of software to explore and evaluate for your research project

  • 1. Gaia Sky

    2. Unity3D (for multiplayer development + general use)

    3. NormCore (multiplayer, again)

    4. Paraview

    5. Fusion360 and VR support

• Using this software and insight from previous projects, create 3-5 potential project ideas in your journal.  Be prepared to share concisely (one sentence each) in class.  We will ponder their levels of: 1) collaboration, 2) immersive VR, 3) scientific visualization.

  • 1. Project idea 1: Using Gaia Sky, develop a way to integrate multiple people into one session. Allows for guided virtual tours in VR.

    2. Project idea 2: Visualize weather data (rainfall, temperature, etc) in large cities. Maybe a cumulative way to view all over world.

    3. Project idea 3:  Visualizing machine parts and possibly learning to assemble machines in a collaborative way.

  • Put course learning goals at top of journal.

2/3/23 - 4 Hours, HW for 2/7

• Worked through Google Earth VR tutorial and activity.

• Setup DinoVR for next class.

• Reviewed wiki and did some web research for my project ideas.

• Explored a bit more into Photon Phun 2 for multiplayer aspects.

• Software evaluation metrics that could be measured.

  • Ease of use - how long it takes the average user to get up and running for a given piece of software.

    • If a programming-based thing, then how long to reach a common "hello world" scenario.

  • Performance/resource consumption - how well it runs on different types of machines, or how well it runs on one machine compared to another piece of software.

    • Can measure in CPU/GPU/RAM/ROM utilization.

    • Can measure FPS on a given system for a given scenario.

    • Measure in a sense optimization between a few sets of programs.

  • Documentation - can compare how comprehensive each program's documentation is, plays into how easy it is to use.

  • Featureset - measure what can be done and can't be done on a given system in terms of features. More features =  better?

  • Plugin/Add-on compatibility support.

  • Compatibility with formats of datasets.

2/6/23 - 3.5 Hours, Continuation of HW for 2/7

• Continued research about following topics:

  • Understanding multiplayer VR

    • Explore Unity, Photon Phun 2, looked at some tutorials for general idea.

• 3 Project ideas (reiterated)

  • Project idea 1: Using Gaia Sky, develop a way to integrate multiple people into one session. Allows for guided virtual tours in VR.

    • What I've learned that is Gaia Sky is quite complex, and for a person who doesn't have that much experience working with graphics and VR, developing multiplayer for an already very well fleshed out application could prove to be a lot.

    • I think beginning out by learning Unity or related VR development software and exploring a system that allows for multi-person collaboration in general would be a good starting point.

    • Things I'll do during the project:

      • Explore the different types of systems that exist for multiplayer development, specifically in VR.

      • Implement an multi-user experience to test specifically collaborative aspects, how users can interact with one another, passing objects around.

      • Learn more about VR development in general.

    • One class activity for project:

      • Test the multiplayer system by allowing a bunch of students to join together and explore through a server.

    • Potential deliverables:

      • Wiki additions on explored software.

      • A framework for setting up multi-user VR experiences, with no VR-development experience at all. (still some programming experience required)

  • Project idea 2: Visualize weather data (rainfall, temperature, etc) in large cities. Maybe a cumulative way to view all over world?

    • (other ideas include: visualizing snowfall, allowing students to experience the largest snowfall amounts in comparison to their given height. Visualizing the size of the largest beings on earth, allowing students to experience sizes compared to them.)

    • Things I'll do during the project:

      • Explore the different types of engines used to develop VR applications, and slowly learn it.

      • Learn more about VR development in general.

      • Develop an application that looks a given dataset and displays it accordingly.

    • One class activity for project:

      • Allow students to try application and observe the varying weather across different regions, notice patterns.

    • Potential deliverables:

      • Wiki additions on explored software.

      • An application that is able to visualize weather data 

  • Project idea 3:  Visualizing machine parts and possibly learning to assemble machines in a collaborative way. (Mindesk, Fusion360, Solidworks)

    • Things I'll do during the project:

      • Explore the different types of CAD applications or CAD-based programs that support VR.

      • Learn more about VR development in general.

      • Display data about machines and mechanisms through CAD and VR.

    • One class activity for project:

      • Students can create and visualize their own creations in VR, understand limitations and benefits of visualizing machines/creations in VR.

    • Potential deliverables:

      • Wiki additions on explored software.

2/7/23 - 3 Hours, Preparation for Class 2/9

• Did some more research in to specifics on multiplayer collaboration on wiki.

• Finished DinoVR activity, filled out feedback form.

• Finished in-class activity reviewing projects.

2/8/23 - 3 Hours, Preparation for Class 2/9

• Repivoted view of project to something different, and less complicated.

  • General Project Idea: Visualizing Ecosystem Structure over a given timeframe.

  • Found NEON Dataset with LiDAR scans that allow us to visualilze a given area's ecosystem, elevation, etc. 

    • Should allow for a general terrain visualization, and vegetation/tree cover in a given area.

• Project Milestones:

  • 2/14

    • Research, download, and understand datasets.

      • Figure out what each dataset shows, and what datasets can be used.

  • 2/16

    • Download Unity, setup and build a test environment to get used to how Unity works in VR Development.

      • Start with a simple "hello world" program.

  • 2/23

    • Begin to import LiDAR Data, test terrain visualization.

  • 2/28

    • Import other ecosystem structure of LiDAR Data, overlay on top of terrain visualization.

  • 3/02

    • Implement ability to change time, and in turn, visualization.

  • 3/07

    • Make UI Tweaks, possibly adding time change function to a menu of sorts.

  • 3/09

    • Present to Class! (May shift depending on success)

    • Class activity: have users explore different timeframes for a given area, and point out the more significant changes over time, similar to a "spot the difference" style experience.

2/11/23 - 2 Hours - Preparation for Class 2/14

• Explored NEON Dataset, researched ways to visualize in Unity & Unreal Engine

2/12/23 - 3 Hours - Preparation for Class 2/14

• Created powerpoint presentation for project 1 proposal.

2/13/23 - 2.5 Hours - Preparation for Class 2/14

• Created new page for project 1 proposal, can be found at top of journal.

2/15/23 - 4 Hours - Experimented With Unreal Engine VR Development

• Self evaluation of journal:

  • Journal activities are explicitly and clearly related to course deliverables - 4

  • deliverables are described and attributed in wiki - 3

  • report states total amount of time - 5

  • total time is appropriate - 4

• Watched a few tutorial videos

  • Tutorials on General Unreal Engine 4

• Downloaded Unreal Engine

• Looked more into this plugin for GEOTIFF Data --> https://github.com/iwer/GeotiffLandscape

• Decided to move to Unreal Engine 5 - Notable Better VR Support, offers additional wiki deliverable

  • Plugin support of LiDAR point clouds --> https://docs.unrealengine.com/5.0/en-US/lidar-point-cloud-plugin-for-unreal-engine/

• Setup android studio & Visual Studio to work with VR

• Unreal Engine 5 Videos:

  • https://youtu.be/DiGh6MxDFds

  • https://youtu.be/Z4sClxhgsxk

2/17/23 - 3 Hours - Continued experimenting with Unreal Engine 5

• Watched multiple tutorials on Unreal Engine 5.

  • https://youtu.be/ptCN4cysDig

  • https://youtu.be/UVTzVtxxgE0

• Played with lighting effects and models, explored Quixel Bridge for useful assets.

2/19/23 - 3 Hours - Revisiting LiDAR dataset and insertion into unreal engine 5

• Watched some UE4 Tutorials, may be able to "recycle" for UE5

2/21/23 - 5 Hours - Experimenting with LiDAR dataset, looking into TerrainMagic

• https://github.com/GDi4K/unreal-terrain-magic

• https://docs.unrealengine.com/5.0/en-US/lidar-point-cloud-plugin-for-unreal-engine/

• https://youtu.be/bLk1-Ulb7Dk

• Visual Studio Setup for Unreal Engine:

  • https://docs.unrealengine.com/5.0/en-US/setting-up-visual-studio-code-for-unreal-engine/

• Ran into issues installing terrainmagic, spent some time reconfiguring visual studio & unreal engine

• https://docs.unrealengine.com/5.0/en-US/editing-landscapes-in-unreal-engine/

• https://docs.unrealengine.com/5.0/en-US/creating-and-using-custom-heightmaps-and-layers-in-unreal-engine/

Finally imported a (admittedly unpretty and issue ridden) copy of a section into Unreal as a heightmap

2/22/23 - 2 Hours - Working on cleaning data, Paraview Install for local and paperspace machines

• Installed paraview and tutorial file for local and paperspace machine.

• Trying to find solutions for overprint in unreal engine with .png heightmap installs.

2/24/23 - 3 Hours - Re-assembling data for visualization

• Combined heightmaps together to generate larger heightmap in photoshop.

  • Matched up heightmaps so that covered area is same at all time points.

2/25/23 - 2 Hours - Re-importing heightmaps into Unreal Engine

• Used landscaping tool to generate heightmap from assembled PNGs

• Noticed issue: peaking in a lot of areas, not really helpful for data visualization.

2/27/23 - 1 Hours - Creating Progress Presentation, Preparation for Class

• Created powerpoint for class.

• Read through Melvin's starting activity.

3/1/23 - 2 Hours - Preparation for class

• Read through Lexi's and Mohammed's documents, prepared for class.

• Finished class Melvin's class activity and filled out google form.

• Looked more into materials for Unreal Engine 5

3/4/23 - 4 Hours  - Refactoring and Regenerating Dataset

• Kept running into an issue of peaking within the dataset, too many fully white spots in images.

• Regenerated images in photoshop manually, matched each year so that all datasets show the same information.

• Also added 2016, 2017, 2019 datasets and compiled them accordingly.

• Ran into issues with collages creating thin white lines between each tile.

3/6/23 - 10 Hours  - Re-doing the dataset in Blender and Importing it into Unreal Engine (again)

• One of the issues I kept running into was the problem of performance. Since I was importing this massive .png file as a landscape, Unreal would import this as a huge landscape, which was unoptimal for performance, even on my personal gaming PC.

  • Additionally, there was no way for me to directly interact with the environment, should I want to change the way I view the dataset.

• So I moved to Blender to regenerate the dataset, which was a process in itself learning how it worked. Fortunately, I was able to find a video that showed how to bring my heightmaps nicely into blender.

  • Link Here

• I exported these heightmaps as .fbx files to Unreal to see if they'd work, and they did. Using the same material system above, I added color to the datasets created using blender as meshes in Unreal, and kept going. 

  • However, one issue I ran into was when the meshes were moved, they changed colors, since my material system was based on world position, which would mess everything up.

• After my issues in Unreal, I had to create a color/material system that would translate colors into height, but keep it static. In this case, I would create the material in Blender, bake it into a .png, which could be used as a texture mapping onto the meshes in Unreal. I made a gradient where green was the shortest values and red the highest values, applied it to the 3d mesh, and then pulled the resulting mesh, "baking" it into a hardcoded file.

  • This involved learning about how to bake images, how to use the material system in blender, and importing it into Unreal accordingly. 

Results below for this portion:

3/7/23 - 8 Hours  - Updating the Virtual world with the new datasets and setting up the environment

• There was a lot that was done, so here we go!

  • Using Unreal Engine Vr Template

    • Created a new world with existing VR functions.

    • Troubleshot issue with teleporting functionality, though now it doesn't have collision mitigation, so it's possible to teleport outside of the designated area and into items (not ideal, but better than no teleportation at all).

    • Added lighting to better view items.

    • Updated the start menu to exit the game, as well as restart in the new level created.

  • Added in new meshes from Blender as grabbable items, so they can be manipulated and moved for comparison.

    • Loaded new materials onto meshes for coloring.

    • Added times to datasets, so that users can differentiate between them. (took longer than expected, had grabbing issues with linking the two items together).

    • Updated physics so that when items are released they aren't subject to gravity and don't float away when released (will remain stationary).

  • Troubleshooted issues regarding outputting to .APK for Quest 2.

  • Currently dealing with performance issues due to many polygons.

3/8/23 - 3 Hours  - Preparing in-class activity.

• Added page for in class activity here.

• Uploaded APK to google drive for sharing.

• Added google form for results compilation.

• Updated APK with new compilation,  fixing some height issues when sitting down.

3/13/23 - 3 Hours - Creating Unreal Engine 5 Tutorial.

• Added page for Unreal Engine 5 here.

• Added information and link for UE5 within the VR Development Software Page.

3/13/23 - 5 Hours - Adding More Wiki Contributions

• Added page for Blender Terrain Vis here.

• Added page for Blender Material Export here.

• Generated final presentation for project 1.

• Performed Self Evaluation.

3/16/23 - 1 Hour - A Short Analysis of Project Survey

• Results of preference to VR over 2D visualization

  • Tends to skew a bit towards VR, but still have some people who prefer 2D.

  • Why?

    • Against VR:

      • Users note jittery performance of web application.

      • Some dizzyness of locomotion in VR.

      • Some noted the color coding associated with the 2D visualization is more intuitive than in 3D VR.

    • For VR:

      • Ability to move VR object around and view it from different angles.

      • Being able to see 3rd Axis is helpful.

      • Easier to view and compare two heightmaps in VR, more readable and engaging.

• Recommended Improvements:

  • Improved movement in world, move to smooth locomotion instead of teleportation.

  • Implement collision boxes and prevent clipping in obstacles.

  • Add depth controls for map manipulating dataset.

  • Improving lighting setting.

3/20/23 - 4 Hours - Preparing for Class 3/21

• Updated wiki contributions with attribution and information regarding contributors.

• Read and took notes on 7 scenarios paper.

• Project 2 Draft Design

  • Exploring different ways to implement multiplayer VR experiences in Unreal Engine 5.

    • Consider: separate plugins, and systems for implementation.

    • Use an already existing world (probably the one that I created in project 1) , and look at different ways to work multiplayer.

      • Random links that

        • https://forums.unrealengine.com/t/vr-pc-multiplayer/691364/2

        • https://www.semantic3d.net/ (point cloud dataset)

        • https://www.cvlibs.net/datasets/kitti/ (more point clouds!)

        • https://pandaset.org/ (autonomous vehicle dataset)

  • Another way to visualize a LiDAR dataset (very unrefined project, but dataset mentioned by Prof. Laidlaw made me interested)

    • Need to get more info from Prof. Laidlaw in order to continue with this.

    • Autonomous vehicle datasets?

  • Another idea: Spatial audio in VR?

    • https://docs.unrealengine.com/5.0/en-US/spatialization-overview-in-unreal-engine/

• 7 Scenarios Paper:

(We'll focus on developing VR multiplayer in Unreal for this one)

• • Understanding Environments and Work Practices (UWP)

    • There are existing visualizations that allow people to collaborate with one another, determining how people want to interact with one another is important. Will people be in the same room? Will they be able to communcate via voice or not? We can use this information to help 'inform the design of a future visualization tool'.

  • Evaluating visual data alaysis and reasoning (VDAR)

    • Assessment of ability to support visual analysis and reasoning about data.

    • We must consider a given plugin's supported actions and interactions between users, which will help visualization on both sides. While this may be a game implementation, it also can be dependent on the capabilities of the multiplayer system.

  • Evaluating communication though visualization (CTV)

    • Must consider how well content is communicated between two users via plugins, namely, movement of player and objects throughout the field.

  • Evaluating collaboration through visualization (CDA)

    • This is fully immersed in the world of collaboration, so this is probably the greatest and most important evaluation here.

  • User performance (UP)

    • Maybe evaluation of how the user does in a set of certain tasks that depend on the other user to participate would be a good metric.

  • User experience (UE)

    • Measuring user experience can be done in various ways during this project, the main method being through a series of surveys about how users felt their collaborative interaction was, how the world was manipulatable between users.

  • Visualization Algorithms (VA)

    • Can assess performance of different plugins, namely latency, and connections between users. Must determine if content is working fast enough to be viable for use.

3/22/23 - 3 Hour - Preparing Presentation for Project 2, Assessing Possible Datasets.

• See Here

Google Earth VR

Google Earth Web