Introduction

In a collaborative group effort, we created a wiki that reviews the current state of software for creating virtual and augmented reality artifacts. This wiki is a resource for developers building VR software and experiences that target three software uses: development, modeling, and scientific visualization. We investigated the capabilities of software for head-mounted displays (HMDs), big-metal displays like caves and the yurt, and desktop displays. We considered software at different levels of the stack including graphics and visualization libraries, prototype application frameworks, game engines, and stand-alone applications. We evaluated these software via web research, hands-on case studies, and surveying. In addition, we created tutorials for selected software and workflows.

Where do I start?

Contents

1. What is VR?

Read about the history, use cases, and capabilities of virtual reality (VR).

1.1. History of VR

Read the history of VR software to get context for your VR research, games, or projects.

1.2. Who uses VR?

Read profiles of user exemplars to guide your search for VR tools and software.

1.3. What is VR for?

Explore VR applications in different domains with the examples below. Find a literature review on the main "What is VR for?" page.

1.3.1. VR in Education

Studies have shown that individualized learning helps students achieve their academic goals at a faster pace. Today, classroom structures are not so different from the structure years ago—teachers create a curriculum for the entire class, and students typically learn at the same pace. Virtual reality learning environments can change that. With VR, students can personalize their learning, choosing how long to spend their time on certain concepts, and walking through simulations themselves. VR environments can increase both motivation and hands-on experience. 

1.3.1.1. VR Education Research Labs

This is a page of descriptions, aims, current projects, and the contact information of academic research labs focusing on the intersection of VR/AR and education. 

1.3.1.2. Example VR/AR Education Software

This page lists several of the most popular VR/AR educational apps across different fields, including science, geography, and art.

1.3.1.2.1. Star Chart 

Star Chart is an AR star library. Users can hold their devices up to the sky, and Star Chart will immediately track their locations and display the constellations they are currently looking at through their device's camera.  

1.3.1.2.2. Sites in VR 

Sites in VR allows users to go on virtual tours of landmarks from Turkey, Egypt, Saudi Arabia, Syria, Morocco, Kuwait, Yemen, Macedonia, Holland, Belgium, France, Italy, Greece and Space. There are thousands of 360 degree panoramic images in high quality for users to explore.

1.3.1.2.3. Titans of Space 

Titans of Space takes users on a guided trip through our solar system. Users control the pace of the adventure, and they can choose to be completely hands off or choose to interact with the planets and moons.

   1.3.2.  VR in The Performing Arts

Having been developed for gaming,  VR is now beginning to influence another form of entertainment. Live performance and the adaptation of designs virtually, before implementation.

VR development software is used to code VR experiences.

2.1. Unity3D

Unity3D is a widely used, free, fast and beginner-friendly 2D/3D game development platform. It is excellent for integrating different assets and can compile to any platform, including VR. It also is capable of creating AR content by allowing  AR development tool plugins such as ARKit and ARCore.

2.2. Unreal Engine

Unreal Engine 4 is a 3D game and animation engine known for creating very high quality visuals. This software is a great tool for anyone who is working with real-time technology or wants to create photorealistic scenes. It can also create VR and AR content (allows for ARKit and ARCore plugins) but is not very beginner-friendly. 

2.3. Cryengine

This software is a video game development engine with beta support for VR.

2.4. Lumberyard

"Lumberyard is a free game engine deeply integrated with AWS and Twitch... [that] provides a growing set of tools to help you create the highest quality games, engage massive communities of fans, and connect games to the vast compute and storage of the cloud." - Amazon.com

2.5. WorldViz (Vizard)

A software intended to allow you to "build once, deploy everywhere." It claims "Universal VR hardware connectivity supports all HMDs, 3D displays, motion trackers, gloves, and input devices." It has been used for experimentation and research at Stanford.

2.6. MinVR

MinVR is a cross-platform VR development library. Unlike most other software listed, it does not make use of a GUI, instead relying on the user to use OpenGL to create the VR application. It supports a multitude of different VR platforms, including Brown University's YURT, various Cave systems, and SteamVR, and supports plugins for other platforms.

2.7. A-Frame

A-Frame is an open-source web framework for building VR software. A-Frame is a component of Three.js, so developers can create WebVR scenes with JavaScript and HTML.

2.8. Xcode/CocoaPods

CocoaPods is a dependency manager for Objective-C and Swift. It supports limited integration with Google Cardboard SDK and contains several pods (code packages) for displaying VR videos on an iPhone. 

2.9. React 360

React 360 is a framework for the creation of interactive 360 experiences that run in your web browser. 

2.10. Tutorials of VR development software

Explore tutorials of VR development software, including Unity3D, WorldViz, MinVR, and A-Frame.

2.11. VR Development Software Comparisons

Read our evaluations of the different capabilities of various VR development software.

2.12 VR for Web

Guide to front-end and back-end frameworks for web, including how to choose between A-Frame, React 360, and Three.js, as well as multiplayer communication SDKs such as Photon. 

VR modeling software is used to design, sketch, and build VR assets, such as realistic renderings of environments or objects. Most software includes physical simulators and animation tools. 

3.1. Adobe Photoshop 3D

Photoshop can build a variety of basic 3D objects using 2D layers as a starting point. After creating a 3D object, you can move it in 3D space, change render settings, add lighting, or merge it with other 3D layers.

3.2. Tinker CAD

Tinkercad is a free online collection of software tools that help people all over the world think, create and make. We’re the ideal introduction to Autodesk, the leader in 3D design, engineering and entertainment software.

3.3. Blender

Widely used, free, fast, and reliable 3D modeling tool, much used for asset creation. Steep learning curve, but suitable for highly complex  modeling operations, including physics simulations and animations, and integrates well with other types of software. Does not directly support HMDs though has an extension (BlenderVR) that allows additional multiscreen output.

3.4. Google Sketchup

Free, powerful, and easy to learn and use 3D modeling program. Quick and straightforward process to make something large, complicated, and/or interesting using the SketchUp 3D Warehouse. 

3.5. Meshlab

Free and reliable mesh editing software, particularly suitable for complex mesh problem solving operations. Steep learning curve, but smooth and quick workflow.

3.6. Netfabb

Free and reliable mesh editing software, particularly suitable for simple mesh problem solving operations. Shallow learning curve, but limited possibilities as opposed to Meshlab.

3.7. World Machine

Free terrain generation software. Uses visual programming language interface to generate large terrains that can be exported and used in other development software (e.g. Unity3D or Blender).

3.8. Tutorials for VR modeling software

Explore tutorials of VR modeling software, including Blender, Meshlab, Netfabb, Maya, and World Machine.

3.9. Maya 

Available college student trial for 2.5 years, generate both terrain and complex 3d models for viewing in VR.

VR visualization software refers to software that takes in input data and graphically illustrates the data so that scientists can gain further insight into that data.  It differs from VR modeling and development software in that in focuses on the conversion of scientific data into realistic renderings of volumes, surfaces, etc.

4.1. Omegalib

Omegalib is a "framework for virtual reality and cluster-driven display systems." It is intended for 3D scientific visualization, especially in the fields of astrophysics, engineering, and biochemistry.   

4.2. VTK

Visualization Toolkit is an open source, scientific visualization library owned by Kitware. VTK is the foundation of many advanced visualization application including ParaView and VisIt.

4.21. VTK.js

VTK.js is a JavaScript implementation of the standard C/C++ VTK library.

4.3. FieldView

Use FieldView to create immersive animations for engineering processes.

4.4. Paraview

ParaView is an open-source, scientific visualization modeling tool owned by Kitware.  It differs from VTK in that it has a GUI that allows users to model data without writing any code.

4.5. Visit

VisIt is an open source, scientific visualization modeling tool used to quickly generate visualizations, animate them through time, and manipulate them with different operators and filters. 

4.6. Amira

Amira is a platform for visualizing scientific data. It has an intuitive GUI that makes it easy to analyze different types of data, especially data within the medical field. 

4.7. OpenSpace

OpenSpace is a platform intended to provide astrophysicists with a model of the current state of the observable universe, giving them the ability to see real-time data or simulate the past or future.

4.8 VRWorks

NVIDIA VRWorks is a suite of APIs and libraries designed specifically to enhance VR experiences. Parts of VRWorks are integrated into engines such as Unreal Engine 4 and Unity.

4.9.  Glyph

Glyph is a student made Unity program developed to help develop VR Visualizations in the CAVE/YURT simultaneously alongside headset VR.

4.91 IATK

2018 - Designed VR viewing plugin for Unity, uses CSV files to generate interactive scatterplots with up to 3 axes of data.

4.92. Tutorials for scientific visualization in VR

Learn to develop scientific visualizations with VR. Examples include VTK, ParaView, VisIt, and Amira.

4.10. Veloview

VeloView performs real-time visualization and processing of live captured 3D LiDAR data from Velodyne’s HDL sensors (HDL-64E, HDL-32E, VLP-32, VLP-16, Puck, Puck Lite, Puck HiRes)

VR hardware brings a VR experience to life. After you develop a VR experience with VR development software and visualize it with VR modeling software, you explore it in a headset.

5.1. Google Cardboard

Google Cardboard is a low-budget VR headset designed to work with Android and iOS smartphones. It is entirely cordless and does not require a powerful computer to render, but compromises with a worse VR experience.

5.2. Google Daydream

Google Daydream is Google's higher end VR platform. Like Google Cardboard, it relies on a user's smartphone to render and is similarly cordless. However, it offers a significantly better VR experience, but also requires a specific higher end Android smartphone.

5.3. Windows Mixed Reality HMDs

Microsoft has partnered with several hardware manufacturers to create a line of VR HMDs. They utilize onboard cameras for 6dof tracking, which makes them cheap and portable but also results in some tracking issues. They are SteamVR compatible, so anything which supports a Vive can be run with a WMR headset.

5.4. Oculus Rift

The Oculus Rift is the most popular PC HMD device. It supports SteamVR and 6dof tracking using a pair of IR cameras. The Rift also supports some exclusive content, but other SteamVR capable devices can access it through intermediary software.

5.5. HTC Vive

The HTC Vive is one of the most prominent VR HMDs. It supports native integration with SteamVR, and utilizes a pair of external trackers to provide 6dof tracking, which is generally the most reliable and accurate tracking of the major VR platforms.

5.6. Samsung Gear VR

Samsung Gear VR is a consumer mobile HMD that leverages the Oculus Rift platform to run flagship Samsung phones as a screen and computer.

5.7. Sony PlayStation VR

Sony's PlayStation VR (PSVR) is an HMD designed for PlayStation 4 and PlayStation Vita. It was launched in 2016 and requires the PlayStation Camera.

5.8. Compare low-budget headsets

Compare low-budget headsets. Most low-budget headsets are self-assembled, cardboard HMDs. They're cordless, so you rest your phone directly on the headset.

5.9. Hardware emulators

Hardware emulation software gives developers and consumers the ability to run VR applications without owning VR hardware. iVRy and Vridge leverage mobile phones for 3dof tracking, and SteamVR also provides a Null Driver for basic rendering.

5.10. VR hardware tutorials

This page is for tutorials of VR hardware. The tutorials demonstrate how to set-up, run, debug, and implement certain hardware configurations.

Read about augmented reality (AR) and similar technologies.

6.1. AR vs VR

Compare virtual and augmented reality to see if AR is right for your particular use case.

6.2. AR Development Software

Explore AR development software and tools, such as the Apple ARKit.

6.21. AR.js

AR.js is an open-source library built on top of A-Frame, three.js, and ARToolKit.

6.22. A-Frame AR

A-Frame AR is an open-source web AR development library enabling A-Frame support for three.ar.js, Google's experimental WebARonARKit/ARCore browsers, and Mozilla's experimental WebXR browser.

6.3. Getting Started A-Frame

Try tutorials of A-Frame AR development software.

6.4. Web AR Comparison – AR.js vs. A-Frame AR

6.5 Google Earth VR vs Open Street Map VR

6.6 Lens Studio by Snap Inc.

7. Data Types and Examples

This section corresponds to the various types of data that are visualized in VR.

7.1 Data Types and Examples

7.2 Software Recommendations by Data Type

8. Student research

This section reports student research into VR. Most content here is specific to Brown or was created for CSCI1951S or subsequent courses.

8.1. VR@Brown

This page is a list of VR communities, labs, and research at Brown.

8.2. 2018 research findings

We conducted a round robin style workshop comprising of 6 activities where participants dabbled in VR software, hardware, and media. We gathered data to understand how to get people more interested and excited about VR in addition to helping formulate an assessment of currently available VR software, hardware, and media. We wrote a Medium article summarizing our results and also put together a page with our results.

8.2. AR user study

To further understand the current state of Augmented Reality Software, we created multiple tutorials for our spring 2018 class to complete. We then got feedback from each user in the form of a survey. This page is a summary of the results. 

8.3. fMRI Data Visualization

I wanted to model memory clusters in the brain to see if you could locate memories (that were detected using fMRI scans) in specific parts of the brain. This page describes fMRI, provides a tutorial of how to model it, and explains the issues that come with using fMRI data.