VR Tools for Art and Science

Contributed by Lexi Henrion in Spring 2023

Motivation

Virtual reality (VR) technology provides an immersive and interactive environment that can be leveraged for both creative expression and scientific data visualization. The following seeks to elucidate exactly how we can combine aspects of these two fields (which make them so primed for VR integration individually) in order to envision a workspace that goes beyond the limits of their current, individual offerings.

Drawing and sketching in VR allows users to manipulate objects in 3D space, view objects under different conditions (e.g. lighting), and with more realistic depth perception than photos can offer, all tools which aid in the intuitiveness, flexibility, and quality of artwork both in simulated 3D and 2D scenarios.

VR environments can also provide intuitive and immersive data visualization tools that enable users to interact with and explore complex scientific data sets, which may not be able to be viewed as effectively as 2D or still images. How can we most effectively analyze this data? The results of my in-class activity suggest that pen tools and the ability to take notes are desired features of those seeking to analyze 3D Data in VR. Five out of seven users surveyed thought that the 3D pen was the single most important tool for relating to the 3D data, with the remaining two citing the whiteboard and notes features.

A 3D pen allows people to make connections through the virtual space, and whiteboarding out ideas can help convey discovered information. These are features which are robustly supported by VR drawing apps, but are primitive or lacking in more multipurpose, or scientifically specialized apps. After exploring several different VR applications with drawing capabilities, I feel confident that there is potential in the idea of hijacking artistic tools for more scientific analysis.

ENGAGE VR, Gesture VR, and Tiltbrush

Meta's ENGAGE VR, Gesture VR, and Tilt Brush/Multibrush by Google are three popular VR applications for creative work and data visualization. ENGAGE VR is a collaborative platform that provides both 2D and 3D sketching capabilities on whiteboards and with a 3D pen. Gesture VR, on the other hand, is focused primarily on 2D drawing and figure drawing (though it does offer limited 3D drawing capability). Tilt Brush/Multibrush is a specialized 3D drawing tool that allows users to paint and sculpt objects in 3D space

ENGAGE VR

One of the strengths of ENGAGE VR is its collaborative environment, which was corroborated by users who engaged with it in my in-class activity. ENGAGE allows multiple users to interact and create in real-time, making it an ideal platform for remote collaboration. Additionally, ENGAGE VR provides both 2D and 3D sketching tools, making it versatile for different types of work. Tools offered are fairly rudimentary, without special effects, dynamic line weights, etc. (features which are provided by the other two entries on this list), but they are effective in conveying ideas casually. However, ENGAGE VR requires users to pay to load in their own meshes into the space, which could be a limiting factor for some use-cases.

Screen Capture of ENGAGE VR In-Class Activity. You can see the primative nature, but effectiveness, of the tools.

Gesture VR

Gesture VR excels in its 2D drawing capabilities, particularly for figure drawing. It provides a range of tools for sketching, including a virtual easel and pencil, making it an excellent option for artists and designers. The 2D capability is shockingly realistic and imitates real material like charcoal. The app also provides adjustable lighting to view the model under various conditions, which can be helpful for studying the form. However, Gesture VR does not offer collaborative capabilities, which could be a drawback for those looking to work in a team environment. It also does not support loading in your own models, making visualizing anything besides preset figures impossible.

~25 minute sketch done in Gesture VR

Tilt Brush

Tilt Brush/Multibrush is a powerful 3D drawing tool by Google that allows users to create complex 3D models and environments. It provides a wide range of brushes and tools for sculpting, painting, and designing in 3D space. However, Tilt Brush/Multibrush does not offer any 2D drawing capabilities, which could limit its usefulness for certain types of creative work. While there apparently are ways to load in models to the app according to certain forum answers, this functionality is not supported by Google and requires a PC and the open source edition of the app, OpenBrush. The documentation for Tilt Brush and its variations is either confusing, outdated, or nonexistent. The library of user-created work and loadable models also went obsolete and is no longer supported. Additionally, while Multibrush offers some collaborative capabilities, the original Tilt Brush does not.

Dress "Sketch" in Engage VR using 3d pen & shape tools. You can make some pretty wild looks!

Concluding Thoughts

In terms of scientific data visualization, ENGAGE VR seems to be the most promising platform. Its collaborative environment and 3D sketching tools make it well-suited for exploring and visualizing complex data sets. However, if one were to repurpose the specialties of Tilt brush or Gesture VR with the purpose of using it for analytical work, it would likely quickly outstrip the primitive capabilities of ENGAGE VR. Overall, Tilt Brush and Gesture VR are too specialized to be effective for scientific use-cases, whereas ENGAGE VR is too generalized to offer robust support of the desired capabilities.

Each of these VR applications has its strengths and weaknesses. ENGAGE VR is ideal for collaborative work, while Gesture VR and Tilt Brush/Multibrush are better suited for individual creative work. By taking note of the most desired tools by users, combined with the specialty implementations of similar tools in different, more specialized niches, I believe we can create excellent environments and tools for data visualization and collaborative analysis. This mission will be the foundation for my improvements to my own app, VRStudio, as I integrate 3D drawing and markup into the space which currently supports loading models from the backend and 2D drawing capabilities.

Page updated

Google Sites

Report abuse