VR in Visual Arts

Contributed by Lexi Henrion in Spring 2023


Virtual Reality (VR) has the potential to revolutionize visual art by providing a real experience of depth that is not possible with flat reference images. Moreover, it is unlikely that an artist will always have a perfect physical reference nearby. I designed VRStudio to be a VR drawing space to study and learn form - any form you can find a 3D model of on the internet! I conducted an in-class activity where my classmates tried out this  VR app that I designed, VRStudio.

Link to the Activity + Demo .apk file HERE

Link to Github Repo HERE

Art is like learning to translate a language - It shouldn't be exact.

Research suggests that figure and life drawing are better for developing skills than sketching from photo references. A copy from flat-to-flat, as is drawing from a screen or photograph, is a vastly different experience from a translation of a truly dimensional 3D form to a flat representation. Note the verbiage of “copy” verses “translation.” Because a photo or screen is already flat, it has done the work of the artist itself in translating from 3D to 2D, and its translations, being done by a machine, lack intention expression by themselves. While valuable in its own right, photos provide an “answer key” to where parts of a 3D form should end up relative to each other on a flat surface. Life drawing, on the other hand, provides a more immersive experience that allows an artist to develop a deeper understanding of form. 

The Limitations of VR (and myself): VRStudio

VRStudio is an app that features a canvas on which users can draw using various pen tools. The app allows users to load various models from the internet and sketch with the tools in VR. However, the app has some limitations, including being pixelated and laggy, lacking precision, and not having pressure sensitivity. Participants found it difficult to control, which affected their ability to create detailed sketches.

~25 minute sketch done in VRStudio

After trying the VRStudio app, participants were asked to sketch on regular paper with a pen instead of VR and fill out a form comparing and contrasting the two experiences. Based on the results, 71.4% of respondents agreed that the paper looked more proportionally correct, and only 50% of participants were able to get the VR app to behave the way they wanted. Disappointing, but understandable given tools that were developed in just weeks. Additionally, an enormous 92.9% of respondents found sketching on paper easier, and 57.1% also found it more engaging.

The free answer portion of the form provided additional insights. Multiple participants mentioned that VR provided a better visual experience, while drawing on paper was easier to control and better for detail. Control in the VR app was notably hard, especially on edges, curves, and outlines. The C# scripting being quite primitive meant that lines sometimes went where users didn't want them to go.

In response to the question, "What did you notice about the experience of viewing the model analytically in VR versus your screen?" users of the app liked being able to walk around and see the full model. It felt intuitive, whereas sketching the same 3D model from a 2D computer screen made the object seem smaller and flatter. Users of the VR app liked seeing different angles, and proportions seemed more straightforward to gauge, although the low, pixelated quality of the VR app was mentioned once.

Overall, users found the app interesting and fun, with a "cool" factor and some technical benefits. The convenience of virtual tools and not running out of ink was also mentioned. However, the app needed work in terms of smoothness and control. Some ideas to improve the app included adding tools, quality-of-life improvements (like teleporting a dropped marker back onto the table automatically), controller vibration to make the surface of the canvas more "real" feeling, and more adjustable pencil orientation. Smoothness won by a landslide in the poll of top priorities in need of improvement.

Breaking Limits:  Gesture VR

As the author and developer of this VR app, I realize that many of the complaints about the VR app are due to the fault of my own technical ability in Unity and C#. I believe that VR should have been a significant improvement to drawing the model from a 2D image. 

After recognizing the limitations myself and VRStudio, I turned to more experienced developers. Gesture VR is an exciting new VR drawing app. The app has a wide range of tools, such as pressure sensitivity and variable size charcoal and eraser, and allows for adjustments to lighting angle and intensity.

~25 minute sketch done in Gesture VR

Many complaints found on the response form to the VRStudio demo were alleviated here.  I found the app engaging and exciting to use while also being responsive and detailed, which leads me to believe that I am correct in assuming that the results of the in-class activity may have been biased by the fact that participants found it hard to engage with an app that wasn't behaving smoothly or responding to their commands accurately. 

 However, there are limitations to the app, such as the inability to add color or use different mediums, and difficulties in measuring form due to the inability to mark pencil with thumb while holding it straight out. Additionally, without a physical surface, hands can be shaky and pressure can be unreliable, making detail work difficult. Large sweeping strokes were more feasible. This means unfortunately VR is probably not best for scientific drawings despite its potential in getting rare specifies documented. As initially suspected in my development of VRStudio, VR may be a medium more suited to loose sketching and light study. 

Overall, the use of VR for visual art has the potential to provide a unique and immersive experience that cannot be achieved with flat reference images. However, the technology is still developing and has its limitations, particularly in terms of control and precision. VR is already valuable to artists, even in the most limited, primitive of forms, and it will only continue to improve from here. VRStudio can provide a foundation for curious developers and users to work off of towards that future.