TiltBrush, Virtual Music, and Spotify for Audio Visualization in VR
Introduction
The goal of this post is to 1) provide an overview of 2 VR audio visualization applications: TiltBrush and Virtual Music, 2) provide an overview of the audio data offered via the Spotify Web API, and 3) provide a proposal on how Spotify data can be used to improve audio visualization applications.
Tilt Brush
Tilt Brush is a VR painting tool developed by Google with audio visualization features. Users can paint with special ink that pulsates with the frequency of the audio (as seen in the picture below) and plans on supporting more audio reactive features in the future. Tilt Brush is available online via Steam, HumbleStore, Vive, and Oculus.
What's Missing?
More Data!
Spotify Web API
In order to improve the VR audio visualization user experience let's consider leveraging the abundance of audio data available online! Spotify's Web API provides the following functionality to be utilized in a VR application:
User authentication grants an application access to someone’s likes, playlists, and stations. Spotify Web Playback SDK offers in-browser audio playback for web applications. Spotify audio feature data rates tracks on different dimensions such as danceability, energy, speechiness, acousticness, instrumentalness, and liveness. Spotify’s audio segment analysis provides data on the similarity of audio segments from a track based on machine-learning clustering techniques. As a bonus, the API is well documented and easy to use.
Virtual Music
Virtual Music is a VR audio visualization application free for download on the Oculus Go app store. Users can manually load music and become immersed in visualizations of audio frequency. Virtual Music provides a number of visualization configurations as seen in the pictures shown below.
Next Steps
•Improved User Experience – incorporate search, user’s playlists, and radio stations into application. Develop compatibility with VR headsets (Oculus, Vive, etc.).
•Music Discovery – Use Spotify’s audio feature analysis data to plot track dimensions in 3D (similar to the plot shown below). Group songs by region for immersive music discovery experience.
•Audio Manipulation - Use Spotify’s audio analysis data to allow users to splice and mix-up tracks and become a DJ instantly!
