3D Modeling in VR

By Alejandro Romero

Overview

This activity will allow you to download and run a 3D Modeling demo in VR that makes use of hand tracking. You will be asked to test and evaluate certain features in the software at the end of the activity,

Setup

Prepare your Oculus for Dev Mode and Hand Tracking (~10 minutes)

Before downloading the .apk file to run the program, you will first need to ensure some settings on your Oculus are enabled. If your Oculus Quest is already configured to support dev mode and hand tracking, you can skip this step.


Set up Dev Mode (~8 minutes):

  1. In order to use your Quest in Developer Mode (which is required for sideloading), you’ll need to register as a developer organization

  2. Visit this page and make sure you’re logged into the same Oculus account that your Quest is registered to. Enter a new organization name and tick the ‘I understand’ box to agree to the Oculus Terms of Service.

  3. Open the Oculus app on your phone and make sure your Quest is turned on.

  4. Tap ‘Settings’ in the bottom right

  5. Locate your Oculus Quest listed in the Settings tab, and make sure it reads ‘Connected'

  6. Tap on the arrow button next to your device, to reveal more options

  7. Tap the ‘More Settings’ button

  8. Tap on ‘Developer Mode’

  9. Flick the switch to On instead of Off

  10. Fully reboot your Quest – hold down the power button on the side and select ‘Power Off’ or ‘Restart’. You Quest should fully reboot in order to properly start in Developer Mode

  11. After rebooting, your Quest should be in Developer Mode


Set up Hand Tracking (~1 minute):

  1. Press the Oculus button on your right controller to pull up the universal menu

  2. Select the Settings gear

  3. In the panel to your left, select Devices

  4. Select Hands and Controllers

  5. In the Hand Tracking section, use the toggles next to Hand Tracking to turn the feature on or off

  6. You will also need to turn the auto switching feature of hand tracking on, allowing your headset to automatically switch between using your hands or controllers

Install and Run .apk File Using SideQuest (~10 minutes)

If you already have SideQuest up and running, skip the first step and go to Installing and Running the .apk File


Set up SideQuest:

  1. If you don't have it already, download SideQuest onto your local machine using https://sidequestvr.com/download

  2. Open SideQuest on your computer

  3. Turn on your Quest and set up a guardian

  4. Plug your Quest into your computer

  5. Inside your Quest, not on your computer, you should see a request to “Allow USB debugging.”

  6. If you don’t want to have to press OK on this screen every time you plug your Quest into your computer, make sure the ‘Always allow from this computer’ box is ticked.

  7. Press OK

  8. Congratulations! Your Quest and PC should now be set up to easily sideload apps via SideQuest


Installing and Running the .apk File:

  1. Open SideQuest on your computer

  2. Download the .apk file from this Google Drive link

  3. Make sure your Quest is turned on connected to SideQuest (denoted by the green circle in the top left)

  4. In SideQuest, go to the download APK file icon in the top right and find the directory with the installed APK file

5. Select the file. It will install to your Quest

6. On the Quest, go to Apps > Unknown Sources

7. Open the app named Hand-Tracked-Modeling-VR [com.alejandroromero.HandTrackedModelingVR]

8. You are now in the program!


Activity 1

3D Modeling with Hand Tracking

  1. You might spawn in on top of an orange cube. Move around via teleportation by turning your hand palm-up and folding in all your fingers except for your index finger and thumb. You should see a teleportation ray. Fold up your index finger in a "come over here" motion to execute the teleportation

  2. Teleport to the menu where you can turn off the Profiler

    1. This will be a floating blue rectangle with various buttons on it. The profiler is the window telling you details about the software like CPU usage and frame rate.

  3. Take a look at the different hand control options

  4. When you're ready, try editing the orange cube

    1. You can tap on the color button to pull up a color picker menu. You can grab it and move it around

    2. You can also move the whole cube, or move individual vertices

    3. Flip your hand palm-up. This should pull up the hand menu, where you can toggle between edit and object mode. Note that there is a menu for adding primitives. It doesn't work as of yet, but should give you an idea of the UI.

      • Edit mode: lets you move vertices without moving the mesh's location

      • Object mode: lets you move the mesh without editing vertices

      • Both: move the mesh or the vertices. Can be a little buggy.

  5. You can rotate or scale a mesh by grabbing it with two hands. To rotate, spin your hands around like you would with an actual object. To scale, pull your hands further or closer together.

  6. You can also scale, translate, and rotate objects without being close to them. If you hold your index finger and thumb close together, you can see a ray shooting out in front of you. Pinch your index and thumb to grab

  7. Try to make a heart out of the cube! If this is too easy, or you can't quite get it down, see what you can make out of the cube!

  8. Include screenshots on the class board


Activity 2

3D Modeling with Controllers

  1. Now that you got a feel for hand tracking, grab your controllers. The program should automatically switch to using your controller input (the hands will look rigid)

  2. Try to replicate your shape from before! Note: The hand menu might not work with controllers, so try to stay in edit mode when switching to controllers

  3. Include screenshots on the class board


Survey

Once you're done with the activities, please fill out this Google form!

https://forms.gle/qMV4r2AVR8RHncvAA