Hand-tracking In-Class Activity
Introduction
The advent of immersive technology has brought about a significant change in the way we interact with digital content. With the Oculus Quest 2, users have the ability to navigate and manipulate virtual environments using both traditional controllers and advanced hand tracking technology.
The traditional controller-based interface uses handheld devices that the headset tracks in real-time, translating their position and orientation into the virtual environment. This interface is familiar to most users and allows for a high degree of precision, making it ideal for complex games and applications.
On the other hand, hand tracking technology represents a newer and more natural form of interaction. Instead of using physical controllers, the headset uses built-in cameras to track the position and movement of the user's hands and fingers. This not only allows for more intuitive interaction but also offers an enhanced sense of immersion, as users can see and use their hands directly within the VR space.
In this in-class activity, we will experience both and compare them while interacting with objects, climbing a ladder, simulating the physics of the world.
Instruction
Visit https://hand-tracking-scene.glitch.me/ in Oculus Browser . Tap "VR" on the bottom right to start interacting with the scene!
The demo support both controller and hand-tracking, make sure the headset is using the control you want before you get onto the website.
Before reading the instructions below, I strongly recommend try playing in the scene first with hand-tracking, then compare with the instruction below to see what you didn't figure out; after figuring out all the interactable in the scene, switch to controller and go over againl.
* Multi-user room is not supported for now, since the scene does not load correctly.
Navigation
Hand-tracking: Only teleporting is supported. Move by pointing your right hand index finger to a location you want to go to.
Controller: You can switch two different kind of navigation mode at the setting menu on your left wrist. Teleport/Move with left joystick, and change view direction with the right joystick
Button on the desk: try to push down the button the change the color of the cube.
Pot on the desk: try to pick up the pot by holding one of the handle; you can also release it and see it falling down.
Ladder: climb the ladder with both hands! When your controller/hand is near the ladder, it will be registered on the "stair". Then pull yourself up, release one hand and grab another stair. You should be able to climb on to the higher surface and stand on it.
Clock: the clock is a example object with physics. You can knock it over : )
Piano: the piano is a imported model from a glb file, it's not interactable, but you can view it in both AR and VR mode.
More Demo on Hand Mesh and Physics
If you want to experience more with hand-tracking mode, here's two simple demo on hand representation and physics interaction:
Hand mesh representation and recognition: https://handtracking-mesh.glitch.me/
Physics simulation: https://handtracking-physics.glitch.me/