Nvidia Instant NeRF

By Yuanbo Li, 2023 Spring


This is a brief introduction of how to install and use Nvidia Instant NeRF and use in VR.

Data Preparation

Time Requirement: 5mins

Hardware Requirement: IOS > 16.1

Step 1: Download Record3D from Apple Store.

Step 2: Record the object

Choose your favorite object and open Record3D. Click on the recording button below. Walk slowly around the object, you will see an UI like below.

Record3D uses Depth Map to record the depth and intrinsics of the recoding. So you will automatically have the result paired with the camera positioning. 

Step 3: Export your object

Export your object to .r3d format and share it with yourself.

Train your own NeRF for VR!

Configuring Instant NeRF can be extremely challenging if you have a different hardware than required.......

Time Requirement: 40mins -- 10 hours

Hardware Requirement: Nvidia GPU

Step 1: Download Instant NeRF from Github.

Here is the official Github Repo. Please follow the guidelines. 

This video might be helpful, if the github repo guideline is too hard to follow.

Step 2: Prepare your dataset

We have already got the .r3d file from the previous section. However, it is far from making it available to train on Instant NeRF. Try this tutorial

Step3: See your result! 

Now you can view your data in Instant NeRF. However, it might be possible that you want to make some small edits. Try this tutorial.

Below is my result. Click on "Conect to VR/AR headset" button can allow you to view it in Occulus. 

How to the visualize 3D object in VR?

Click here to see which stratgey you want to use

Connect to VR in Nvidia Instant NeRF

Connect your headset to the computer with Nvidia Instant NeRF running.

Click on "connect to VR/AR" button on the top left. Then you can view the object in VR.

 (Screen shot of VR view is on the left)

Export Mesh and Visualize in Unity

You can also choose the "Mesh it" button in the Export Mesh column and then save it to your local computer. You will have a .obj file. Drag it into your Unity App, and you will have a pale object like this.