Native Point Cloud Visualization for Unreal Engine 5
Written By Austin Phan, Spring 2023
Based on Unreal Engine 5.0.3
Intro
Unreal Engine has a very useful plugin for visualizing point clouds! It's called the LiDAR Point Cloud plugin, which'll help you natively import point clouds into your world, for visualization in VR.
You can find good documentation here, but below I'll go through a quick tutorial on how to get everything setup and working, as well as note some of the quirks related to the plugin.
Example Dataset
I've taken the open source dataset from PandaSet, which includes point cloud information gathered by LiDAR scanners mounted on a moving vehicle.
I wrote a short Python Script that was able to open a dataset and load the point cloud coordinates into a .txt file.
It looks like this (but it's 169,171 lines long):
The first 3 values are the x, y, and z coordinates, with the last 3 being the r, g, and b, values for a given point. The script I wrote calculates the color based on the distance from 0, 0, 0, or the location of the sensor.
There are two ways to proceed. Option A, you can begin with a VR template, or Option B, you can begin with a blank world and add VR
Importing LiDAR into a world
Open Unreal Engine, and load in a world. If you're starting out and just want a basic world, you can use the VR template, or a blank template. I'll use a blank world here (any settings are fine for the purposes of this tutorial). Here's what I'll use:
Once you're in the world, you'll need to enable the LiDAR point cloud plugin.
Go to Edit > Plugins
Type in the searchbar 'lidar'
You should be presented with the 'LiDAR Point Cloud Support' plugin.
Enable it. If it's in beta in your given version, you'll need need to accept the extra dialogue that pops up.
A popup should appear at the bottom for you to restart Unreal Editor to enable the plugin. Restart now.
Restarting may take a while, if shaders need to be recompiled.
LiDAR plugin in plugins menu.
Warning dialog that pops up in the case of beta version of plugin.
Once the plugin is enabled, and UE5 reloads, you will be able to import a dataset into the editor.
Open the content drawer, and select import.
Browse to the location of your datafile (ones supported by the point cloud dataset, found here). Select import. I'll show an example with the .txt I showed earlier.
Select the file, and an import dialog will open in the editor.
Content browser open, select import to find the datafile you want to import.
Import point cloud options.
While UE5 gets it right most of the time making some selections to the import cloud options can be helpful in assisting it to find the corresponding value for each variable.
For example, my .txt file has X, Y, Z, R, G, and B, point values. Assigning the Loc X, Loc Y, Loc Z, Red, Blue, Green values to the right values can help UE5 better import the values more accurately. It'll read the first line of values, and you just have to assign the corresponding values.
For example, the first line in my dataset is:
-58.45719909667969 -61.26926803588867 2.881014347076416 97.40891796396164 255.0 0.0
Selecting in the dropdown those datapoints will be helpful for UE5.
The last thing is the color range sliders, the left being min, right being max. If you have color data in your file, you can set the max color value if known, or select scan/match to let the plugin find the largest/smallest values.
After that, you can simply press 'import' and the file will be imported into the corresponding area in your content browser.
Imported point cloud object
After that, simply drag the point cloud object into your world viewport, and voila! You have a point cloud dataset. You may need to move the data around, it may be partially sunken into the ground. Here's our newly-imported point cloud!
Additional Appearance Settings
There are some extra settings you can change to change the appearance of your dataset in VR. Here's a quick rundown:
Point size:
Sets a custom scaling size for the points. Below 1 is smaller, above 1 is larger.
Scaling Method:
Sets scaling method for nodes based on their densities. Hover over each to learn more.
Gap Filling Strength
The plugin will attempt to fill gaps between dots if set above 0.
Color Source:
Sets how the data is colored. If you want to custom color, and you set it in your datafile, use Data. You can also color based on elevation, or classification, amongst other things.
Point Shape:
Sets shape of each point.
Point Orientation:
Sets how points are oriented in relation to the user.
Shortcomings of the LiDAR point cloud plugin
I experienced issues with later versions of Unreal Engine 5. Version 5.1.1, the latest version at the time of writing, continued to crash when enabled in a VR world. (5.0.3 works though!)
The LiDAR point cloud plugin does not support packaging as a .apk for android use (i.e., sideloading it onto Quest). You will be forced to run VR off of another machine, rather than running it natively on Meta Quest.