plane tracking and objects

1. Begin by opening up Xcode, pressing Create a new Xcode project, and selecting "Augmented Reality App." In the option menu that appears, most of the information should be filled out for you—just give your project a name like plane_tracking, and make sure to select Personal Team under the Team dropdown menu. We'll be using Apple's SceneKit (under Content Technology), which makes incorporating 3D animated objects/scenes/effects less tedious.

Your screen should look something like Figure 1. Press next and save the project in a folder somewhere.

Figure 1: Project options

2. The two main files here are AppDelegate.swift and ViewController.swift. AppDelegate is kind of the "controller" for app, checking system states/interruptions such as active/inactive state or notifications. We won't have to worry about this file in this project.

ViewController.swift is where we create the meat of our project. By default, the scene will render an airplane, whose texture is rendered in the function viewDidLoad(). This function is called after the viewWillAppear() function, which sets up the view controller (e.g. stuff related to orientation, tracking mode, etc.).

Starting from the very beginning, we set our viewWillAppear() function to allow horizontal plane tracking:

override func viewWillAppear(_ animated: Bool) {


// Create a session configuration

let configuration = ARWorldTrackingConfiguration()

configuration.planeDetection = .horizontal

// Run the view's session


Now, in viewDidLoad, simply create and set the scene:

override func viewDidLoad() {


// Set the view's delegate

sceneView.delegate = self

// Create a new scene

let scene = SCNScene()

// Set the scene to the view

sceneView.scene = scene


Most of this should have already been in the existing functions.

3. Now we'll create the plane object that the app will be overlaying on the scene as the phone pans. Create a new Swift file called Plane.swift.

class Plane: SCNNode {

var anchor: ARPlaneAnchor!

var planeGeometry: SCNPlane!


Here, we use the SCNNode class, which stands for a SceneKit node. This is just an object that we can attach geometry and other displayable elements to. The ARPlaneAnchor gives us the information of the plane that we're detecting, and a SCNPlane is the most basic of planes, a flat 2D plane with fixed size.

4. When the AppDelegate gets a call to renderer (which we'll implement later), the Plane class is supposed to create a new new plane that is ready to display. To do so, we implement the plane class's init() function:

init(anchor: ARPlaneAnchor) {


// initialize the anchor and define dimensions for the plane

self.anchor = anchor

self.planeGeometry = SCNPlane(width: CGFloat(anchor.extent.x), height: CGFloat(anchor.extent.z))

// initialize material of plane so we can see it

let material = SCNMaterial()

material.diffuse.contents = UIColor.white.withAlphaComponent(0.50)

self.planeGeometry!.materials = [material]

// create the SceneKit plane node. As planes in SceneKit are vertical, we need to initialize the y coordinate to 0, use the z coordinate, and rotate it 90º.

let planeNode = SCNNode(geometry: self.planeGeometry)

planeNode.position = SCNVector3(, 0,

planeNode.transform = SCNMatrix4MakeRotation(-Float.pi / 2.0, 1.0, 0.0, 0.0)

// update the material representation for this plane


// add this node to our hierarchy



This sets a material and dimensions for the plane, then add the node to the Plane object.

5. As we move around, however, we have to make sure to update the plane, which includes updating the dimensions of the plane as the anchor reports different dimensions. The function below simply updates the material dimensions to scale per the object's self-reported geometry.

func updatePlaneMaterialDimensions() {

let material = self.planeGeometry.materials.first!

// scale material to width and height of the updated plane

let width = Float(self.planeGeometry.width)

let height = Float(self.planeGeometry.height)

material.diffuse.contentsTransform = SCNMatrix4MakeScale(width, height, 1.0)


Now, as we move around, the anchors will change. Every time a new anchor is added, indicating a new plane, we must poll the dimensions and position of the anchor, using updatePlaneMaterialDimensions as needed to render the plane.

func updateWithNewAnchor(_ anchor: ARPlaneAnchor) {

// first, we update the extent of the plane, because it might have changed

self.planeGeometry.width = CGFloat(anchor.extent.x)

self.planeGeometry.height = CGFloat(anchor.extent.z)

// now we should update the position (remember the transform applied)

self.position = SCNVector3(, 0,

// update the material for this plane



6. Now, back in ViewController, there should be a commented out renderer() method. We actually have to create three different versions of this method, one for each of adding a new anchor, updating an anchor, and removing an anchor. As we move, more and more anchors will be added and rendered. To keep track, we'll use a dictionary object with a UUID. Define var planes = [UUID: Plane]() at the top of ViewController.

Now our add anchor function should look something like this:

func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {

if let arPlaneAnchor = anchor as? ARPlaneAnchor {

let plane = Plane(anchor: arPlaneAnchor)

self.planes[arPlaneAnchor.identifier] = plane




Updating and removing are self-explanatory:

func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {

if let arPlaneAnchor = anchor as? ARPlaneAnchor, let plane = planes[arPlaneAnchor.identifier] {




func renderer(_ renderer: SCNSceneRenderer, didRemove node: SCNNode, for anchor: ARAnchor) {

if let arPlaneAnchor = anchor as? ARPlaneAnchor, let index = planes.index(forKey: arPlaneAnchor.identifier) {

planes.remove(at: index)



And that's it for plane detection! Try running it on your phone. In Xcode, under Product > Destination, you should see the name of your connected device. Select it, then press Command-R to sideload the app to your device and run it from there.