ARCoachingOverlayView and SwiftUI

These last few weeks I’ve been diving into Apple’s RealityKit for iPhone. I’ve chosen to use SwiftUI instead of UIKit as I like the similarities between it and ReactJS. I thought I’d start posting a few things I learn as I find Apple’s documentation is great if you already know what you’re doing. Unfortunately, if I knew what I was doing I probably wouldn’t use the documentation.

This brings us to ARCoachingOverlayView which helps with the onboarding of a user to an AR app. It basically tells the user to start moving the phone so that the underlying engine can start recognizing anchors, positions, and objects in the scene. Using this with SwiftUI is as simple as making an extension.

I created the following extension:

extension ARView: ARCoachingOverlayViewDelegate {
    func addCoaching() {
        let coachingOverlay = ARCoachingOverlayView()
        coachingOverlay.delegate = self
        #if !targetEnvironment(simulator)
        coachingOverlay.session = self.session
        #endif
        coachingOverlay.autoresizingMask = [.flexibleWidth, .flexibleHeight]
        coachingOverlay.goal = .horizontalPlane
        self.addSubview(coachingOverlay)
    }
    
    public func coachingOverlayViewDidDeactivate(_ coachingOverlayView: ARCoachingOverlayView) {
        print("did deactivate")
    }
}

A few things to note about this code:

• We add a function addCoaching() to the extension to show the view and add to the ARView’s session.
• We use the #if !targetEnvironment(simulator) … #endif to ensure this doesn’t break our simulator view. Otherwise we get errors.

Using the standard RealityKit demo, it gives us a struct ARViewContainer: UIViewRepresentable with the function makeUIView defined. In here we add the login to start the coaching view:

var arView : ARView
#if !targetEnvironment(simulator)
arView.addCoaching()
#endif

Notice that we once again enclose this in the if statement so the simulator can still compile. Now this works on our app:

Now we see the overlay view pop up when the app starts.

Apple is still in the early days of AR with their new RealityKit framework but I am excited about the possibilities. The documentation, as with most Apple documentation, leaves a lot to be desired, but I’m excited enough to try to bang my head through it and start developing more on this platform.