How to Create an Augmented Reality App in Unity

How to Create an Augmented Reality App in Unity

Introduction

Welcome to the tutorial, where we’re going to create an augmented reality (AR) app inside of the Unity game engine. You’ll learn about installing Unity, setting up AR and building it to your Android or iOS device.

AR app with reticle being moved around

Project Overview

In this project, we’ll be creating an augmented reality app for our Android or iOS device. Specifically, this app will feature a reticle that snaps to real-world surfaces.

You can download the complete project here.

What is Augmented Reality (AR)?

Augmented reality (AR), is the real world with virtual elements added in. This isn’t the same as virtual reality (VR), which creates an entirely virtual experience. AR augments your existing one.

What is Unity?

Unity, is one of the most popular game engines available today. It allows you to create 3D, 2D, AR, VR, mobile, console – really any type of game for any platform you can think of. With Unity being so versatile, there’s a large amount of documentation and help available online, including this tutorial.

What is AR Foundation?

Right now, there are two main AR SDK’s available. ARCore for Android and ARKit for iOS. AR Foundation is an API which allows the developer to create AR apps for both platforms seamlessly. Developers don’t need to create separate projects for Android and iOS, or have even more convoluted code. It’s all packaged into one with AR Foundation.

So what can we do with AR Foundation?

  • Detect surfaces
  • Identify feature points in the world
  • Track virtual objects in the real world
  • Adjust the lighting in Unity based on a real world estimate
  • Face, image and object tracking
  • Unity documentation

Due to these advanced features, there are restrictions to which devices can actually run this.

Installing Unity

Start of by going to https://unity3d.com/get-unity/download and click Download Unity Hub. The Unity Hub is where we can manage our projects and versions of Unity.

Download Unity Hub

Download the installer and install the Unity Hub. When you open it up, let’s navigate to the Installs tab (you may have to setup a license). Here, click on Add. The image below shows 4 versions of Unity already installed (your page will be blank).

Installing a new version of Unity on the Unity Hub

A new window will pop-up. Select the most recent version of Unity (not including versions with a b or a (beta and alpha), these can be unstable). Then click Next.

Add Unity version

Next is the modules screen. These are basically different platforms we can build to. Unity includes the standard ones (PC, Mac, Linux, Web) but we can choose extras to add. If you want to build for Android, select the three Android Build Supports. If you want to build for iOS, select the iOS Build Support. Then click Next. Do also note, that if you’re on a Windows computer – you cannot build to iOS.

Adding Android or iOS modules to Unity

This will begin to install Unity along with your extra modules.

Setting up AR Foundation

To begin, open the Unity Hub, go to the Projects tab and click on the New button.

Creating new project on Unity Hub

AR Foundation requires a minimum Unity version of 2019.1, although it’s recommended to use the latest version. Make sure your project has the 3D template selected, then click Create.

New Unity project 3D

In your project, let’s navigate to the Package Manager window (Window > Package Manager). Wait a few seconds for the packages to load, then we need to install…

  • AR Foundation
  • ARCore XR Plugin (if you’re developing for Android)
  • ARKit XR Plugin (if you’re developing for iOS)

Unity package manager

We can now close that window. Next, let’s go over to the Hierarchy (a list of all the objects in our current scene) and delete the MainCamera object. This is because we’re now going to add in 2 new objects (which includes an AR camera). Right click the Hierarchy and select:

  • XR > AR Session Origin
  • XR > AR Session

The AR Session Origin defines the center of the AR world. It also has an AR camera as a child which has the scripts needed to work in AR. Then we create the AR Session object. This basically runs the AR session and detects inputs.

Unity hierarchy AR foundation

Select the AR Session Origin object and look over in the Inspector. Click Add Component and add two components:

  • AR Raycast Manager
  • AR Plane Manager

Unity inspector ar session origin

Before we continue, let’s switch our platform over to whatever device we’re building for. Open up the Build Settings window (File > Build Settings…).

First, let’s click the Add Open Scenes button to add the current scene to the build list. Then find your platform (Android or iOS) in the list. Select it and then click Switch Platform.

Unity build settings android ios

Creating the Placement Indicator

In this tutorial, we’re going to be creating a reticle that will snap to detected surfaces. You can use this to spawn in objects, aim at things, etc.

In the downloadable files, there is a PNG file called PlacementMarkerBase. Drag this into the Project panel in Unity.

Unity project window panel

Select the file and you should see that some import settings appear in the Inspector panel. All we need to do here is enable Alpha is Transparency, then click Apply. You should no longer see the black background.

Unity inspector image import settings

In Unity, if you want something to be visible, have a texture, color, shininess, etc – it needs a material. Right click in the Project panel and select Create > Material. Call this PlacementIndicator.

Unity project panel window

Let’s now edit the material. Select it and in the Inspector, we can set the properties.

  • Set the Shader to Unlit/Transparent
  • Drag the PlacementIndicator.PNG into the texture field

Unity inspector material

Now we can add the in-game object. In the Hierarchy, right click and select Create Empty. Rename this object to PlacementIndicator. If you select it, in the Inspector, set the Position to 0, 0, 0.

Unity editor

Right click the object in the Hierarchy and select 3D Object > Plane. This will create a white plane object. Drag the material we just made onto it to apply. You might see that the object has a MeshCollider component attached to it. In the Inspector, right click on the collider’s name and select Remove Component.

Unity editor plane material

Right now, our placement indicator would be very large in AR. To make it more manageable and around 1×1 inch, select the Plane and set the Scale to 0.01, 0.01, 0.01.

Unity inspector plane scale

Scripting the Placement Indicator

Next, let’s create the script which will be the code controlling the placement indicator. In the Project panel, right click and select Create > C# Script. Call this PlacementIndicator.

Unity project panel window c# script

Let’s attach the script to the object. Select the PlacementIndicator and drag the script into the Inspector. You should see the script component attached.

Unity attach script

Double click on the script file to open it up in Visual Studio. You should have the default script template inside.

  • The public class PlacementIndicator : MonoBehaviour code is where the contents of the script are contained
  • The Start function gets called once when the object is initialized (at the start of the game)
  • The Update function gets called every frame (e.g. 60 times a second if the app is running at 60 FPS)

Above the class, you should see three lines of code with using before it. These are external libraries we’re using. We need to add two more libraries specifically for AR Foundation.

Back down in our class, let’s start by adding in some variables to keep track of.

Inside of the Start function we can set the two variables and start off by hiding the placement indicator visual.

Then down in the Update function, we’re going to be shooting a raycast from the center of our screen. A raycast is basically a line that can detect objects (think shooting a bullet). We give it an origin position and direction and it will then tell us what it hits. We’re looking to hit an AR plane (automatically generated when it detects the world).

Let’s start by creating a list to hold all of the AR planes we hit. Then we can cast the raycast.

Under that, let’s check if we hit anything.

Inside of the if statement, let’s set the position and rotation of the placement indicator to be where the raycast hit is.

Let’s also enable the visual if it’s disabled.

The script is now complete. We can return back to the editor now.

Building to Android

If you have an Android device and want to build your app to it, follow these steps. First, we need to enable Developer Options on your device.

  • Open the Settings app and select the System button
  • Scroll to the bottom and select About Phone
  • Tab the Build Number 7 times
  • Now there should be a Developer Options tab in the settings app
  • Go into that and enable USB Debugging

Back in Unity, let’s open up the Build Settings window (File > Build Settings…). Click Refresh then find your plugged in device. After that, click Player Settings to open the Player Settings window.

Unity build settings

  • Go to the Player screen
  • The Product Name is what the app will be called on your device
  • In the Other Settings tab…
  • Enable Auto Graphics API
  • Disable Multithread Rendering

Unity project player settings

Scroll down a bit and create a Package Name. This is a unique identifier for your app. The format is com.CompanyName.ProductName. This can really be whatever you want, just follow the structure.

Also set the Minimum API Level to Android 7.0 “Nougat” (API level 24).

Unity player settings package name

Now go to the XR tab and click the Create button to create a new AR Core setting asset. Save this in the Assets folder. Then the screen should change. Set Requirement to Optional.

Unity player settings arcore create

Let’s go back to the Build Settings window and click Build and Run. You can save the .apk where ever you want as it’s automatically going to install onto our device.

Unity build settings window

When the app launches, it should only take a second or so for the placement indicator to appear.

AR foundation placement indicator

Building to iOS

Building to iOS is similar to Android, yet it includes a few extra steps. First, we need to install an app called Xcode. You can download it from the App Store or Apple website.

Inside of Unity, we first need to change our build platform. Go to the Build Settings window (File > Build Settings…). Select the iOS platform and click Switch Platform.

Then click on the Player Settings button to go to the Project Settings window. Here, make sure you’re on the Player tab.

  • Set the Product Name to what you want the app to be called on your device
  • In Other Settings, make sure Auto Graphics API is enabled

Unity player settings

Scroll down a bit more as we need to change a few more things.

  • Set the Bundle Identifier to a unique identifier for your app. The format is normally com.CompanyName.ProductName
  • Set the Camera Usage Description to what we want to use the camera for
  • Set the Architecture to ARM64

Unity player settings ios

Now click on the XR > ARKit tab to go to the ARKit screen. Here, we want to click Create and save the file to the Assets folder.

Unity player settings arkit create

After that, the screen will change so set the Requirement to Optional.

Unity project settings arkit

Now we can build our app. Go back to the Build Settings window (File > Build Settings…) and click Build. Save where ever you want.

Unity build settings ios

After the project has been built, let’s go to that folder and double-click on the Unity-iPhone.xcodeproj file to open it up in Xcode.

iOS finder xcode project

Inside of Xcode is where we publish to our device. To open the project, click on Unity-iPhone in the top left.

Xcode project unity

In order to build to a device, we need an account connected. Go to the Preferences window (Xcode > Preferences) and then click on the Accounts tab. Here, let’s click on the ‘ + ‘ icon and select Apple ID. Login and then you should see your account pop-up in the list.

Xcode unity account

Back in the main Xcode window, let’s enable Automatically manage signing.

Xcode unity auto manage

Then, set your Team to your account. Now we’re good to go, so make sure your device is connected and click the Play button to begin the build.

Xcode unity

When the app launches, it should only take a second or two for the placement indicator to appear.

Unity ar foundation placement indicator

Conclusion

Congratulations on finishing the tutorial! You now know the basics of creating an augmented reality app in Unity. From here, you can make any sort of AR app you wish. With AR Foundation, there are many different features for you to explore.

If you wish to download the completed project, you can get it from here.