Create a Crane Training App in AR – Part 2

Intro to Part 2

Check out the other part of this series: Part 1

When we left off, we had a working crane controller that could rotate and move the hook around. Now, we are going to make it so the crane can pick up a container and place it in the truck.

FREE COURSES
Python Blog Image

FINAL DAYS: Unlock coding courses in Unity, Godot, Unreal, Python and more.

Creating the Container and Truck

Now it’s time to create the container and truck. The container will be an object that the crane can pick up and place in the truck.

First, go to the “Models” folder and drag the “Container” model into the container image target object, and drag the “Truck” model into the truck image target object. You will need to resize these as they will be fairly large when dragging them in. In the image below, I made sure the container could fit in the back of the truck, which I made a bit wider.

Container and Truck objects in Unity Crane app

We’re going to start with the container. First, we need to add a Box Collider to the object, so that it will be able to interact with the hook. You can resize the collider by changing the “Size” values, or you can click on the “Edit Collider” button to directly resize it in the scene. We also need to make sure that we tick “Is Trigger” as we’re going to be checking for trigger enters instead of collisions.

Container object with Box Collider highlighted in Unity Inspector

We also need to add a Rigidbody component to the container. This is because for a trigger or collision event to happen, at least one of the objects must have a Rigidbody component attached. But since we’re not going to be using physics in this app, we can disable “Use Gravity” and enable “Is Kinematic”. This basically makes it so no physics like gravity or other forces is applied to the object.

Container object in Unity Inspector with Rigidbody

The container is also going to be the main object that manages collisions between the hook to the container and the container to the truck. For this we need to create a new C# script called “Container” and attach it to the container object, along with the other components we just added.

After opening it, we need to create our only variable. “inTruck” is a boolean which becomes true once the container is placed in the truck. We’re doing this so the container can’t be picked up once it’s in the truck.

private bool inTruck = false;   // is the container currently in the truck?

Next we need to add in the OnTriggerEnter function. When typing it in, you could be able to auto-complete it which will create something similar to code below. If not, then you must type in the function as written down below. This is because it’s a specific event that gets called when the container’s Box Collider enters another objects collider.

// called when we enter the collider of another object
void OnTriggerEnter (Collider other)
{

}

Inside the OnTriggerEnter function, we want to make sure that if we’re already in the truck, we just return. As if that’s the case, we don’t need to worry about any more collisions.

// if we're in the truck, then don't worry about triggering anything
if (inTruck)
    return;

Our first collision check is with the “Hook” we haven’t made any tags yet, so we’ll do that after we finish the script. But to let you know, there’s only going to be 2 tags. “Hook” is going to be the tag for the hook object, and “Truck” is going to be the tag for the truck object.

The code below first of all checks if the object we collided with has the tag of “Hook”. If it does, then we set our position to the hook’s “ContainerHoldPosition” which we set up earlier. As well as this, we will make the container a child of the hook so that it follows along when it moves.

// was it the hook that hit us?
if(other.CompareTag("Hook"))
{
    // attach to the hook
    transform.position = other.transform.Find("ContainerHoldPosition").position;
    transform.parent = other.transform;
}

Our next collision check is going to be just under the previous and that will check for a collision with the truck.

We check if the collided object is tagged as “Truck”. If so, we set our position to be the “ContainerHoldPosition” of the truck. Now we haven’t set that up yet, but we will do shortly. Note that this isn’t the same “ContainerHoldPosition” as the hook. We then set our rotation to be the same as the truck so it fits nicely in the back, we parent the container to the truck and set “inTruck” to true, so no more collisions can take place.

// was it the truck?
else if(other.CompareTag("Truck"))
{
    // attach to the truck
    transform.position = other.transform.Find("ContainerHoldPosition").position;
    transform.rotation = other.transform.rotation;
    transform.parent = other.transform;
    inTruck = true;
}

Now going back the editor, we can work on setting up the truck.

First, we’re going to create an empty GameObject and call it “ContainerHoldPosition”. This will be a child of the “Truck” object and be the position where the container will go. In the image below, I use the container as reference to show me how to position the “ContainerHoldPosition”.

Container object on truck object in Unity crane app

Then we need to add a BoxCollider component to the truck as this is what the container will check for when it’s being placed on the back. Resize it and re position it like before with the container. Make sure that this collider is also enabled as “Is Trigger”.

Unity Crane App's truck object with Box Collider

Now for all the collisions to work, we need to create tags, as that it what we’re looking for when colliding with an object.

First, select any object and at the top underneath the name, click on the “Tag” dropdown. Select “Add Tag…”.

Unity Inspector with Add Tag option highlight from Tag menu

We want to add 2 tags. “Hook” and “Truck”. To do this, simply click on the plus sign at the bottom right, type in the tag name and then click “Save”.

Unity Tags with Truck and Hook Tags shown

Now we can add these tags to the objects. First, find the “Hook” object (make sure it’s the one with the collider) and set the tag to “Hook”.

Hook object in Unity app with Hook tag added

Then find the truck object (one with the collider) and set its tag to “Truck”.

Truck object in Unity app with Truck tag added

Before we can test it, we need to add a collider to the hook. Click on the “Hook” object and add a SphereCollider. Set “Is Trigger” to true and resize and reposition it to your liking.

Unity crane object with Sphere Collider added

You should now be able to press play and test it out. The crane should be able to pick up the container and place it in the back of the truck.

Unity crane app with crane picking up container

Creating the Dashboard

Now something cool that we’re going to be doing with this project, is having an AR dashboard that you can reposition to where you like.

To start, I moved the dashboard image target away a bit from the others, due to the fact that this is going to be bit large.

Unity scene with pink plane shown on ground

Then, go to the “Models” folder and drag in the “Dashboard” as a child of the image target. Resize it to the size you want. Keep in mind that we’re going to be placing buttons on it for each action of the crane. So make it big, but also not too big. I also made it a bit wider, due to the fact that we’re going to be adding 6 buttons.

Unity crane app scene with dashboard object added

Now we can start to make the buttons. Follow these steps to get to the result below.

  1. Right click in the hierarchy and create a new 3D Object > Cylinder.
  2. Change the name to “Button”.
  3. Scale down the Y axis so that it’s shorter.
  4. Change the material from the default one to “Button”.

Unity crane app scene with button object added

Now create a new C# script called “DashboardButton”, attach it to the “Button” object and open it up.

This script is just going to be an event that links to a function. In our case, an action of the crane.

So first up, add the below code to the other “using” lines at the top. We need access to Unity’s “Events” library to create a UnityEvent.

using UnityEngine.Events;

Now all we need inside of our class is the below variable. “onHold” is a UnityEvent, which we can link functions to. When the event is invoked, all the functions linked will be called.

public UnityEvent onHold;

Now going back to the editor, you should see the “DashboardButton” script look something like this on the “Button”.

Dashboard Button with On Hold() list in the Unity Inspector

Each button is also going to have an icon, displaying a direction or action that will happen when it’s pressed. So first, right click in the hierarchy (not the button) and select 2D Object > Sprite.

Change the name to “Icon” and change the “Sprite” in the Sprite Renderer to the “StraightArrow”. it should look something like this.

Icon object in Unity with Spirte Renderer component

The reason we didn’t create it as a child of the button is because since we’ve scaled the button, creating a sprite will make it inherit the scaled properties. Making it looked warped. So what we need to do now, is rotate the icon so it’s horizontal along the face of the button.

Arrow in Unity pointing to Dashboard button object

Then we can parent the icon to the button and set the position to 0 on all axis’ so it’s directly in the center.

Arrow cutting through dashboard button object in Unity

Now you can simple scale and move the icon so it fits nicely on top of the button.

Arrow made to fit on dashboard button object in Unity

Now you want to make the button a child of the “Dashboard” model. Then re position, rotate and scale it up a bit so it’s on the surface of the dashboard. An X axis rotation of -50 makes it lined up with the surface. I also scaled up the button around twice the original size.

Button object positioned on dashboard

Next, you want to duplicate the button so you have a total of 6 and re position them in the layout shown below.

Crane dashboard with various buttons added in Unity

For the 2 buttons on the left, change their icon’s sprite to “CircularArrow” and flip (negative on the X scale) the bottom one so it’s mirrored. For the other icons simply rotate them on the Y axis so that they are facing away from each other like below. Also rename them to what they will do.

Crane dashboard with buttons in Unity

Now click on the “TurnClockwise” button and in the “onHold” event, click on the plus sign to add a new event. Drag the “Crane” object which contains the crane script into the object property. Click on the “No Function” drop down and select Crane > TurnClockwise (). This will make it so as long as the button is being held down, the “TurnClockwise” function will be called. Now do this for all the other buttons, making sure that you link the correct function for their button.

Crane object in Unity Hierarchy being added to Dashboard button script

We also want to create a new tag like before and call it “Button”. Make sure that all the button objects has this tag assigned to it.

Button tag in Unity added to TurnClockwise object

Creating Touch Controls

Since we’re going to be building this to a touch device and we have in-world buttons to press, we need to setup a touch manager. This is a script that will…

  1. Check for touches.
  2. Shoot a raycast from the screen towards the touch.
  3. If it hit anything, check if it’s a button.
  4. If so, then call the button’s “onHold” event.

Let’s begin. First create an empty GameObject called “TouchManager”.

TouchManager highlighted in Unity Hierarchy

Then create a new and out final C# script and call it “TouchManager”. Attach it to the “TouchManager” object and open it up.

This is the code we are going to write inside of the Update function. It’s quite a bit, so let’s get started.

void Update ()
{
    // are there any touches on the screen?
    if(Input.touchCount > 0)
    {
        // loop through all the touches
        foreach(Touch touch in Input.touches)
        {
            // create a ray from the camera towards where we are touching
            Ray ray = Camera.main.ScreenPointToRay(touch.position);
            RaycastHit hit;

            // send out a raycast at that ray
            if(Physics.Raycast(ray, out hit))
            {
                // did we hit something?
                if(hit.collider != null)
                {
                    // if we hit a button, invoke that button's event
                    if(hit.collider.gameObject.CompareTag("Button"))
                    {
                        hit.collider.gameObject.GetComponent<DashboardButton>().onHold.Invoke();
                    }
                }
            }
        }
    }
}

We first check if there are any touches on the screen.

// are there any touches on the screen?
if(Input.touchCount > 0)
{

}

If so, we make a foreach loop and look through all the current touches on the screen.

// loop through all the touches
foreach(Touch touch in Input.touches)
{

}

For each touch on the screen, we’re going to be shooting a raycast. So first, we create out ray. ScreenPointToRay() shoots a ray from a screen position (in this case the position of the touch) towards the scene. We also need to make a RaycastHit variable to store the data of the object we hit.

// create a ray from the camera towards where we are touching
Ray ray = Camera.main.ScreenPointToRay(touch.position);
RaycastHit hit;

Then we shoot the raycast. If we hit something we check the tag to make sure it’s “Button”. If it is, then we get the “DashboardButton” component from the button and invoke the “onHold” event. This will make it so as long as the button is being pressed, the “onHold” event will be called.

// send out a raycast at that ray
if(Physics.Raycast(ray, out hit))
{
    // did we hit something?
    if(hit.collider != null)
    {
        // if we hit a button, invoke that button's event
        if(hit.collider.gameObject.CompareTag("Button"))
        {
            hit.collider.gameObject.GetComponent<DashboardButton>().onHold.Invoke();
        }
    }
}

If you want, you can add in the code below so that the dashboard will work in the editor while you test it out.

// as long as the mouse button is being pressed down
if(Input.GetMouseButton(0))
{
    // create a ray from the camera towards the mouse position
    Ray ray = Camera.main.ScreenPointToRay(Input.mousePosition);
    RaycastHit hit;

    // send out a raycast at that ray
    if(Physics.Raycast(ray, out hit))
    {
        // did we hit something?
        if(hit.collider != null)
        {
            // if we hit a button, invoke that button's event
            if(hit.collider.gameObject.CompareTag("Button"))
            {
                hit.collider.gameObject.GetComponent<DashboardButton>().onHold.Invoke();
            }
        }
    }
}

For testing it out in the editor, I just moved the test camera back a bit and re positioned the dashboard image target in-front of the crane. If you have errors popping up while you try to click on the buttons, you need to set your test camera’s tag to “MainCamera”.

Unity crane app functioning with crane dashboard

Final Steps

Before we can build to our device, we first have to do a few things. If you haven’t already, change the build platform to Android. Go to File > Build Settings. There, if your scene isn’t in “Scenes to Build”, click on the “Add Open Scenes” button. The select “Android” and click on the “Switch Platform” button.

Unity Build Settings window

Then go to Player Settings (Edit > Project Settings > Player). Make sure “Android Settings” is selected and open up the “Other Settings” tab. There, untick “Auto Graphics API” and add “OpenGLES2” to the “Graphics APIs” list below.

Graphics APIs with OpenGLES2 added in Unity's other settings

Finally, scroll down to “Package Name” and enter in your package name you created when making your EasyAR SDK licence key.

Identification information for crane app in Unity

Before we build though, we need to disable any of the editor testing things we made earlier and re-enable EasyAR. If you added the editor testing features from earlier, do the following…

  • Activate the “EasyAR_Startup” object.
  • Disactivate the test camera object.
  • Select all of the image targets and enable their “Image Target Behaviour” script.

EasyAR_Startup object with Image Target Behaviour script

From there, you should be able to go back to the Build Setting screen and press “Build and Run”. If you have your device connected to your computer with both the Android SDK and Java JDK installed and connected to Unity, the app should launch on your phone.

Conclusion

Now with the app installed onto your device, you should be able to use it. Print out the image targets and place them on a flat surface. Aim the phone at the targets and their corresponding models should appear. With the dashboard, you should be able to press the buttons on your screen and the crane should react – picking up and placing the container in the truck should work and if so, congratulations! You just completed the Crane Operator AR app!

Unity crane app functioning in augmented reality