Creating Augmented-Reality Test Objects From "Templates"

One of the main challenges of developing an AR application is the fact that object placement must be created in real-time via scripting. Unlike regular 3D applications in Unity or even VR, we cannot rely or code for existing world geometry or meshes. This means that any time we want to create something, several considerations must be taken into account:

• How to get objects to be created not only relative to each other, but to the game "world" as well
• How to make sure objects are not occluded by the game mesh when they're created
• How to make sure objects are oriented in ways that make sense (objects should not be clipping through walls)

But most importantly:

• How do you have control over object placement, without actually knowing what world mesh on which you're going to be placing objects

One of the solutions can be seen in the Magic Leap version of Angry Birds, titled Angry Birds FPS. Their solution for object placement is likely one of the more common methods in AR applications: ask the user to pick a spot on which to place the game, and the rest of the game will take place inside this small area.

Image credit: Rovio Entertainment

This has some advantages. First, the developer has a considerable amount of control over what constitutes a valid area. For one, it must be flat enough and large enough to hold the game's expected play space. It also puts the onus on the player to pick a spot they judge is valid; a player is unlikely to pick a hard-to-see or hard-to-reach spot even if it is technically valid.

However, it also has some disadvantages. Restricting the entire game into a rigid, small space inherently gives up one of the advantages of AR: room-scale human-computer interaction where game objects interact with the room. The biggest concern, however, comes from the fact that the placement of the playing field is an expected task in the User Experience. You must pick a spot every time. When creating an AR app as a psychometric test, asking users to navigate a world mesh and place their test in a "valid location" inherently poses a variety of behavioral confounders.

My solution is the creation of object "templates". This is a tree of child objects grouped together by a single invisible parent. The process is as follows:

1. Decide ahead of time which scale, orientation and rough placement of the objects you would like by editing the prefab
2. Instantiate the prefab immediately when starting the test. The placement of the prefab will be predetermined but will be in front of the participant, facing them
3. Cast rays from the center of the headset to each prefab object. Make use of masking layers to ensure the rays go through the template objects so that the rays continue onward and stop only when hitting the world mesh
4. Instantiate the objects onto the world mesh, making sure they face perpendicular to the world geometry so they don't clip into the walls, floors, and objects
5. Because the Magic Leap does not populate the world mesh immediately, continue casting rays until they hit a mesh. Do this for every individual mesh, not all at once.
6. Delete the "template" prefab

A lo-fi diagram of the process can be seen below

Adjusting the prefab spacing and distance of instantiation in front of the participant greatly affects how these objects are spaced out. In future blog posts I will elaborate on how I used this technique to populate a specific layout and order of objects across walls in a room.