How Does Augmented Reality Work? (with examples)

Augmented reality (AR) this, augmented reality that, we’re guessing you’ve heard a lot about augmented reality lately. But how does it actually work?

Originally popularised in video games and entertainment, AR is now bleeding into new applications such as education, eCommerce, healthcare, retail and more. The potential uses for AR are virtually endless, making it an exciting technology to watch expand. But how does it work?

In this article, we’re going to look at how augmented reality actually works, what hardware is required, the types of AR, and some examples of AR in practice.

What is AR?

This a question we answer in almost every article we post… but here it is again, just for you. Augmented reality is a technology that blends digital images and information with the physical world. By using a device such as a phone or tablet, you can view the real world through its camera and see virtual objects overlaid on top of it. This allows you to enhance your surroundings with computer-generated visuals and information.

AR’s unique characteristics mean that the technology sits somewhere between the physical world and virtual reality (VR). AR’s applications have expanded immensely in recent years and will continue to do so. Soon, we will all be using AR every day and wonder how we lived without it.

Learn more about AR in our augmented reality ultimate guide.

How does augmented reality work?

Augmented reality works by overlaying digital objects, information, or other sensory elements on top of the physical world to provide users with a beneficial, informative, or entertaining experience depending on the application. AR can be used on a range of devices, including smartphones or tablets, headsets or glasses.

AR needs specific components to create a truly realistic, and immersive experience. Including (but not limited to):

• Input devices: input devices include cameras and sensors, which are used to capture the real-world environment, this data is then sent to be processed.
• Computer processors: depending on the device the processor(s) will differ. The data sent by the input devices are then processed using algorithms to track location, orientation and objects in the real-world environment, allowing digital objects to be placed accurately. Objects are also rendered before being integrated into the physical environment.
• Output devices: also known as screens, projectors or other displays show the combined digital and physical world for a seamless augmented reality experience.

The most common AR devices are mobile phones, you know the thing in your hand or pocket 90% of the time? Mobile phones have all the necessary components to make AR experiences possible, these are by no means the most immersive, but they are still incredibly powerful, and not to forget, widely accessible. A smartphone will follow the same process as above, but here is a simplified image to explain further:

What are the different types of augmented reality?

There are two main types of augmented reality, marker-based AR and markerless AR. Although they mostly aim to achieve the same goal, it’s important to know their differences, as each type is more suitable for certain applications. Although the name is fairly self-explanatory, here is an explanation of them both:

Marker-based AR

Marker-based AR is somewhat outdated nowadays but still has specific use cases. These applications use (you guessed it) markers to position digital objects in a given space.

Engine Creative, the creators of REYDAR, were among the first to embrace augmented reality technology and had the opportunity to work on marker-based AR projects at an early stage. One of the earliest marker-based AR projects we worked on was an AR album cover for The Ting Tings, check it out:

These applications rely on the use of visual markers in the real world to trigger the overlay of digital content. If you try to use the application somewhere other than this, prepare to be disappointed, as not much will happen. Marker-based AR is a lot simpler to implement than some other types, which is why it is quite popular, but also comes with obvious limitations.

Although the technology was once groundbreaking, so was sliced bread, we’re not saying marker-based AR is useless today, but the technology has grown since its introduction.

Markerless AR

As you’ve probably worked out just by reading the name of this type of AR, markerless AR does not require a specific image or pattern like its counterpart.

Instead, markerless AR relies far more on the device’s components’ power, meaning that the sensors (camera, GPS, accelerometers etc) detect and track the physical environment to accurately place digital objects or information. Seamless markerless AR objects will appear anchored to the users surrounding in the real world.

Markerless AR is more complex than marker-based AR as it requires more comprehensive systems to identify, track and render objects in a physical environment without the use of markers. Despite its complexity, it delivers a far more seamless AR experience for the users and there are no true restrictions on location or orientation.

Applications can also be stretched further with markerless AR. For example, markerless AR could be used to display patient information during surgery, display a product in a customer’s home, or enhance education with digital interactive content.

Apple’s ARKit and Google’s ARCore have allowed markerless AR available on smart devices, including mobile phones and tablets, making this powerful type of AR incredibly accessible. Thanks to its capabilities, markerless AR is the preferred choice for the majority of applications utilising AR technology. REYDAR used markerless technology for our 3D and AR viewers, check out an example below and try it out on your smart device:

Here are some of the most popular types of markerless augmented reality (with examples):

Location-based AR

Location-based AR uses GPS and other sensors to place virtual objects in the user’s environment based on, you guessed it, location. A great example of this type of AR is the mobile game Pokemon GO, which brought the user’s environment to life depending on where they were and where they were looking.

Location-based AR can provide an incredibly immersive experience with far fewer restrictions than marker-based AR by creating an interactive environment with digital objects that are anchored to their physical surroundings.

Projection-based AR

Projection-based AR will use a fixed projector in a physical space to overlay 3D objects within the user’s physical space. This type of markerless AR is unique, as the output device is a projector, which means that users do not need to wear a headset/glasses or need to use a handheld device, such as a smartphone.

Projection-based AR could be interactive, such as a projected keyboard on a desk that allows you to type, or it may not be interactive, such as an advertisement projected onto a large building or another surface. This type of AR is one of the least common types of AR, but as technology develops its uses will become more valued and interactive.

Overlay AR

Overlay AR does what it says on the tin, it overlays an existing physical object/space with a virtual one. Although most markerless AR offers a lot of freedom, overlay AR can be quite restrictive as it can only be used in certain places or on certain objects. Despite this, this type of AR is still incredibly useful and interactive.

This type of augmented reality is often used to provide additional information or data about specific objects. For example, this AR could be used in a museum and when the user scans a specific piece with their smart device, additional information could be displayed, around or on that specific piece.

REYDAR, your augmented reality agency

Choosing an augmented reality agency is tough, trust REYDAR, we have been developing AR solutions and experiences for over a decade now and we are all ears when it comes to new projects. If you’ve got an idea, talk with one of our AR experts today.