Augmented reality works by tricking the viewer that a virtual object is present in the real world.
However what's actually happening is that the view is watching a 2D representation of the virtual object
on a screen through which they can see the real world or video fade over it.
There are a multitude of applications for I from retail to manufacturing to games.
It works by using computer vision algorithms to detect the environment through analysis of a video feed.
There are two types of data.
The first requires the use of a marker.
This is a pre-processed and recognize pattern that the software looks for in the video feed.
You've already used markers in the applications you've made so far.
Originally Marcuse were black and white as they produce the best contrast.
In short the algorithm attempting to detect them in an image or video looks for features by first converting
the frame to black and white then running an edge detection algorithm.
This highlights where the contrasting areas meet each other key features in the image then picked out
where the areas of contrast mate is going to be a lot more areas of interest in this particular image.
But you get the idea.
These data points are then compared with the data points obtained from initial training of the image.
So the algorithm knows what it's looking for when it finds it's trying data set or close enough.
Inside the frame it knows it has detected the Mocca with its knowledge of what the flat marker looks
like.
It can determine the orientation of the mark in the frame.
This provides transformations scaling and rotational information to the application such as the virtual
object can be positioned in the correct way over the Mokka on the frame making it look as though it's
in the environment in order to capture light and accurate transformation for the virtual object.
The market needs to be not only well contrasting but also asymmetrical.
For example this image while having a nice contrast is the same no matter which way it is rotated.
This makes it unsuitable as an Mocca while black and white markers are still the best type of Mokka
given their inherent contrast.
We can also find other images with color even photographs to become markers.
However they must be visually interesting with plenty of contrast.
Take a look at these two images that we might want to use as markers.
Which one do you think is most suitable.
We can get a better idea by looking at them.
After an edge detection algorithm has been run the image on the right clearly has a far better amount
of contrast from which the algorithm can extract a strong set of comparison points.
As you can imagine this type of pattern recognition requires a lot of processing power and although
augmented reality has been a thing since 1992 when the US Air Force developed the very first immersive
system called Virtual fixtures.
It didn't really take off until the advent of mobile devices fitted with cameras.
Even then it was a little clunky.
But now with the processing power available in such devices the possibilities are almost endless.
Now you might be wondering about Pokemon gold.
It's an ideal location based game.
We discover the presence of Pokemon in the real world that can only be seen with your mobile device
by traveling to different geographical locations.
It displays augmented content and yet doesn't use predefined markers.
It is an example of Markkula.
The technology used for this is called Slam or simple tiniest localization and mapping it allows virtual
objects to be placed over the top of a real sane and allows the camera viewing the scene to move and
have the virtual object update its transformation accordingly instead of a predefined Mocca a frame
from the video scene is selected as the frame of reference and processed for feature points as the camera
moves and the video feed changes.
These points are tracked the way the points move provides information on how the virtual objects should
be transformed by the camera.
This provides for an extra level of usability in apps as the viewer doesn't need a pre-prepared marker
to use the app another type of Mocca.
We will also consider in this course of 3D objects instead of just a flat image.
A real world object can be used as a Mokka its feature points are trained and then it is detected in
video Faid a 3D Mocca is trained through a series of photographs or a video taken from as many angles
as possible instead of a 2D set of data points a 3D set of points or point.
Cloud is created that define the object the 3D object is then detected in the video fade and with its
position determined augmented content can be added.
The great thing about this form of ah is that the virtual object can be even more embedded in the real
world using occlusions shaders that make the virtual objects go behind the real the why the algorithms
track the marker determines not how the objects in the environment move but rather the virtual camera.
In fact the augmented objects themselves do not move at all unless you program and to do so but to appear
as though they are sitting on top of the mark in the video feed the virtual camera re-orient writes
itself.
Given the camera has the axes x y and Zed with Zed pointing at the Mocca the augmented object sitting
on the Mokka has its axes orientated with Y being the up vector.
When the marker rotates around its y axis it causes the camera to rotate in the opposite direction around
its zid axis.
In this case making the cube appear to rotate on the spot when the market rotates around its Zed axis.
The camera rotates around its y axis.
You can see here that the Zed and y axes are flipped between the camera and the Marka.
This is due to the camera having to look down at Zed axis to see the Marcuse position when the Mach
rotates around x.
So too does the camera.
The x axes is used for both camera and Marka for X movements
so when the marker trains lights along its x axis so does the camera.
However because of the flipped axes when the market moves up and down along the zid axis the camera
moves in Y and when the market moves along it's y or closer and further away from the camera the virtual
camera moves up and down along its Zayd axis.
All of the techniques we have discussed in this video are available to us through the use of the wicked
shoot SDK and we will be discovering how to use them all in this course.