WELCOME

This tutorial will discuss "Tutorial8 - Optical Marker Tracking" from the tutorials included in your GoblinXNA directory. This document will elaborate on the various settings that diverge from the "Hello, World" tutorial: setting up the camera, setting up the patterns, and attaching graphical objects and behaviors to the patterns.

 

THE ALVAR LIBRARY

ALVAR is the library we will use to identify patterns with our camera so that we can assign the placement of generated graphics to the physical objects that display these patterns. Created by the VTT Technical Research Centre of Finland, it has been designed so that a camera can identify its patterns readily. Stark contrast between black and white in the pattern as well as unique shapes aid it in distinguishing its patterns from the shapes and colors of the background scene.

A full description of the ALVAR project can be found at http://virtual.vtt.fi/virtual/proj2/multimedia/alvar.html.

 

THE ENVIRONMENT

To setup the environment we must install everything in the right order. First you must install the general environment - you have accomplished this if you ran Tutorial 1 successfully. For this tutorial you will also need ALVAR. GoblinXNA's installation guide discusses this installation in detail. Make sure to read through the whole paragraph before attempting to follow the instructions.

Obtain the ALVAR tracking library (basic version) from the VTT Technical Research Centre website: http://virtual.vtt.fi/virtual/proj2/multimedia/alvar.html. This library is free only for trial and non-commercial use. (A professional version of the library, with additional features, is available for commercial use by contacting alvar.info@vtt.fi.) First, install ALVAR 1.4.0 and download and install OpenCV 1.0, which is required for running ALVAR 1.4.0. (You will need to restart your computer after installing OpenCV 1.0. to make sure that your path environment variable is set correctly.) Then, open ALVARWrapper1.4.sln under the GoblinXNA\wrappers\ALVARWrapper1.4 directory (You will need either Visual Studio 2008 Professional Edition or Visual C++ 2008 Express Edition to open this solution file as well as ALVAR project files). Compile this project and copy the generated ALVARWrapper.dll from the GoblinXNA\wrappers\ALVARWrapper1.4\Release directory to the GoblinXNA\dlls\unmanaged directory. When you compile, make sure that the “Additional Include Directories” under Configurations Properties C/C++ General and the “Additional Library Directories” under Configurations Properties Linker General in the project settings have the correct path for the ALVAR 1.4.0 and OpenCV 1.0 installations. Finally, copy the alvar140.dll and alvarplatform140.dll from the Alvar1.4.0\bin\msvc90 directory to the GoblinXNA\dlls\unmanaged directory, and cv100.dll, cvaux100.dll, cvcam100.dll, cxcore100.dll, and highgui100.dll from the OpenCV\bin directory to the GoblinXNA\dlls\unmanaged directory.

If you have problems with header files not being found, make sure you have included all the necessary directories containing those files in your project's Headers folder.

Then make sure to read the README.pdf located in the Tutorial 8 directory. (Mine is located in "\GoblinXNAv3.5\tutorials\Tutorial8 - Optical Marker Tracking\".) It explains that you must create the calib.xml file before proceeding. This step is important, since a calib.xml file not created using your own camera will not likely work.  On this site are two tutorials that will walk you through the two different ways to complete the process.

 

THE APPLICATION

The real objects we will be using as physical reference points for our application are two arrays of ALVAR patterns called the "ground" and the "toolbar". The ground is a 4x7 array of symbols that have been provided to you as a pdf file called "groundALVAR.pdf", located in "\GoblinXNAv3.5\tutorials\Tutorial8 - Optical Marker Tracking". The toolbar is a simple set of two symbols provided named toolbarALVAR.pdf. These arrays can be printed - the ground on 14x8.5 "legal" sized paper, and the toolbar on something smaller. For optimal pattern recognition it is then recommended that these pages be glued to cardboard or foam-core, allowing the patterns to be presented as clearly as possible and without the distortion caused by folding or bending.

Note that you can produce your own arrays of markers using the MarkerLayout tool provided with GoblinXNA. Instructions for using this tool are provided at the end of this tutorial.

The tutorial's application will display a sphere and a cube. When the ground array is present, both objects will appear on top of that array. When the toolbar array is present, the cube will instead appear on top of the toolbar.

 

THE CODE

First, open the Tutorials solution in your GoblinXNA directory (e.g. mine is located at C:\Users\Colin\Documents\GoblinXNAv3.5\tutorials\Tutorials.sln, and I am using Visual Studio 2008). Once it is opened, navigate to "Tutorial8 - Optical Marker Tracking" and open that project. Then open Tutorial8.cs to see the main body of custom code, including very helpful explanations.

The first thing to notice about the code is that there are new objects and methods being created that allow you to (1) register the physical camera, (2) establish the marker arrays as objects in the program, and (3) attach visual artifacts to those objects and overlay them in your camera's display window.

The first two items are configured in the new SetupMarkerTracking() method.

 

REGISTERING THE CAMERA

Examine the first section of code in SetupMarkerTracking().

private void SetupMarkerTracking()

{

    // Create our video capture device that uses DirectShow library. Note that

    // the combinations of resolution and frame rate that are allowed depend on

    // the particular video capture device. Thus, setting incorrect resolution

    // and frame rate values may cause exceptions or simply be ignored, depending

    // on the device driver. The values set here will work for a Microsoft VX 6000

    // and many other webcams.

    DirectShowCapture captureDevice = new DirectShowCapture();

    captureDevice.InitVideoCapture(0, FrameRate._30Hz, Resolution._640x480,

        ImageFormat.R8G8B8_24, false);

 

    // Add this video capture device to the scene so that it can be used for

    // the marker tracker

    scene.AddVideoCaptureDevice(captureDevice);

 

First, DirectShow is an C++ API provided by Microsoft that provides media streaming on the Windows platform. DirectShowNET is an opensource library that allows .NET applications to access the DirectShow libraries. GoblinXNA then provides the DirectShowCapture class to work with DirectShowNET. All of this allows the InitVideoCapture method to set the configuration of the camera. The settings shown here work for most webcams; if you have trouble you should cross-check the specifications of your camera. Finally, AddVideoCaptureDevice makes the camera available to our 3d scene processing and rendering class.

 

ESTABLISHING THE MARKERS

       IMarkerTracker tracker = null;

 

#if USE_ARTAG

       // Create an optical marker tracker that uses ARTag library

       tracker = new ARTagTracker();

       // Set the configuration file to look for the marker specifications

       tracker.InitTracker(638.052f, 633.673f, captureDevice.Width,

       captureDevice.Height, false, "ARTag.cf");

#else

       // Create an optical marker tracker that uses ALVAR library

       tracker = new ALVARMarkerTracker();

       ((ALVARMarkerTracker)tracker).MaxMarkerError = 0.02f;

       tracker.InitTracker(captureDevice.Width, captureDevice.Height,

              "calib.xml", 9.0);

#endif

 

       // Set the marker tracker to use for our scene

       scene.MarkerTracker = tracker;

 

       // Display the camera image in the background. Note that this parameter

       // should be set after adding at least one video capture device to the

       // Scene class.

       scene.ShowCameraImage = true;

 

This code allows the ALVAR marker tracker to recognize patterns, given the idiosyncratic distortions, gamma, etc. of your camera. Then, you add the marker tracker to the scene object.

The next step is to define the parameters of our unique ground array. These are laid out in the following code:

private void CreateGround()

{

       GeometryNode groundNode = new GeometryNode("Ground");

               

#if USE_ARTAG

       groundNode.Model = new Box(85, 66, 0.1f);

#else

       groundNode.Model = new Box(95, 59, 0.1f);

#endif

       // Set this ground model to act as an occluder so that it appears

       // transparent

       groundNode.IsOccluder = true;

 

       // Make the ground model to receive shadow casted by other objects with

       // CastShadows set to true

       groundNode.Model.ReceiveShadows = true;

      

       Material groundMaterial = new Material();

       groundMaterial.Diffuse = Color.Gray.ToVector4();

       groundMaterial.Specular = Color.White.ToVector4();

       groundMaterial.SpecularPower = 20;

 

       groundNode.Material = groundMaterial;

 

       groundMarkerNode.AddChild(groundNode);

}

 

 

ATTACHING VIRTUAL ARTIFACTS TO PHYSICAL OBJECTS

Then note that in the CreateObjects() method a configuration file is used to define the patterns' placements on the ground object:

       groundMarkerNode = new MarkerNode(scene.MarkerTracker,
       "ALVARGroundArray.xml");

This XML file is found in your Tutorial 8 root folder (the same place where you copied calib.xml.)

Farther down that method a sphere node is added to the ground. What this means is that changes to the ground's position will be inherited by the sphere.

       scene.RootNode.AddChild(groundMarkerNode);

       groundMarkerNode.AddChild(sphereTransNode);

       sphereTransNode.AddChild(sphereNode);

 

The same steps are used to create a toolbar. However, the box will jump from the ground array to the toolbar only when the toolbar is visible in the scene. This is accomplished with an if-else test to see if the toolbar has appeared. If the toolbar is visible then make this transformation:

       shiftVector = new Vector3(4, -4, 4);

       Matrix mat = Matrix.CreateTranslation(shiftVector) *

              toolbarMarkerNode.WorldTransformation *

              Matrix.Invert(groundMarkerNode.WorldTransformation);

 

 

WRAP-UP

The ALVAR library and patterns are important allies for making Goblin XNA a tool for building augmented reality applications. Especially when using a head-mounted display device such as the iWear VR920 coupled with an attached camera for relaying the pattern positions, you have powerful AR capabilities with minimal code.

 

Last edited Mar 19, 2011 at 10:15 PM by dcoli, version 7

Comments

Fai247 Jun 14, 2011 at 10:39 PM 
Great tut - thank you.