This project is read-only.

Not able to Show dude Model in Augmented Reality Application.

Jul 20, 2010 at 8:29 AM
Edited Jul 20, 2010 at 10:35 AM

Hi,

I am trying to animate dude model on the marker. For that I have combined Tutotial 8 and Tutorial 14 of the Goblin XNA tutorials example. But I am not able to show model dude.fbx on the marker . Currently I am getting nothing on the marker. Following is the code in Tutorial.cs file.. I have changed CreateObject and Draw method the file and added different classes in my Tutorial 8, used in Tutorial 14 for animation..

 

/************************************************************************************
 * Copyright (c) 2008-2010, Columbia University
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are met:
 *     * Redistributions of source code must retain the above copyright
 *       notice, this list of conditions and the following disclaimer.
 *     * Redistributions in binary form must reproduce the above copyright
 *       notice, this list of conditions and the following disclaimer in the
 *       documentation and/or other materials provided with the distribution.
 *     * Neither the name of the Columbia University nor the
 *       names of its contributors may be used to endorse or promote products
 *       derived from this software without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY COLUMBIA UNIVERSITY ''AS IS'' AND ANY
 * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED. IN NO EVENT SHALL <copyright holder> BE LIABLE FOR ANY
 * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
 * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 *
 *
 * ===================================================================================
 * Author: Ohan Oda (ohan@cs.columbia.edu)
 *
 *************************************************************************************/

//#define USE_ARTAG

using System;
using System.Collections.Generic;
using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Audio;
using Microsoft.Xna.Framework.Content;
using Microsoft.Xna.Framework.GamerServices;
using Microsoft.Xna.Framework.Graphics;
using Microsoft.Xna.Framework.Input;
using Microsoft.Xna.Framework.Net;
using Microsoft.Xna.Framework.Storage;

using GoblinXNA;
using GoblinXNA.Graphics;
using GoblinXNA.SceneGraph;
using Model = GoblinXNA.Graphics.Model;
using GoblinXNA.Graphics.Geometry;
using GoblinXNA.Device.Capture;
using GoblinXNA.Device.Vision;
using GoblinXNA.Device.Vision.Marker;
using GoblinXNA.Device.Util;
using GoblinXNA.Physics;
using GoblinXNA.Helpers;
using Tutorial14___Skinned_Model_Animation;
using GoblinXNA.Sounds;

namespace Tutorial8___Optical_Marker_Tracking
{
    /// <summary>
    /// This tutorial demonstrates how to use both the ALVAR and ARTag optical marker tracker
    /// library with our Goblin XNA framework. If you're using ARTag, then uncomment the define
    /// command at the beginning of this file. Please read the README.pdf included with this
    /// project before running this tutorial. If you're using ARTag, please remove calib.xml
    /// from the solution explorer so that you can successfully build the project.
    /// </summary>
    public class Tutorial8 : Microsoft.Xna.Framework.Game
    {
        AnimatedModel animatedModel;
        TransformNode modelTransNode;
        GraphicsDeviceManager graphics;
        TransformNode shipTransParentNode;
        Scene scene;
        MarkerNode groundMarkerNode, toolbarMarkerNode;
        GeometryNode boxNode;
        GeometryNode modelNode;
        // set this to false if you are going to use a webcam
        bool useStaticImage = true;

        public Tutorial8()
        {
            graphics = new GraphicsDeviceManager(this);
            Content.RootDirectory = "Content";
        }

        /// <summary>
        /// Allows the game to perform any initialization it needs to before starting to run.
        /// This is where it can query for any required services and load any non-graphic
        /// related content.  Calling base.Initialize will enumerate through any components
        /// and initialize them as well.
        /// </summary>
        protected override void Initialize()
        {
            // Initialize the GoblinXNA framework
            State.InitGoblin(graphics, Content, "");

            // Initialize the scene graph
            scene = new Scene(this);

            // Use the newton physics engine to perform collision detection
            scene.PhysicsEngine = new NewtonPhysics();

            // For some reason, it causes memory conflict when it attempts to update the
            // marker transformation in the multi-threaded code, so if you're using ARTag
            // then you should not enable the marker tracking thread
#if !USE_ARTAG
            State.ThreadOption = (ushort)ThreadOptions.MarkerTracking;
#endif
            Sound.Initialize("Tutorial9");
            // Set up optical marker tracking
            // Note that we don't create our own camera when we use optical marker
            // tracking. It'll be created automatically
            SetupMarkerTracking();

            // Set up the lights used in the scene
            CreateLights();

            // Create 3D objects
            CreateObjects();

            // Create the ground that represents the physical ground marker array
            CreateGround();

            // Use per pixel lighting for better quality (If you using non NVidia graphics card,
            // setting this to true may reduce the performance significantly)
            scene.PreferPerPixelLighting = true;
            // Make the physics simulation space larger
            // to 500x500x500 centered at the origin
            ((NewtonPhysics)scene.PhysicsEngine).WorldSize =
                new BoundingBox(Vector3.One * -250, Vector3.One * 250);
            // Increase the gravity
            scene.PhysicsEngine.Gravity = 30.0f;

            ((NewtonPhysics)scene.PhysicsEngine).MaxSimulationSubSteps = 5;
            // Enable shadow mapping
            // NOTE: In order to use shadow mapping, you will need to add 'PostScreenShadowBlur.fx'
            // and 'ShadowMap.fx' shader files as well as 'ShadowDistanceFadeoutMap.dds' texture file
            // to your 'Content' directory
            scene.EnableShadowMapping = true;

            // Show Frames-Per-Second on the screen for debugging
            State.ShowFPS = true;

            base.Initialize();
        }

        private void CreateLights()
        {
            // Create a directional light source
            LightSource lightSource = new LightSource();
            lightSource.Direction = new Vector3(1, -1, -1);
            lightSource.Diffuse = Color.White.ToVector4();
            lightSource.Specular = new Vector4(0.6f, 0.6f, 0.6f, 1);

            // Create a light node to hold the light source
            LightNode lightNode = new LightNode();
            lightNode.LightSources.Add(lightSource);

            scene.RootNode.AddChild(lightNode);
        }

        private void SetupMarkerTracking()
        {
            // Create our video capture device that uses DirectShow library. Note that
            // the combinations of resolution and frame rate that are allowed depend on
            // the particular video capture device. Thus, setting incorrect resolution
            // and frame rate values may cause exceptions or simply be ignored, depending
            // on the device driver.  The values set here will work for a Microsoft VX 6000,
            // and many other webcams.
            IVideoCapture captureDevice = null;

            //if (useStaticImage)
            //{
            //    captureDevice = new NullCapture();
            //    captureDevice.InitVideoCapture(0, FrameRate._30Hz, Resolution._800x600,
            //        ImageFormat.R8G8B8_24, false);

            //    ((NullCapture)captureDevice).StaticImageFile = "testImage800x600.jpg";
            //}
            //else
            //{
                captureDevice = new DirectShowCapture();
                captureDevice.InitVideoCapture(0, FrameRate._30Hz, Resolution._640x480,
                    ImageFormat.R8G8B8_24, false);
            //}

            // Add this video capture device to the scene so that it can be used for
            // the marker tracker
            scene.AddVideoCaptureDevice(captureDevice);

            IMarkerTracker tracker = null;

#if USE_ARTAG
            // Create an optical marker tracker that uses ARTag library
            tracker = new ARTagTracker();
            // Set the configuration file to look for the marker specifications
            tracker.InitTracker(638.052f, 633.673f, captureDevice.Width,
                captureDevice.Height, false, "ARTag.cf");
#else
            // Create an optical marker tracker that uses ALVAR library
            tracker = new ALVARMarkerTracker();
            ((ALVARMarkerTracker)tracker).MaxMarkerError = 0.02f;
            tracker.InitTracker(captureDevice.Width, captureDevice.Height, "calib.xml", 9.0);
#endif

            // Set the marker tracker to use for our scene
            scene.MarkerTracker = tracker;

            // Display the camera image in the background. Note that this parameter should
            // be set after adding at least one video capture device to the Scene class.
            scene.ShowCameraImage = true;
        }

        private void CreateGround()
        {
            GeometryNode groundNode = new GeometryNode("Ground");

#if USE_ARTAG
            groundNode.Model = new Box(85, 66, 0.1f);
#else
            groundNode.Model = new Box(95, 59, 0.1f);
#endif
                
            // Set this ground model to act as an occluder so that it appears transparent
            groundNode.IsOccluder = true;

            // Make the ground model to receive shadow casted by other objects with
            // CastShadows set to true
            groundNode.Model.ReceiveShadows = true;

            Material groundMaterial = new Material();
            groundMaterial.Diffuse = Color.Gray.ToVector4();
            groundMaterial.Specular = Color.White.ToVector4();
            groundMaterial.SpecularPower = 20;

            groundNode.Material = groundMaterial;

            groundMarkerNode.AddChild(groundNode);
        }

        private void CreateObjects()
        {

            AnimatedModelLoader loader = new AnimatedModelLoader();
            animatedModel = (AnimatedModel)loader.Load("", "dude");
            animatedModel.UseInternalMaterials = true;
            animatedModel.ShaderTechnique = "SkinnedModelTechnique";
            animatedModel.CastShadows = true;
            animatedModel.ReceiveShadows = true;

            modelNode = new GeometryNode("Dude");
            modelNode.Model = animatedModel;
            modelNode.AddToPhysicsEngine = true;
            modelNode.Physics.Mass = 20;
            animatedModel.LoadAnimationClip("Take 001");

            modelTransNode = new TransformNode();
            modelTransNode.Translation = new Vector3(20, 0, 0);


    



           

        
#if USE_ARTAG
            groundMarkerNode = new MarkerNode(scene.MarkerTracker, "ground");
#else
            // Create an array to hold a list of marker IDs that are used in the marker
            // array configuration (even though these are already specified in the configuration
            // file, ALVAR still requires this array)
            int[] ids = new int[28];
            for (int i = 0; i < ids.Length; i++)
                ids[i] = i;

            groundMarkerNode = new MarkerNode(scene.MarkerTracker, "ALVARGroundArray.txt", ids);
#endif

           

//            // Create a marker node to track a toolbar marker array.
                #if USE_ARTAG
                            toolbarMarkerNode = new MarkerNode(scene.MarkerTracker, "toolbar1");

                            // Since we expect that the toolbar marker array will move a lot, we use a large
                            // smoothing alpha.
                            toolbarMarkerNode.Smoother = new DESSmoother(0.8f, 0.8f, 1, 1);
                #else
                            ids = new int[2];
                            ids[0] = 29;
                            ids[1] = 30;

                            toolbarMarkerNode = new MarkerNode(scene.MarkerTracker, "Toolbar.txt", ids);
                #endif

                            scene.RootNode.AddChild(toolbarMarkerNode);


            scene.RootNode.AddChild(groundMarkerNode);
            groundMarkerNode.AddChild(modelTransNode);
            modelTransNode.AddChild(modelNode);



        }

        /// <summary>
        /// A callback function that will be called when the box and sphere model collides
        /// </summary>
        /// <param name="pair"></param>
        private void BoxSphereCollision(NewtonPhysics.CollisionPair pair)
        {
            Console.WriteLine("Box and Sphere has collided");
        }

        /// <summary>
        /// UnloadContent will be called once per game and is the place to unload
        /// all content.
        /// </summary>
        protected override void UnloadContent()
        {
            Content.Unload();
        }

        /// <summary>
        /// Allows the game to run logic such as updating the world,
        /// checking for collisions, gathering input, and playing audio.
        /// </summary>
        /// <param name="gameTime">Provides a snapshot of timing values.</param>
        protected override void Update(GameTime gameTime)
        {
            base.Update(gameTime);
        }

        /// <summary>
        /// This is called when the game should draw itself.
        /// </summary>
        /// <param name="gameTime">Provides a snapshot of timing values.</param>
        protected override void Draw(GameTime gameTime)
        {
            // If ground marker array is detected
            if (groundMarkerNode.MarkerFound)
            {
                // If the toolbar marker array is detected, then overlay the box model on top
                // of the toolbar marker array; otherwise, overlay the box model on top of
                // the ground marker array
                if (toolbarMarkerNode.MarkerFound)
                {
                    // The box model is overlaid on the ground marker array, so in order to
                    // make the box model appear overlaid on the toolbar marker array, we need
                    // to offset the ground marker array's transformation. Thus, we multiply
                    // the toolbar marker array's transformation with the inverse of the ground marker
                    // array's transformation, which becomes T*G(inv)*G = T*I = T as a result,
                    // where T is the transformation of the toolbar marker array, G is the
                    // transformation of the ground marker array, and I is the identity matrix.
                    // The Vector3(4, 4, 4) is a shift translation to make the box overlaid right
                    // on top of the toolbar marker. The top-left corner of the left marker of the
                    // toolbar marker array is defined as (0, 0, 0), so in order to make the box model
                    // appear right on top of the left marker of the toolbar marker array, we shift by
                    // half of each dimension of the 8x8x8 box model.  The approach used here requires that
                    // the ground marker array remains visible at all times.
                    Vector3 shiftVector = Vector3.Zero;

#if USE_ARTAG
                    shiftVector = new Vector3(4, 4, 4);
#else
                    shiftVector = new Vector3(4, -4, 4);
#endif
                        
                    Matrix mat = Matrix.CreateTranslation(shiftVector) *
                        toolbarMarkerNode.WorldTransformation *
                        Matrix.Invert(groundMarkerNode.WorldTransformation);

                    // Modify the transformation in the physics engine
                    ((NewtonPhysics)scene.PhysicsEngine).SetTransform(boxNode.Physics, mat);
                }
                else
                    ((NewtonPhysics)scene.PhysicsEngine).SetTransform(modelNode.Physics,
                        Matrix.CreateTranslation(Vector3.One * 4));
            }

            // TODO: Add your drawing code here

            base.Draw(gameTime);
        }
    }
}

Jul 20, 2010 at 5:07 PM
I believe you forgot to add this line to Update method: animatedModel.Update(gameTime);
Jul 20, 2010 at 8:07 PM

Thanks, It Worked..

What changes do I need to do to implement this one for mobile Augmented reality..

Jul 21, 2010 at 6:05 AM
What do you mean by mobile AR?
Jul 21, 2010 at 7:42 AM
Edited Jul 21, 2010 at 7:43 AM

This Application which I have created is for Web Cam Augmented Reality, now I want to change this Apps for Mobile.

So how can I do Mobile AR using Goblin XNA?

I am using ALVAR for maker detection.

Jul 21, 2010 at 3:54 PM
This is what i am hoping to do in the future. Unfortunately it is not possible now because GoblinXNA is base on XNA 3.1 for now. So hopefully when they upgrade to XNA 4.0 we will be able to deploy it on Windows Phone 7.