Support for Skinned Models

Apr 11, 2009 at 11:38 PM
Edited Apr 13, 2009 at 7:55 PM
Hi there! First of all, I'd just like to say Goblin XNA Rocks!

Please accept my appologies if you've received about 1 billion emails regarding this post!  I've been tweeking and adding to it all evening and it suddenly dawned on me that you may be receiving an email notification for every update!  If this is the case, please let me know and I'll be more careful in future!

I would be very grateful if you could help me with my query regarding animated (skinned) models and Goblin XNA.  The verbose post below details all my endeavours, successes and failures, however this is all based on the integration of the Skinned Model sample from the XNA Creators Club and the Goblin XNA framework.  So if there's a simpler way to do animation with Goblin XNA then you needn't read on any further ;) But I would be very interested in hearing how this is done :)

Overview 

I am having some trouble adding an animated character to my game.  Being new to XNA (and 3D game development in general!) I went straight for the Skinned Model sample, from the XNA Creators Club.  Once I got my head around the code, I set about integrating it into my Goblin XNA game.  I've been able to successfully render an animated character* however the shadow effects are incorrect.  The interaction with occluder objects seems to behave correctly.  The problem I believe is that the shadow effect is bound to the none animated model (which is not drawn because I override the  base model object's Render() method).

* The 'tiny_4anim.x' model from the Microsoft DirectX SDK (March 2009) download.

Background to the 'Skinned Model sample' (XNA Creators Club)

The Skinned Model sample implements a custom Content Processor called 'SkinnedModelProcessor'.  My .x model file is configured to use this processor instead of the default processor 'Model - XNA Framework'.  The custom content processor uses a custom effect called 'SkinnedModel'.  This effect does not have a 'World' parameter like the default 'BasicEffect' effect.  Instead it has a 'Bones' parameter (an array of Matrix objects which represent the Skin Transforms generated from the animation player object).  The game's Update method calls the AnimationPlayer's Update method, passing in the model's 'World' transform.  The animation player then generates the next set of Skin Transforms to be used by the custom 'SkinnedModel' effect.

How I've integrated the Skinned Model code with my GoblinXNA game

I have sub-classed 'GoblinXNA.Graphics.Model' and 'GoblinXNA.Graphics.ModelLoader', creating a 'SkinnedModel' class and a 'SkinnedModelLoader' class.  The 'SkinnedModel' class has two additional properties and overrides the Render() method.  The two additional properties are as follows:-

1. Matrix[] SkinTransforms  -  An array of Matrix objects to represent the skin transforms (generated by an AnimationPlayer object).
2. SkinningData SkinningData  -  Combines all the data needed to render and animate a skinned object.  Obtained via the inner Model's Tag property.

See the bottom of this post for the class code.

The overridden Render() method uses the new 'SkinTransforms' property to set the custom 'Bones' parameter on the custom 'SkinnedModel.fx' effect.
 
The 'SkinnedModelLoader' class overrides the Load() method to create and return a new 'SkinnedModel' object (with a Tag property containing the SkinningData).  See the bottom of this post for the class code.  

This solution actually works, however there is at least one issue that I am aware of, which is explained below.

Problem Definition

The shadow effects are still bound to the basic effect therefore the shadows are drawn incorrectly.  Put another way, if I add a call to the base render method within the overridden Render() method, then the shadows follow the model being rendered by the basic effect.  My current work-around is to set the 'CastShadows' property to false (on my skinned model's 'Geometry' node).

Am I on the right track here or am I going completely against the Goblin XNA framework?  I know that the default GoblinXNA.Graphics.Model object has two interesting properties: namely AnimatedMeshes and AnimationTransforms, however I don't know how to use them.

Do I just need to implement my own shadow effect?

Any advice would be greatly appreciated!

Regards
Simon

CODE SNIPPETS

1) SkinnedModel.cs
2) SkinnedModelLoader.cs
3) AnimationPlayer.cs
4a) Game.cs - CreateObjects()
4b) Game.cs - Update()

/********************************************************************************/
/* 1) SkinnedModel.cs                                                                                                                            */
/********************************************************************************/
namespace GoblinXNAExtensions.Graphics
{
    public class SkinnedModel : GoblinXNA.Graphics.Model
    {
        private SkinningData _skinningData;
        private Matrix[] _skinTransforms;

        public SkinnedModel(Matrix[] transforms, ModelMeshCollection meshes, SkinningData skinningData)
            : base(transforms, meshes, null, null)
        {
            _skinningData = skinningData;
        }

        public SkinningData SkinningData
        {
            get
            {
                return _skinningData;
            }
        }

        public Matrix[] SkinTransforms
        {
            get
            {
                return _skinTransforms;
            }
        }

        /// <summary>
        /// Method to allow a caller to update the skin transformations.
        /// </summary>
        /// <rremarksThis method is used as the event handler for the AnimationPlayer event SkinTransformsUpdated.</remarks>
        /// <param name="skinTransforms"></pparam
        public void UpdateSkinTransforms(Matrix[] skinTransforms)
        {
            _skinTransforms = skinTransforms;
        }

        public override void UpdateAnimationTransforms(Matrix[] animationTransforms)
        {
            throw new NotImplementedException();
        }

        public override void Render(Matrix renderMatrix, Material material)
        {
            //base.Render(renderMatrix, material);

            GraphicsDevice device = State.Device;

            // Render the skinned mesh.
            foreach (ModelMesh mesh in this.Mesh)
            {
                foreach (Effect effect in mesh.Effects)
                {
                    effect.Parameters["Bones"].SetValue(_skinTransforms);
                    effect.Parameters["View"].SetValue(State.ViewMatrix);
                    effect.Parameters["Projection"].SetValue(State.ProjectionMatrix);
                }
                mesh.Draw();
            }
        }
    }
}

/********************************************************************************/
/* 2) SkinnedModelLoader.cs                                                                                                                 */
/********************************************************************************/
namespace GoblinXNAExtensions.Graphics
{
    public class SkinnedModelLoader : IModelLoader
    {
        public IModel Load(String path, String modelAssetName)
        {
            SkinnedModel skinnedModel = null;
            path = (path.Equals("")) ? State.GetSettingVariable("ModelDirectory") : path;
            String filePath = Path.Combine(path, modelAssetName);
            XNAModel xnaModel = State.Content.Load<XXNAModel(@"" + filePath);

            // Get matrix transformations of the model
            if (xnaModel != null)
            {
                Matrix[] transforms = new Matrix[xnaModel.Bones.Count];
                xnaModel.CopyAbsoluteBoneTransformsTo(transforms);

                bool preConditionPassed = false;
                if (xnaModel.Tag == null)
                {
                    preConditionPassed = false;
                }
                else if (xnaModel.Tag is SkinningData == false)
                {
                    preConditionPassed = false;
                }
                else
                {
                    preConditionPassed = true;
                }

                if (!preConditionPassed)
                {
                    throw new InvalidOperationException("This model does not contain a SkinningData tag.");
                }
                 
                skinnedModel = new SkinnedModel(transforms, xnaModel.Meshes, xnaModel.Tag as SkinningData);
            }
            else
            {
                Log.Write("Model " + filePath + " does not exist ");
            }

            return skinnedModel;
        }
    }
}

/********************************************************************************/
/* 3) AnimationPlayer.cs                                                                                                                          */
/*                                                                                                                                                            */
*  Please note: The model object is subscribed to the SkinTransformsUpdated event.  The event is fired    */ 
/* at the end of the UpdateSkinTransforms() method.                                                                               */
/********************************************************************************/
namespace GoblinXNAExtensions.Graphics
{
    public delegate void SkinTransformsUpdated(Matrix[] skinTransforms);

    /// <summary>
    /// The animation player is in charge of decoding bone position
    /// matrices from an animation clip.
    /// </summary>
    public class AnimationPlayer
    {
        #region Fields

        private SkinTransformsUpdated _skinTransformsUpdatedDelegate;

        public event SkinTransformsUpdated SkinTransformsUpdated
        {
            add { _skinTransformsUpdatedDelegate += value; }
            remove { _skinTransformsUpdatedDelegate -= value; }
        }

        // Information about the currently playing animation clip.
        private AnimationClip _currentClipValue;
        private TimeSpan _currentTimeValue;
        private int _currentKeyframe;
        
        // Current animation transform matrices.
        private Matrix[] _boneTransforms;
        private Matrix[] _worldTransforms;
        private Matrix[] _skinTransforms;

        // Backlink to the bind pose and skeleton hierarchy data.
        private SkinningData _skinningDataValue;

        #endregion
        
        /// <summary>
        /// Constructs a new animation player.
        /// </summary>
        public AnimationPlayer(SkinnedModel model)
        {
            if (model.SkinningData == null)
            {
                throw new ArgumentNullException("skinningData");
            }

            _skinningDataValue = model.SkinningData;
            _boneTransforms = new Matrix[model.SkinningData.BindPose.Count];
            _worldTransforms = new Matrix[model.SkinningData.BindPose.Count];
            _skinTransforms = new Matrix[model.SkinningData.BindPose.Count];

            SkinTransformsUpdated += new SkinTransformsUpdated(model.UpdateSkinTransforms);
        }

        #region properties

        /// <summary>
        /// Gets the current bone transform matrices, relative to their parent bones.
        /// </summary>
        public Matrix[] GetBoneTransforms()
        {
            return _boneTransforms;
        }
        
        /// <summary>
        /// Gets the current bone transform matrices, in absolute format.
        /// </summary>
        public Matrix[] GetWorldTransforms()
        {
            return _worldTransforms;
        }
        
        /// <summary>
        /// Gets the current bone transform matrices,
        /// relative to the skinning bind pose.
        /// </summary>
        public Matrix[] GetSkinTransforms()
        {
            return _skinTransforms;
        }
        
        /// <summary>
        /// Gets the clip currently being decoded.
        /// </summary>
        public AnimationClip CurrentClip
        {
            get { return _currentClipValue; }
        }

        /// <summary>
        /// Gets the current play position.
        /// </summary>
        public TimeSpan CurrentTime
        {
            get { return _currentTimeValue; }
        }

        #endregion

        #region public methods

        /// <summary>
        /// Starts decoding the specified animation clip.
        /// </summary>
        public void StartClip(AnimationClip clip)
        {
            if (clip == null)
            {
                throw new ArgumentNullException("clip");
            }

            _currentClipValue = clip;
            _currentTimeValue = TimeSpan.Zero;
            _currentKeyframe = 0;

            // Initialize bone transforms to the bind pose.
            _skinningDataValue.BindPose.CopyTo(_boneTransforms, 0);
        }

        /// <summary>
        /// Advances the current animation position.
        /// </summary>
        public void Update(TimeSpan time, bool relativeToCurrentTime, Matrix rootTransform)
        {
            UpdateBoneTransforms(time, relativeToCurrentTime);
            UpdateWorldTransforms(rootTransform);
            UpdateSkinTransforms();
        }

        /// <summary>
        /// Helper used by the Update method to refresh the BoneTransforms data.
        /// </summary>
        public void UpdateBoneTransforms(TimeSpan time, bool relativeToCurrentTime)
        {
            // Update the animation position.
            if (relativeToCurrentTime)
            {
                time += _currentTimeValue;

                // If we reached the end, loop back to the start.
                while (time >= _currentClipValue.Duration)
                    time -= _currentClipValue.Duration;
            }

            if ((time < TimeSpan.Zero) || (time >= _currentClipValue.Duration))
                throw new ArgumentOutOfRangeException("time");

            // If the position moved backwards, reset the keyframe index.
            if (time < _currentTimeValue)
            {
                _currentKeyframe = 0;
                _skinningDataValue.BindPose.CopyTo(_boneTransforms, 0);
            }

            _currentTimeValue = time;

            // Read keyframe matrices.
            IList<KKeyframe keyframes = _currentClipValue.Keyframes;

            while (_currentKeyframe < keyframes.Count)
            {
                Keyframe keyframe = keyframes[_currentKeyframe];

                // Stop when we've read up to the current time position.
                if (keyframe.Time > _currentTimeValue)
                {
                    break;
                }

                // Use this keyframe.
                _boneTransforms[keyframe.Bone] = keyframe.Transform;

                _currentKeyframe++;
            }
        }
        
        /// <summary>
        /// Helper used by the Update method to refresh the WorldTransforms data.
        /// </summary>
        public void UpdateWorldTransforms(Matrix rootTransform)
        {
            // Root bone.
            _worldTransforms[0] = 
                _boneTransforms[0] * 
                rootTransform; 
            
            // Child bones.
            for (int bone = 1; bone < _worldTransforms.Length; bone++)
            {
                int parentBone = _skinningDataValue.SkeletonHierarchy[bone];
                _worldTransforms[bone] =  _boneTransforms[bone] * _worldTransforms[parentBone];
            }
        }
        
        /// <summary>
        /// Helper used by the Update method to refresh the SkinTransforms data.
        /// </summary>
        public void UpdateSkinTransforms()
        {
            for (int bone = 0; bone < _skinTransforms.Length; bone++)
            {
                _skinTransforms[bone] = 
                    _skinningDataValue.InverseBindPose[bone] * _worldTransforms[bone];
            }

            // Raise update event
            if (_skinTransformsUpdatedDelegate != null)
            {
                _skinTransformsUpdatedDelegate(_skinTransforms);
            }
        }

        #endregion
    }
}

/********************************************************************************/
/* 4a) Game.cs                                                                                                                                        */
/*                                                                                                                                                            */
/*  Method CreateObjects() - not all the code is shown                                                                            */
/********************************************************************************/
private void CreateObjects()
   ...

             // Create an animation player and start decoding an animation clip.
            _tinyAnimationPlayer = new AnimationPlayer(tinyModel);
            AnimationClip clip = tinyModel.SkinningData.AnimationClips["Loiter"];
            _tinyAnimationPlayer.StartClip(clip);
 
   ...

}

/********************************************************************************/
/* 4b) Game.cs                                                                                                                                        */
/*                                                                                                                                                            */
/*  Method Update()                                                                                                                               */
/********************************************************************************/
 protected override void Update(GameTime gameTime)
{
     HandleInput();

     _tinyAnimationPlayer.Update(gameTime.ElapsedGameTime, true, _tinyTransformNode.WorldTransformation * _tinyNode.MarkerTransform);

     base.Update(gameTime);
}

END OF POST
Coordinator
Apr 14, 2009 at 12:56 AM
Hi Simon,

Nice work on very detailed descriptions^^. I don't set notifiers for new discussion post, so please feel free to modify as much as you need.

Thanks for bringing up the topic of animated models, and your approach of integrating the skinned model sample is correct (namely, extending the Model and ModelLoader classes). I actually incorporated the XNAnimation library (http://xnanimation.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=14889) downloadable from codeplex to GoblinXNA almost a year ago, but I couldn't get the shadows to work correctly either since I couldn't get the transform for each of the model parts in the skinned model at that time. I didn't have time to work on the animation capability since then, and thus it's not included with the current Goblin XNA distribution. It seems like the XNAnimation library is improved, and probably I can get it to work correctly if I spend some time. 

The shadow mapping in GoblinXNA is very robust, and not very extensible. You can implement your own shadow shader, but then you will need to also change many lines of code in Scene class. Once I incorporate the newer XNAnimation library, I hope I can make the shadow effect work correctly, but probably it won't happen until the next next release (v3.3).

AnimatedMeshes and AnimatedTransformations of the Model class is used to animate model parts of a static model (which is not a skinned model, and doesn't have any animation information in the model file). For example, if you want to animate the rotation of the rotating blades of a static windmill model, then you can assign the mesh of the rotating blades to AnimatedMeshes, and apply your rotation transformation to the AniamtedTransformation and manually rotate it.

Hope I answered all of your questions.

Ohan
Apr 14, 2009 at 12:49 PM
Edited Apr 14, 2009 at 8:59 PM

Hi Ohan,

Thanks for your response!  It's good to know that I was on the right track.  If the 'XNAnimation' integration piece is on the road map then I can happily forgo the shadow casting for now.  That said, I wouldn't mind having a go at attempting the integration myself.  Please feel free to encourage / discourage me as appropriate!  If the framework changes are quite deep-rooted then it might be better to leave this to someone who knows what they are doing ;)

 

Thanks also for explaining the usage of the 'AnimatedMeshes' and 'AnimationTransforms'.  I suspected these properties were for "static" models although I wasn't 100% sure.

Cheers

Simon

Coordinator
Apr 15, 2009 at 8:42 PM
Hi Simon,

I will probably release v3.2 in next few weeks, and there will be several modifications from v3.1. Since I can not guarantee to be able to include the animation stuff anytime soon, I would suggest that you incorporate XNAnimation with Goblin after the v3.2 release if animation is critical for your application. If the integration goes well, it'll be great if you could share the information with us.

Thanks
Ohan
Apr 15, 2009 at 10:25 PM
Hi Ohan,

I'll hold off then until the release of v3.2.  In the meantime I'll familiarize myself with the latest version of XNAnimation.  I'll be happy to share my findings with you, assuming of course that I find any findings ;)

Cheers
Simon
Feb 8, 2010 at 11:23 PM

HI Simon,

Thanks for such an informative post.

How have u incorporated modelanimator class in order to load an animated unskinned model?

Looking forward to your reply!