More XNA Cel Shading - Cel Shaded Animated Models using a SkinnedModelProcessor

This short post builds upon my previous article A Cel-Shading Example in XNA 4.0, applying what we did there to a skinned model with animation.

Before you begin
First you will need to complete and understand the Skinned Model tutorial on the MSDN AppHub site http://create.msdn.com/en-US/education/catalog/sample/skinned_model. After that this example is pretty straight forward as long as you understand what digitalerr0r did in his post XNA Shader Programming Tutorial 7, Toon shading and what we in my previous post A Cel-Shading Example in XNA 4.0.

Update CelShader.fx
First we will update out CelShader.fx and call it SkinnedCelShader.fx.  Using the Microsoft stock effects as an example add the following message to get the skinning data from the model.

/* Get the skinning data from each of the bones
 */
void Skin(inout VertexShaderInput vin, uniform int boneCount)
{
    float4x3 skinning = 0;

    [unroll]
    for (int i = 0; i < boneCount; i++)
    {
        skinning += Bones[vin.Indices[i]] * vin.Weights[i];
    }

    vin.Position.xyz = mul(vin.Position, skinning);
    vin.N = mul(vin.N, (float3x3)skinning);
}

And then call Skin from your vertex shader before making any of the other position calculations

VertexShaderOutput VertexShaderFunction(VertexShaderInput input)
{
    VertexShaderOutput output;

    Skin(input, 4);

    float4 worldPosition = mul(input.Position, World);
    float4 viewPosition = mul(worldPosition, View);
    output.Position = mul(viewPosition, Projection);

    output.Tex = input.Tex;
    output.L = normalize(LightDirection);
    output.N = normalize(mul(InverseWorld, input.N));

    return output;
}

The game class
The modifications to the game class are pretty simple.  Start with the game class you implemented in the SkinnedEffect tutorial.

Add some class members to hold our new effect data and some parameters we will need to set their properties:

/* CelShader effects and data
 */
Effect celShader; // Toon shader effect
Texture2D celMap; // Texture map for cell shading
Vector4 lightDirection; // Light source for toon shader

Effect outlineShader; // Outline shader effect
float defaultThickness = 1.0f; // default outline thickness
float defaultThreshold = 0.9f; // default edge detection threshold
float outlineThickness = 1.0f; // current outline thickness
float outlineThreshold = 0.9f; // current edge detection threshold
float tStep = 0.01f; // Ammount to step the line thickness by
float hStep = 0.001f; // Ammount to step the threshold by

// Render target for post render outlining
RenderTarget2D celTarget;

Set our light direction in the initialize method:

/* Set our light direction for the cel-shader
 */
lightDirection = new Vector4(2f, 45f, -110f, 1.0f);

And in LoadContent load our custom effects:

// load and initialize our cel shader effect
celShader = Content.Load<Effect>("SkinnedCelShader");
celMap = Content.Load<Texture2D>("celMap");

celShader.Parameters["Projection"].SetValue(proj);
celShader.Parameters["View"].SetValue(view);
celShader.Parameters["LightDirection"].SetValue(lightDirection);
celShader.Parameters["CelMap"].SetValue(celMap);

/* Load and initialize the outline shader effect
 */
outlineShader = Content.Load<Effect>("OutlineShader");
outlineShader.Parameters["Thickness"].SetValue(outlineThickness);
outlineShader.Parameters["Threshold"].SetValue(outlineThreshold);
outlineShader.Parameters["ScreenSize"].SetValue(
new Vector2(GraphicsDevice.Viewport.Bounds.Width, GraphicsDevice.Viewport.Bounds.Height));

/* Set up a render target to draw our cel shaded model to
 * for post render outlining
 */
celTarget = new RenderTarget2D(GraphicsDevice, GraphicsDevice.Viewport.Width, GraphicsDevice.Viewport.Height,
    false, SurfaceFormat.Color, DepthFormat.Depth24);

Finally update Draw to draw our shaded model (you will notice this is pretty much identical to the way we drew the cel shaded model in my previous post).  The only trick here is that we need to get the model's texture.  We do not have that directly because its reference is baked into the fbx.  Because our model is loaded with its default effect as SkinnedEffect though we can get it from there with the following line of code:

Texture2D texture = ((SkinnedEffect)meshPart.Effect).Texture;

The full draw method:

protected override void Draw(GameTime gameTime)
{
    /* Set our render target
     */
    GraphicsDevice.SetRenderTarget(celTarget);

    /* Make sure we have a depth stencil for proper
     * depth culling
     */
    GraphicsDevice.DepthStencilState = DepthStencilState.Default;

    /* clear the graphics device with a color with a clear
     * alpha channel.
     */
    Color alpha = Color.White;
    alpha.A = 0;
    GraphicsDevice.Clear(alpha);

    /* Get our model bones from the animation player
     * so that we can pass this to the cel shader effect
     */
    Matrix[] bones = animationPlayer.GetSkinTransforms();

    // for each model in the mesh
    foreach (ModelMesh mesh in model.Meshes)
    {
        // for each mesh part
        foreach (ModelMeshPart meshPart in mesh.MeshParts)
        {
            // Set the vertex buffer and indices in the graphics device
            GraphicsDevice.SetVertexBuffer(meshPart.VertexBuffer,
            meshPart.VertexOffset);
            GraphicsDevice.Indices = meshPart.IndexBuffer;

            /* The texture of a skinned model is available in SkinnedEffect.Texture.
             * We are loading our model with the SkinnedModelProcessor and using
             * SkinnedEffect as the model's default effect. Use that to get the
             * texture of the current mesh part.
             */
            Texture2D texture = ((SkinnedEffect)meshPart.Effect).Texture;

            /* Set up a simple world, no translation, scaling or rotation
             * for this example
             */
            Matrix world = Matrix.Identity;

            /* Set the color map, world, inverse world and bones
             * properties of the cel shader effect
             */
            celShader.Parameters["ColorMap"].SetValue(texture);
            celShader.Parameters["World"].SetValue(world);
            celShader.Parameters["InverseWorld"].SetValue(Matrix.Invert(world));
            celShader.Parameters["Bones"].SetValue(bones);

            // for each effect pass in the cell shader
            foreach (EffectPass pass in celShader.CurrentTechnique.Passes)
            {
                // apply the effect
                pass.Apply();

                // and draw the current mesh part
                GraphicsDevice.DrawIndexedPrimitives(PrimitiveType.TriangleList, 0, 0,
                    meshPart.NumVertices, meshPart.StartIndex, meshPart.PrimitiveCount);
            }
        }
    }

    /* We are done with the render target so set it back to null.
     * This will get us back to rendering to the default render target
     */
    GraphicsDevice.SetRenderTarget(null);

    /* Clear the device to get ready for more drawing
     */
    GraphicsDevice.Clear(Color.Wheat);

    /* Draw the game model again without cell shading so we can do a side
     * by side comparison
     */
    foreach (ModelMesh mesh in model.Meshes)
    {
        foreach (ModelMeshPart meshPart in mesh.MeshParts)
        {
            foreach (SkinnedEffect effect in mesh.Effects)
            {
                effect.SetBoneTransforms(bones);
                effect.View = view;
                effect.Projection = proj;
                effect.EnableDefaultLighting();
                effect.SpecularColor = Vector3.Zero;
            }
            mesh.Draw();
        }
    }

    /* Draw the cel shaded model with outlining
     */
    spriteBatch.Begin(SpriteSortMode.Deferred, BlendState.NonPremultiplied,
        null, null, null, outlineShader);
    spriteBatch.Draw(celTarget, new Vector2(-175, 0), Color.White);
    spriteBatch.End();

    /* Draw our debug message
     */
    spriteBatch.Begin();
    spriteBatch.DrawString(debugFont, debugMsg, debugLoc, consoleFgColor);
    spriteBatch.End();

    base.Draw(gameTime);
}

Things to watch out for:
After adding your animated model to the content project make sure you change its content processor property to SkinnedModelProcessor and then under set the default effect to SkinnedEffect.  If you miss either of these steps you will get errors.

Here is what the result will look like (cel shaded model on the left compared to a naturally lit model on the right).

The full code for this project can be downloaded here.

Good luck!