can you rig it?

john patrick character td

So what is this “corrective displacement” thing? In a nutshell, it’s a way to give your blendshapes a very high level of detail that you can create sculpturally. When working with characters created in a package like zBrush or Mudbox, it’s common to export a lower-resolution mesh to Maya to rig and animate, and to use a displacement or normal map that will restore the high-resolution detail to a mesh that is subdivided at rendertime. Unfortunately (or fortunately), this means we don’t have nearly the same level of detail within our 3D package to create blendshapes as we do in our sculpting package.

This guide covers a workflow for creating both base sculptures and blendshapes in zBrush, and using displacement maps generated for each blendshape to retain the high frequency detail that can help sell the realism of our shapes. We can then use a node network in Maya to subtract out the differences between the base displacement map and the blendshape maps, which allows us to layer up our corrective displacements (more on this later). In my opinion, zBrush's masking and sculpture tools make it far easier to create broad, realistic blendshapes in a more artistic way, without having to worry so much about individual point pulling.

Once we have a base sculpture in zBrush, it's time to get it into Maya. Step down to your level 1 mesh, and export an obj, then import this into Maya. It's important to set the scale of the mesh at this point, as 32-bit displacement maps from zBrush rely on the absolute scale of the model, so scale it up however you'd like, then re-import it into zBrush at level 1. From here we need to export our base displacement map from the level 1 mesh. Keep in mind that at the end of this process we will end up with quite a few separate displacement maps, weight maps, etc, so naming all your files sensibly is important (I called mine “head_base_displacement”). Describing the displacement mapping pipeline is outside the scope of this guide, so check out Scott Spencer's tutorials for more info. Once I have my base sculpt defined, I'll save out a zTool with “base” appended to it, just so that I don't ever lose what I've defined the default shape to be.

Now it's time to sculpt the blendshapes. My approach generally means spending more time creating my masks then actually sculpting, and this is the great thing about creating blendshapes in zBrush. Another great thing is that we can use symmetry in zBrush to create symmetrical blenshapes, then later in Maya we can split them into left/right shapes.

First, I'll store a morph target on the highest subdivision level so that I can look at how the high-resolution mesh deforms. I can paint a mask that defines the broad area I want to be affected by the blendshape, then blur the hell out of the mask so that the points that move will blend off gradually around the shape. I find it easy to paint my mask on a high subdivision level, then step down and blur on a low subdivision level to get a broad falloff. On my low level, I'll use a very large move brush to create the major mass movement, then step up to higher levels to sculpt in wrinkles. Generally speaking, try to pull form out to create wrinkles (hint – paint more masks). The reason for this is that your skin wrinkles to preserve the mass of the flesh as the area compresses, and it pushes outwards to achieve this volume preservation.

Another tip is to avoid sculpting on the very highest subdivision levels, or you can accidentally erase skin texture, which will look odd in the final shape. You want to be at a level that's high enough to push some skin around, but not so high as to blur the tertiary detail. I find myself frequently stepping up to my highest level to toggle my blendshape and see how things are moving. One of the disadvantages to this approach is that you can't switch the blendshapes on a level other than the one you created it upon. I'd be great in a future version of zBrush if this was possible, but for now we're stuck stepping up and down a bit. If you have a really dense mesh, you might consider creating the morph target one or two levels down from the highest, so that you're not sitting around waiting for zBrush to compact memory.

Once I've sculpted a blendshape I'm happy with and checked how it's moving (you can also do this in zMapper), I'll export the low-res obj and export another displacement map, keeping the same resolution and settings as the original. Once I've gone through this process for all the shapes I want to create, I can import all my objs into maya and use them to create my blendshape deformer (you'll want to rename them to describe the shapes).

Now if I want to split my shapes into left/right shapes, I need to use the Paint Blendshape Weights tool to paint out one side of the shape. I'll then duplicate this head, export the weight map, and name it based on the side that I'm left with (i.e., “r_squint_weight” for a map where I've painted the left side out), and rename the duplicate of the shape (“r_squint”). Then I will get the complement of this shape by inverting the map in Photoshop and re-importing the map, making another duplicate, and renaming. If you haven't read Stop Starring, Jason Osipa goes into this technique at length (which he calls “tapering”), and provides some very nice scripts for more quickly tapering off these shapes. However, even if using his scripts, we do still need to export the first weight map because we are going to use it in our final node network to control how our corrective displacements are applied. You may need to go through the process of creating and deleting blendshape nodes a bit when splitting up the shapes, but at the end of the process they should be re-connected into a single blendshape and you should have a single weight map for each pair of shapes.

So now we get to the meat of this process – setting up our node network in Maya. The fundamental concept here is that we want to have our base displacement map as well as a “corrective” map that corresponds with each individual blenshape we've created. What we have at the moment though is our base displacement plus a set of full (non-corrective) maps for each pair of shapes. We can't simply blend between the full displacement maps, because there will almost certainly need to be multiple blendshapes active at once, and we want each map to trigger its own corrective displacement. So the first step is to create a corrective map for each full displacement map we exported from zBrush, then to split it with our weight maps to match the individual shapes we created by painting blenshape weights, then to add these corrective displacements back on top of the original displacement (but only by the amount that the blendshape is activated).

To start, create file texture nodes for the base displacement map and the blendshape displacement map – in this example I'm creating a squint, so I'll stick with that naming. I'll name the fileTexture nodes “head_baseDisplace_ftx” and “head_squintDisplace_ftx.” Make sure to keep the alpha gain at 1 and alpha offset at 0 for these – we'll take care of setting the proper alpha at the end of the process, but they need to be normalized between 1 and 0 until the end of the pipe.

Now we are going to do a little math to get the differences between the maps. This is the “corrective” aspect of corrective displacement mapping – we get just the differences between the blendshape and the original displacement maps (whether they are positive or negative), then add these back on top of the original displacement map by a percentage that corresponds with how much the blendshape is active.

To get the difference, we simply use a plusMinusAverage node, which I've named “head_squintCorrective_pma” and set to subtract. Then plug head_squintDisplace_ftx.outColor into head_squintCorrective_pma.input3[0], and plug head_baseDisplace_ftx.outColor into head_squintCorrective_pma.input3[1]. Bingo, we have the corrective map coming out of the corrective PMA node. Anywhere the two maps are the same, the value will be 0. Higher points on the blenshape displacement map will be positive, lower points will be negative. I created an image below to help visualize what's happening with this map...you can't see the negative values, but you can see where the blenshape map's points were higher than the original displacement map.

In order to split this corrective map into Create a third file texture called “head_squintWeightR_ftx, ” and load your weight map for this shape into it; this is if you painted black into the left side – switch the side in the name otherwise. We just want the side to correspond to the white area of the map when it's mapped on the character. Also create a reverse node called “head_squintWeightL_rev”, and plug the color from the squintWeightR into the input value, so that the values are flipped.

Now create two multiplyDivide nodes, called “head_squintWeightR_mdn” and “head_squintWeightL_mdn.” Make sure these are set to multiply, and plug the output from the corrective PMA into the input1 of both these nodes. Then take the outColor from the squintWeightR_ftx and put this into input2 of the squintWeightR_mdn, and take the output from the squintWeightL_rev and put it into the squintWeightL_mdn. Now we've got our corrective map split out into two sides.

We also want to multiply the effect these shapes have by the amount that the blendshape is active. To do this, create two more MDNs (one per side), and name them “head_squintBlendWeightR_mdn”
and “head_squintBlendWeightL_mdn.” Plug the outValue from the squintWeight MDNs into the input1 of each respective BlendWeight MDN. Now open the connection editor, and load your mesh's blendshape node into the output, and load one of the MDNs into the input. Connect the weight value of the appropriate blendshape/side into the x,y, and z of the MDN, and repeat for the other side. Now our corrective maps are split into sides and being activated by the blendshapes, but they're still not being added to the displacement.

Finally, we need to create a couple more nodes – a ramp called “head_finalDisplacement_rmp” and another PMA called “head_finalDisplacement_pma.” Set the PMA to add, and plug the outColor from the displacement base file into the input.3D[0] of the PMA. Now connect the output X,Y,Z from each corrective shape into new slots in the PMA. For the new ramp node, set the alpha gain to 2.2 and the alpha offset to -1.1, and connect the output of the PMA into the ramp. The outAlpha from this ramp can then go right into the displacement node – and we're done!

Copyright (c) 2010 John Patrick