Shape from mocap. MoSh computes body shape and pose from standard mocap marker sets. Bodies in purple are estimated from 67 mocap markers, while scans in green are captured with a high-resolution 3D body scanner. Split-color bodies compare the shape estimated from sparse markers with scans. MoSh needs only sparse mocap marker data to create animations (pink posed bodies) with a level of realism that is difficult to achieve with standard skeleton-based mocap methods.
While marker-based motion capture (mocap) is widely used to animate human characters in films and games, it is also widely criticized as producing lifeless and unnatural motions. We argue that this is the result of “indirecting” through a skeleton that acts as a proxy for the human movement. In standard mocap, visible 3D markers on the body surface are used to infer the unobserved skeleton. This skeleton is then used to animate a 3D model and what is rendered is the visible body surface. While typical protocols place markers on parts of the body that move as rigidly as possible, soft-tissue motion always affects surface marker motion. Since non-rigid motions of surface markers are treated as noise, subtle information about body motion is lost in the process of going from the non-rigid body surface to the rigid, articulated, skeleton representation. We argue that these non-rigid marker motions are not noise, but rather correspond to subtle surface motions that are important for realistic animation.
We present a new method called MoSh (for Motion and Shape capture) that replaces the skeleton with a 3D parametric body model. Given a standard marker set, MoSh simultaneously estimates the marker locations on a proxy 3D body model, estimates the body shape, and recovers the articulated body pose. By allowing body shape to vary over time, MoSh is able to capture the non-rigid motion of soft tissue. We show that significant soft tissue motion is present in small marker sets and that capturing it results in more nuanced and lifelike animations. MoSh also recovers qualitatively and metrically accurate body shapes from small numbers of markers. While fine details are missing, MoSh enables users of standard mocap to obtain reasonable 3D body shapes from markers alone.
You may download the dataset, for scientific purposes, here
Marker-based motion capture (mocap) is widely criticized as producing lifeless animations. We argue that important information about body surface motion is present in standard marker sets but is lost in extracting a skeleton. We demonstrate a new approach called MoSh (Motion and Shape capture), that automatically extracts this detail from mocap data. MoSh estimates body shape and pose together using sparse marker data by exploiting a parametric model of the human body. In contrast to previous work, MoSh solves for the marker locations relative to the body and estimates accurate body shape directly from the markers without the use of 3D scans; this effectively turns a mocap system into an approximate body scanner. MoSh is able to capture soft tissue motions directly from markers
by allowing body shape to vary over time. We evaluate the effect of different marker sets on pose and shape accuracy and propose a new sparse marker set for capturing soft-tissue motion. We illustrate MoSh by recovering body shape, pose, and soft-tissue motion from archival mocap data and using this to produce animations with subtlety and realism. We also show soft-tissue motion retargeting to new characters and show how to magnify the 3D deformations of soft tissue to create animations with appealing exaggerations.