Registering 3D body scans. a) The corpus registration problem involves bringing a template mesh into alignment with a database of 3D scans. This is challenging because: b) alignment may be ambiguous; c) scans may have noise and holes; d) poses can vary widely.
e) Co-registration: When registering a corpus of 3D scans to a common 3D template, person-specific articulated 3D models can provide very accurate priors on how the template is allowed to deform. By using such priors in place of more generic forms of geometric regularization, one can reliably obtain a set of alignments that closely fit the surface of the 3D scans, while simultaneously maintaining an accurate point-to-point correspondence across alignments. Since each person-specific model is repeatedly re-estimated from the alignments of that person, it is able to capture a person's 3D shape in a wide range of poses. This provides a highly realistic shape prior in regions where scan data is is missing.
f) We have used co-registration to align hundreds of bodies varying poses.
Three-dimensional (3D) shape models are powerful because they enable the inference of object shape from incomplete, noisy, or ambiguous 2D or 3D data. For example, realistic parameterized 3D human body models have been used to infer the shape and pose of people from images. The first step in training such models involves registering a database of 3D scans with a common 3D template. Once all the scans are in alignment, one can build powerful statistical models of shape.
Unfortunately, registering 3D scans is a challenging problem. In smooth regions, matching is ambiguous. Scan data may have noise or holes. Objects, like the human body may be articulated and may deform non-rigidly. Scans may have self contact (e.g. hands on the hips) that changes the topology of the surface. Many previous methods have used manual intervention, clean data, and limited datasets. Our work addresses the corpus registration problem where we have thousands of example scans and the processing must be automatic. Here we focus on human bodies, but the work can apply to more varied shapes.
The registration is an ill-posed problem and typically involves solving an optimization problem with regularization terms that penalize implausible deformations of the template. When aligning a corpus, however, we can do better than generic regularization. If we have a realistic statistical model of how the template can deform then alignments can be regularized by this model. Constructing a model of deformations, however, requires having a corpus that is already registered.
To address this chicken-and-egg problem, we approach modeling and registration together. By minimizing a single objective function, one can reliably obtain high quality registration of noisy, incomplete laser scans, while simultaneously learning a highly realistic articulated body model. The model greatly improves robustness to noise and missing data. Since the model explains a corpus of body scans, it captures how body shape varies across people and poses. Since the alignments and model are optimized together, we ensure not only that the alignments tightly fit the 3D data, but that the resulting model fits each alignment, and hence the data as well.
Our current work is addressing several issues including full automation and registering meshes with differing topology due to self contact. In some cases our scans have texture information that can be used to improve registration and we are developing new optimization methods to exploit this. We area also addressing the registration of hand scan, which have significant self occlusions. Finally, we are addressing other forms of contact between bodies and objects that result in deformation and occlusion.
New scanning devices like the Kinect could provide vasts amounts of training data for building statistical models of 3D shape. To be useful, however, the registration problem must be solved. Easy, robust, automatic, and accurate registrations of scans opens up the opportunity to build statistical models of shape for use in vision and graphics.
2 Body Labs, Inc.
3 Body Labs Inc.
