a. Range images are captured as a person moves in front of a single Kinect sensor. Body pose changes while the person moves and we allow a wide range of natural poses. b. The pose and shape parameters of a parametric body model are fit to the range data. The key idea is that body shape remains constant even though pose may vary. This allows multiple, low-fidelity, scans to be combined to produce a high-fidelity model. c. The resulting model can be animated and anthropometric measurements can be extracted with an accuracy approaching that of expensive laser scanners.
The 3D shape of the human body is useful for applications in fitness, games and apparel. Accurate body scanners, however, are expensive, limiting the availability of 3D body models. We have developed a method for human shape reconstruction from noisy monocular image and range data using a single inexpensive commodity sensor, the Microsoft Kinect.
The approach combines low-resolution image silhouettes with coarse range data to estimate a generative parametric model of the body. Any single view from the Kinect is noisy and only provides information about one side of the body. To obtain accurate 3D shape estimates we combine multiple monocular views of a person moving in front of the sensor. Each view captures different information about the person's shape.
The problem with using multiple views of the person is that they move between views and, as their pose changes, so does their body shape. To cope with varying body pose, we use a SCAPE body model that factors 3D body shape and pose variations. SCAPE combines a learned model of body shape with a model of how body shape varies with pose. Assuming the person's pose-independent shape does not change, we are able to estimate a single consistent body shape while allowing pose to vary across frames.
Optimization of body shape and pose is challenging because they are both high dimensional. To address this we have developed a novel method to minimize the distance between the projected 3D body contour and the image silhouette that uses analytic derivatives of the objective function. This provides a significant improvement over methods that use iterative closet point (ICP) methods.
We propose a simple method to predict standard body measurements from the recovered SCAPE model using linear regression. We show that the accuracy of our method is competitive with commercial body scanning systems costing orders of magnitude more. The result is the first demonstration of a Kinect-based body scanner.
There are many applications of such a scanner and our current research is focused on making it practical. In particular we are addressing the computational cost by developing new optimization methods and are performing a detailed accuracy study.
2
3 Body Labs, Inc.
