Input image `turtle' (1a.) and `panther' (1b.) with their ground truth separation (2.) and (3.) from `Ground truth dataset and baseline evaluations for intrinsic image algorithms' by Grosse et al. (ICCV09). Figures (4a.) and (5a.) show the results by our proposed method using cross validation to set the parameters. Figures (4b.) and (5b.) show the full flexibility of the proposed method by setting the parameters to the result that yields the lowest error.

Intrinsic Images
Peter Gehler, Martin Kiefel, Carsten Rother 1, Lumin Zhang 2, Bernhard Schölkopf 2


The problem of decomposing an image into its different intrinsic layers, such as shading, reflectance, and shape components, is one of the fundamental problems of computer vision. We believe that many computer vision tasks will benefit from having access to such separations. As an example, object detection should benefit from factoring out shadow and lighting effects, or image segmentation of materials should be improved by working on reflectance properties alone. In this project we aim to both decompose images into their intrinsic layers, and also show that this results to empirical improvements on different tasks. 

Our approach is to model the physical process of light interacting with materials. We model material properties, shape, and lighting conditions as separate sources that result in the observed pixel color and brightness. In "Recovering Intrinsic Images with a Global Sparsity Prior on Reflectance" we extend the Retinex theory by including models of material reflectances. Our conditional random field model encourages solutions in which the image is explained by a sparse set of reflectance types. This corresponds to the intuition that there are only few materials in an image. In the figure we show some example outputs of this model

Intrinsic image decomposition problems arise in many different contexts and we study several different variants that arise when some  scene modalities may be known. For example when using multiple depth cameras for full body scanning, we are provided with a good 3D estimate of the scene, or given a video we know that light varies smoothly. Our approach is to model the physical process using techniques from computer graphics together with probabilistic inference to reason about the unknown sources that constituted the scene. 

1 Microsoft Research Cambridge
2 MPI for Intelligent Systems
Recovering Intrinsic Images with a Global Sparsity Prior on Reflectance
Gehler, P., Rother, C., Kiefel, M., Zhang, L. and Schölkopf, B.
In Advances in Neural Information Processing Systems (NIPS), pages 765-773, 2011.