Facade Segmentation

Research > Light into Meaning > Facade Segmentation

Schematic of different components in our facade segmentation pipeline with a sample facade from ECP dataset

Overview
Datasets
Code
Results
Contact
Publications

Facade Segmentation

Varun Jampani, Peter Gehler

Paper Title: Efficient Facade Segmentation using Auto Context

Authors: Varun Jampani*, Raghudeep Gadde* and Peter V. Gehler (*equal contribution)

Abstract: In this paper we propose a system for the problem of facade segmentation. Building facades are highly structured images and consequently most methods that have been proposed for this problem, aim to make use of this strong prior information. We are describing a system that is almost domain independent and consists of standard segmentation methods. A sequence of boosted decision trees is stacked using auto-context features and learned using the stacked generalization technique. We find that this, albeit standard, technique performs better, or equals, all previous published empirical results on all available facade benchmark datasets. The proposed method is simple to implement, easy to extend, and very efficient at test time inference.

Affiliations: Varun Jampani and Peter Gehler are with MPI for Intelligent Systems in Tübingen. Raghudeep Gadde is with Ècole des Ponts ParisTech and Ècole Centrale Paris in Paris.

Main paper: [pdf]

Supplementary material: [pdf]

Facade Segmentation Benchmarks

ECP Dataset [1]: This prominent dataset consists of 104 Hausmannian architectural buildings from Paris. There are seven semantic classes in this dataset. Can be accessed from here: url.
Graz Dataset [2]: Has 50 facade images of various architectures (Classicims, Biedermeier, Historicism, Art Noveau) from buildings in Graz. There are 4 semantic classes in this dataset. Can be accessed here: url.
eTRIMS Dataset [3]: Consists of 60 non-rectified facade images which are more irregular and follow only weak architectural principles. Can be accessed here: url.
CMP Dataset [4]: Has 378 rectified facades of diverse styles and 12 semantic classes in its base set. Can be accessed here: url.
LabelMeFacade Dataset [5]: This consists of building facade images taken from LabelMe segmentation dataset [6]. Facades in this dataset are highly irregular with lot of diversity across images. More information here: url.

References:

[1] Teboul, O. Ecole centrale paris facades database. 2010.

[2] Riemenschneider, H., Krispel, U., Thaller, W., Donoser, M., Havemann, S., Fellner, D., & Bischof, H. Irregular lattices for complex shape grammar facade parsing. In Computer Vision and Pattern Recognition (CVPR), pp. 1640-1647, June 2012.

[3] Korc, F., & Förstner, W. eTRIMS Image Database for interpreting images of man-made scenes. Dept. of Photogrammetry, University of Bonn, Tech. Rep. TRIGG-P-2009-01, April 2009.

[4] Tyleček, R., & Šára, R. Spatial Pattern Templates for Recognition of Objects with Regular Structure. In Proc. of German Conference on Pattern Recognition (GCPR), pp. 364-374, 2013.

[5] Frohlich, B., Rodner, E., & Denzler, J. A fast approach for pixelwise labeling of facade images. In International Conference on Pattern Recognition (ICPR), pp. 3029-3032, 2010.

[6] Russell, B. C., Torralba, A., Murphy, K. P., & Freeman, W. T. LabelMe: a database and web-based tool for image annotation. International journal of computer vision, 77(1-3), 157-173, 2008.

Coming soon...

Results with 3-stage Auto-Context classification

Here, we provide facade segmentation results obtained using our auto-context segmentation framework to facilitate comparisons and further research. If you find these results useful for your publication, please consider citing our paper:

@inproceedings{jampani15wacv,
  title = {Efficient Facade Segmentation using Auto-Context},
  author = {Jampani, Varun and Gadde, Raghudeep and Gehler, Peter V.},
  booktitle = {IEEE Winter Conference on Applications of Computer Vision (WACV)},
  month = jan,
  url = {http://wacv2015.org},
  year = {2015}
}

For each dataset, we include the following:

Stage-3 Probabilities: They contain the classification probability output of third stage classifier in our auto-context classifiation framework. Results for each image are stored in separate binary file. Refer to this readme file [txt] on how to read the binary files.
Folds: Image file names present in each of the folds used in our experiments.
Quantitative results: These excel files contain the pixel accuracies and IoU scores for each individual image separately. Note that these are different from the quantitative results reported in the paper which are computed by using the pixels of all the images together.
Visual results: PNG image files showing the stage-3 visual segmentation results for each image in the dataset.