Automatic façade recovery from single nighttime image

Abstract

Nighttime images are difficult to process due to insufficient brightness, lots of noise, and lack of details. Therefore, they are always removed from time-lapsed image analysis. It is interesting that nighttime images have a unique and wonderful building features that have robust and salient lighting cues from human activities. Lighting variation depicts both the statistical and individual habitation, and it has an inherent man-made repetitive structure from architectural theory. Inspired by this, we propose an automatic nighttime facade recovery method that exploits the lattice structures of window lighting. First, a simple but efficient classification method is employed to determine the salient bright regions, which may be lit windows. Then we group windows into multiple lattice proposals with respect to facades by patch matching, followed by greedily removing overlapping lattices. Using the horizon constraint, we solve the ambiguous proposals problem and obtain the correct orientation. Finally, we complete the generated facades by filling in the missing windows. This method is well suited for use in urban environments, and the results can be used as a good single-view compensation method for daytime images. The method also acts as a semantic input to other learning-based 3D image reconstruction techniques. The experiment demonstrates that our method works well in nighttime image datasets, and we obtain a high lattice detection rate of 82.1% of 82 challenging images with a low mean orientation error of 12.1 ± 4.5 degrees.