Interactive piecewise planar building reconstruction from a single image based on geometric priors

Abstract

The development of methods to reconstruct the piecewise planar structures of buildings from a single image is an important and challenging problem in the field of computer vision. Although most prior works have considered fully automatic reconstruction solutions, these methods typically perform poorly for complex building structures. To address this problem, we propose an interactive single-image 3D building reconstruction method to integrate automatic plane inference with interactive plane refinement based on geometric priors. In this method, based on geometric priors (e.g., the orientation of the plane and the intersection angle between planes) learned using convolutional neural networks, building regions are first partitioned into multiple polygonal regions using click interactions. Then, the planes corresponding to these polygonal regions are automatically and progressively inferred by taking full advantage of the constraints constructed using the geometric priors. Finally, under the guidance of interactive plane refinement based on click correlations, the resulting planes are globally optimized by jointly utilizing image cues and geometric priors. The results of an experimental evaluation on three datasets demonstrate that the proposed method achieved better performance than existing methods in terms of both qualitative and quantitative measures.