Residential building facade segmentation in the urban environment

Abstract

Building retrofit is an important facet in the drive to reduce global greenhouse gas emissions. However, delivering building retrofit at scale is a significant challenge, especially in how to automate the process of building surveying. On-site survey by expert surveyors is the main approach in the industry. This can lead to a high workload if planning retrofit at a large-scale. An advanced vehicle-mounted data capturing system has been built to collect urban environmental multi-spectral data. The data contains substantial information that is essential in identifying building retrofit needs. Although the data capturing system is able to collect data in a highly-efficient manner, the data analysis is still a big data challenge to apply the system into delivering building retrofit plans. In this paper, a street-view building facade image segmentation model is designed as the foundation of the holistic data analysis framework. The model is developed on the deep learning-based semantic segmentation technology and uses an ensemble learning strategy. The object detection technology is fused into the model as an magnifier to improve the model performance on small objects and boundary predictions. The model has achieved state-of-the-art levels of accuracy on a built street-view building facade image dataset.