Semantic segmentation of urban building surface materials using multi-scale contextual attention network

Abstract

Distributed solar photovoltaic (PV) harvesting is an effective way to collect solar energy in existing metropolitan cities. To facilitate the installation of PV modules at solar abundant locations, an accurate estimation of solar PV spatial potential is indispensable. Solar energy could be reflected on high-albedo building surfaces inside the urban canyon. However, using conventional ways to construct albedo datasets for different building surfaces is extremely labor-intense. In this work, we proposed to use semantic segmentation to identify façade materials from street view images. Due to the distinguishable features between materials in terms of the subtle texture and patterns rather than just their shapes and colors, identification requires more details from images, which makes multi-scale inference structure a promising solution. Compared with existing methods combining scales features at pixel-level, we proposed a novel Multi-Scale Contextual Attention Network (MSCA), using a Multi-Scale Object-Contextual Representation (OCR) block to exploit and combine contextual information from different scales in high dimensional layers. The experimental results show that the proposed model significantly outperforms the existing models, achieving a mean Intersection over Union (mIOU) of 70.23%. The results indicate that the MSCA can effectively obtain the materials information from street views and can be a reliable solution to providing urban albedo information for solar estimation.