A Dataset and Ensemble Model for Glass Façade Segmentation in Oblique Aerial Images

Abstract

Glass façade segmentation provides important semantic information for widespread application, such as light pollution assessment, energy conservation, emergency response, etc. With the advances in Unmanned Aerial Vehicle (UAV) technology, aerial imagery is indispensable for urban environmental studies because it can quickly cover large-scale city areas that contain rich thematic information. However, various methods are elaborately designed for glass or transparent objects detection in natural images but not for glass façades in oblique aerial images. In oblique aerial images, glass façade panels with salient boundaries are spatially related and have various sizes and appearances. For the glass façade segmentation task, we notice that semantic and instance segmentation methods have different strengths and failures. To get the best of both worlds, this letter presents an ensemble method to segment glass façades in oblique aerial images. A novel annotation strategy is adopted to preserve the spatial relationship between glass façade panels, and edge features are extracted to improve the detector’s performance. Furthermore, we contribute a glass façade segmentation dataset (GFSD) to evaluate our method, which is a supplement to UAV image datasets. Experimental results show that the proposed strategy effectively improves instance segmentation methods, and the ensemble method outperforms the single models with an F1 score of 95.48%.