Building façade element extraction based on multidimensional virtual semantic feature map ensemble learning and hierarchical clustering

Abstract

• Definition of multidimensional semantic features of the building façade. • Multidimensional virtual semantic feature map are constructed based on Multiple View Stereo techniques. • Ensemble-learning are used for multi-semantic feature block pre-segmentation. • Color&Texture, orientation and curvature semantics segmentation. • Different virtual semantic feature map and clustering strategies are combined to accurately extract of diverse building façade elements. Building façade elements are an important foundation for smart cities. As buildings exhibit an array of textures and geometric forms, the process of image acquisition is easily affected, although the robustness of texture in scenes (e.g., dilapidated buildings) is poor, with high point cloud data, and low recognition efficiency; therefore, the accuracy of building element extraction based on a single data source remains limited. In this research, a method for building façade element extraction based on multidimensional virtual semantic feature map ensemble learning and hierarchical clustering is proposed. Point clouds were obtained by multi-view images, and then the multidimensional virtual semantic feature maps, including color, texture, orientation, and curvature semantics, were acquired via reprojection. The multi-semantic feature block pre-segmentation, considering multiple features, was obtained by ensemble learning, and a hierarchical clustering strategy was established for to achieve fine extraction of building façade elements. Experiments were conducted across multiple building types, and the results showed that: 1) The method can use different virtual semantic feature map and clustering strategies to achieve accurate extraction of diverse building façade elements; 2) The method achieved joint learning tasks in both 2D and 3D space; and, 3) The proposed method achieved fine extraction of building elements with pixel accuracy (PA) over 70% in all experiments and mean intersection over union (mIoU) up to 95%, which were better than the image based method. In summary, this method offers a novel, more reliable method for segmenting and extracting building façade elements, which has important theoretical and practical significance.