Automated detection of exterior cladding material in urban area from street view images using deep learning

Abstract

Street View Images (SVIs) can be utilized in urban analysis to assess the exterior cladding materials of buildings. However, this process is labor-intensive and time-consuming, particularly when applied to extensive geographic areas. Currently, there is no automated study to streamline this manual process. This research introduces a deep learning approach suitable for image classification and demonstrates its application in two case study areas: London and Scotland. Six material types were identified: ‘Brick’, ‘Concrete’, ‘Glass’, ‘Stone’, ‘Mixed’, and ‘Others’. To ensure reliable accuracy, several advanced architectures were employed: InceptionV3, EfficientNetV2, ResNet-101, ResNet-152, and MobileNetV3. Transfer learning was applied to each architecture, and six distinct image augmentation techniques were utilized to artificially expand the training dataset. Consequently, ten models were developed for each area by combining five different architectures with two datasets: the original and the augmented. The best-performing model was selected for evaluation on an unseen dataset. For London and Scotland, the model using MobileNetV3 with augmented data emerged as the most effective, achieving average accuracy rates of 75.65 % and 73.45 %, respectively. Based on these findings, the paper explores the potential applications of the proposed approach.