Automatizing the generation of building usage maps from geotagged street view images using deep learning

Abstract

Building usage maps are inputs in various urban applications. Although Street View Images (SVIs) are applied in many studies, their usage in the generation of building usage maps is limited. Further, their application to Indian cities remains void. This research aims to fill the discussed gaps by employing geotagged SVIs through web-scraping, Google Street View (GSV), and manual collection. Manually collected data from mobile phone and web-scraped images also cover the dense regions, which are generally not covered in GSV. Hence, it helps the generation of accurate maps. We first applied a pretrained Inception-ResNet-v2 model to detect the building instances from training images of various Indian cities. The extracted building instances were manually labeled as different usage class (residential, commercial, industrial, and others) to develop the classification training set. EfficientNet-B7, a Deep Learning (DL) model leveraged the façade features in the training process to predict the usage class of the building instances. The model achieved an accuracy of 81% on training and 77% on test sets. The predictions were attributed with building footprint using Geographic Information Systems (GIS) to prepare building usage maps for various cities in India. Although the model faced challenges in predicting a lot of buildings with similar façade, we still found it to be robust in predicting the usage of buildings across various geography. The study can act as a decision-making tool in various smart applications that require high-resolution building-use data, which can help developing sustainable cities.