Mapping Irregular Local Climate Zones from Sentinel-2 Images Using Deep Learning with Sequential Virtual Scenes

Abstract

Recently, the local climate zone (LCZ) system has been presented to establish the connection between urban landscape and local thermal environment. However, LCZ entities are very difficult to be identified by pixel-based classifiers or object-oriented image analysis, as they are often a complicated combination of multiple ground objects (e.g., buildings, roads, grassland, etc.). Scene classifiers, especially deep learning methods can exploit the structure or contextual information of image scenes and then improve the performance of LCZ classification. However, the square and uniform-sized image patches often bring about extra challenges, as they cannot exactly match LCZ entities of diverse sizes and shapes in most cases. In this study, a sequential virtual scene method is presented to identify LCZ entities of diverse shapes and sizes, which consists of a small “core patch” for scanning diverse entities and sequential virtual scenes for providing abundant context. Specifically, the Bidirectional Long Short-Term Memory (Bi-LSTM) were used to learn the spatial relationship among virtual scenes, respectively. Importantly, a “self-attention” mechanism is designed to weigh the contribution of every virtual scene for alleviating the influences of mixed patches, according to the similarity between its hidden state and the final hidden state. Experiments prove SVS achieves better accuracies than random forest and ResNet and has the outstanding capacity of identifying irregular LCZ entities. It is a promising way to carry out LCZ mapping in cities of different types due to its flexibility and adaptability.