Large-scale deep learning based binary and semantic change detection in ultra high resolution remote sensing imagery: From benchmark datasets to urban application

Abstract

With the acceleration of urban expansion, urban change detection (UCD), as a significant and effective approach, can provide the change information with respect to geospatial objects for dynamic urban analysis. In recent years, through the use of machine learning and artificial intelligence, change detection methods have gradually developed from the traditional pixel-based comparison methods in the 1980s to data-driven deep learning methods. Deep learning methods have huge advantages in the application of remote sensing big data, by virtue of their huge feature extraction and expression capabilities. Many change detection datasets have been released to meet the requirements of deep learning. However, the existing datasets suffer from three bottlenecks: (1) the volume of the datasets is small, which can easily cause overfitting; (2) most datasets have a spatial resolution of meters, making it difficult to detect changes in small objects because there are multi-scale objects in urban areas; and (3) most of the datasets have been designed for binary change detection (BCD), and lack semantic annotation, so that they cannot be used to obtain the direction of the change or to analyze the type of change, for further application in urban areas. Therefore, it is difficult to apply these datasets to detect large-scale urban semantic changes in complex environments. To address these issues, a large-scale ultra high resolution (0.1m) UCD dataset for deep learning based BCD and semantic change detection (SCD) is introduced in this article, which is named the Hi-UCD dataset. We selected an area of 102m2 in Tallinn, the capital of Estonia, as the study area. There are a total of 40800 pairs of 512 × 512 patches, nine types of land cover and 48 types of semantic change in the Hi-UCD dataset. We developed three metrics–binary consistency, change area consistency and no-change area consistency–to evaluate the semantic consistency of the SCD methods from different aspects. A comprehensive analysis and investigation is provided in this article after we benchmarked this dataset using deep learning methods for BCD and SCD. We also found that the unchanged samples can help distinguish the changed area, and HRNet used as the backbone to construct a multi-task model can perform well in the Hi-UCD dataset. Meanwhile, the visualization results obtained with the Hi-UCD test set, which is a large geographic area covering 54km2, are shown to reflect the real-world urban application scenarios. The experimental results show that the Hi-UCD dataset is a challenging yet useful benchmark dataset, which can be used for analyzing large-scale refined urban changes.