Cross-spatiotemporal land-cover classification from VHR remote sensing images with deep learning based domain adaptation

Abstract

Automatic land use/land cover (LULC) classification from very high resolution (VHR) remote sensing images can provide us with rapid, large-scale, and fine-grained understanding of the urbanization and ecosystem processes on the Earth’s surface. Although the burgeoning use of deep learning technology has boosted land cover classification from VHR images, one of the key challenges has not been investigated in depth, i.e., the current data-driven deep learning models are heavily reliant on the high similarity between the distributions of the labeled training data, i.e., the source data, and the unlabeled data, i.e., the target data. However, in practice, this condition is rarely met, and preparing labels manually every time for the target data is unrealistic. In this paper, we comprehensively evaluate the domain adaptation methods for modern deep learning based classification models. Domain adaptation is aimed at narrowing the domain shift between the labeled source data and the unlabeled target data, providing a practical way for a deep learning based model to fully utilize historical training data and get rid of the need for continual manual work. Furthermore, we propose a novel two-stage Domain Adaptation method for Cross-Spatio-Temporal classification called the DACST method with the inputs of labeled source data and the unlabeled target data. It consists of an image-level adaptation stage that aligns the appearance of the source and target data and produces the target-stylized source images, and a feature-level adaptation stage that further narrows the domain shift in the deep feature space. DACST significantly improves the spatiotemporal transferability of the classification model, which is embedded in the feature-level adaptation stage, to output a satisfactory classification map. In this study, we also conducted a comprehensive performance evaluation of the conventional and deep learning based image-level and feature-level domain adaptation methods for VHR LULC classification. Both binary and multi-class classification was conducted in cross-temporal and cross-spatiotemporal scenes in five large-scale datasets from around the world. The very high performance and the best robustness of the proposed method suggests that a new baseline of cross-domain VHR land cover classification in the deep learning age is being witnessed. The experiments also indicate that both the conventional and deep learning based image-level domain adaptation methods function in various situations, but almost all the feature-level methods are highly unstable on different data.