Unsupervised domain adaptation for global urban extraction using Sentinel-1 SAR and Sentinel-2 MSI data

Abstract

Accurate and up-to-date maps of built-up areas are crucial to support sustainable urban development. Earth Observation (EO) is a valuable data source to cover this demand. In particular, Sentinel-1 Synthetic Aperture Radar (SAR) and Sentinel-2 MultiSpectral Instrument (MSI) missions offer new opportunities to map built-up areas on a global scale. Using Sentinel-2 images, recent urban mapping efforts achieved promising results by training Convolutional Neural Networks (CNNs) on available built-up data. However, these results strongly depend on the availability of local reference data for fully supervised training or assume that the application of CNNs to unseen areas (i.e. across-region generalization) produces satisfactory results. To alleviate these shortcomings, it is desirable to leverage Semi-Supervised Learning (SSL) algorithms that can take advantage of unlabeled data, especially because satellite data is plentiful. In this paper, we propose a novel Domain Adaptation (DA) approach using SSL that jointly exploits Sentinel-1 SAR and Sentinel-2 MSI to improve across-region generalization for built-up area mapping. Specifically, two identical sub-networks are incorporated into the proposed model to perform built-up area segmentation from SAR and optical images separately. Assuming that consistent built-up area segmentation should be obtained across data modality, we design an unsupervised loss for unlabeled data that penalizes inconsistent segmentation from the two sub-networks. Therefore, we propose to use complementary data modalities as real-world perturbations for consistency regularization. For the final prediction, the model takes both data modalities into account. Experiments conducted on a test set comprised of sixty representative sites across the world showed that the proposed DA approach achieves strong improvements (F1 score 0.694) over fully supervised learning from Sentinel-1 SAR data (F1 score 0.574), Sentinel-2 MSI data (F1 score 0.580) and their input-level fusion (F1 score 0.651). To demonstrate the effectiveness of DA, we also performed a comparison with two state-of-the-art products, namely GHS-BUILT-S2 and WSF 2019, on the test set. The comparison showed that our model is capable of producing built-up area maps with comparable or even better quality than the state-of-the-art global human settlement maps. Therefore, the multi-modal DA offers great potential to be adapted to produce easily updateable human settlements maps at a global scale.