Optimal transport-based domain adaptation for semantic segmentation of remote sensing images

Abstract

ABSTRACT Thanks to its great power in feature representation, deep learning (DL) is widely used in semantic segmentation tasks. However, the requirements for high distribution consistency of different domains are too tight to be met by large-scale remote sensing tasks due to the domain shift in imaging modes and geographic environments. In this case, trained models in a source domain can hardly achieve sufficient accuracy in a target domain with domain shift. To address this issue, a novel unsupervised domain adaptation (UDA) method driven by optimal transport (OT) with two-stage training is proposed to alleviate domain shift in remote sensing images (RSIs). In the first stage, a colour distribution alignment (CDA) module and a feature joint alignment (FJA) module based on OT were designed to mitigate the discrepancy between different domains. CDA transports source-domain distribution according to the target-domain colour style, and FJA aligns source and target domains in both feature and output spaces by minimizing OT-based losses. In the second stage, self-training with pseudo-label denoising (STPD) was proposed, which alleviated the interference of noises in pseudo-labels based on a joint OT distance. For the experiments, the Potsdam, Vaihingen and UAVid datasets were employed. Based on the characteristics of these datasets, five UDA tasks were introduced. The results of these UDA experiments indicate the superiority of our method. Our code will be available at https://github.com/Hcshenziyang/OT-Domain-Adaptation-Semantic-Segmentation.