Coarse-to-Fine Joint Distribution Alignment for Cross-Domain Hyperspectral Image Classification

Abstract

Domain adaptation aims to enhance the feature transferability of a model across different domains with feature distribution differences, which has been widely explored in many computer vision tasks such as semantic segmentation and object detection, but has not been fully studied in hyperspectral image (HSI) classification task. Compared with the natural image-based domain adaptation, HSI-based domain adaptation still faces two main challenges: 1) Due to the strong spectral variability of HSIs, it is difficult to extract discriminative and domain-invariant features from different domains, resulting in the misalignment of cross-domain features; 2) Class-wise (or fine-grained) spectral feature inconsistency between domains also inevitably degrades the classification accuracy. To address these issues, in this paper, we propose a novel coarse-to-fine joint distribution alignment framework for cross-domain classification of HSIs. Specifically, the training samples from source and target domains are first fed into a coupled variational auto-encoders (VAE) module, which is composed of two well-designed VAEs equipped with mutual information metric to learn high-level domain-invariant representations in a shared latent space, so that the network can learn a coarse-grained source-target feature consistency. Furthermore, to alleviate the class-wise inter-domain feature inconsistency, a joint distribution alignment (JDA) module is constructed to perform a fine-grained cross-domain alignment by matching the joint probability distributions between the source and target domains through adversarial learning. Extensive experiments on both simulated and real hyperspectral datasets demonstrate the superiority of the proposed method in comparison with several conventional and state-of-the-art methods.