Adversarial MixUp with implicit semantic preservation for semi-supervised hyperspectral image classification

Abstract

MixUp generates virtual samples by performing the identical convex combination in feature and semantic spaces simultaneously. However, convexly combining random pairs of images and their associated labels is likely to result in misunderstandings between the generated sample and its assigned pseudo label, particularly for hyperspectral images (HSI), leading to a confusing effect on the classifier. In this paper, adversarial MixUp with implicit semantic preservation (AdvMixup) is proposed for semi-supervised HSI classification (HSIc). The proposed AdvMixup exploits interpolation in the latent space for unlabeled samples. The virtual hyperspectral sample is generated by a deconvolutional network on the interpolated latent data. A semantic preservation network is performed on the newly generated virtual sample to recover the MixUp coefficient, where the semantic is preserved implicitly in the virtual sample. Meanwhile, an adversarial regularization is also conducted on the recovered mixup coefficient to ensure the authenticity of the virtual HSI. In such a way, the proposed AdvMixup sees the unseen HSI and knows the unknown semantic. Throughout the learning process, the feature extraction network is shared in both supervised and unsupervised branches to obtain a superior decision boundary. Experimental results on three HSI benchmark datasets demonstrate that AdvMixup achieves state-of-the-art performance in semi-supervised HSI classification.