Semi-supervised hierarchical Transformer for hyperspectral Image classification

Abstract

ABSTRACT Transformer has achieved outstanding performance in many fields such as computer vision benefiting from its powerful and efficient modelling ability and long-range feature extraction capability complementary to convolution. However, on the one hand, the lack of CNN’s innate inductive biases, such as translation invariance and local sensitivity, makes Transformer require more data for learning. On the other hand, labelled hyperspectral samples are scarce due to the time-consuming and costly annotation task. To this end, we propose a semi-supervised hierarchical Transformer model for HSI classification to improve the classification performance of the Transformer with limited labelled samples. In order to perturb the samples more fully and extensively to improve the model performance, two different data augmentation methods are used to perturb the unlabelled samples, and two sets of augmented samples are obtained respectively. The pseudo-label obtained on the original unlabelled sample is used to simultaneously supervise the augmented sample obtained on this unlabelled sample. Among them, only the pseudo-labels above the threshold are retained. To further improve the model stability and classification accuracy, hierarchical patch embedding is proposed to eliminate the mutual interference between pixels. Extensive experiments on three well-known hyperspectral datasets validate the effectiveness of the proposed semi-supervised Transformer model. The experiments show that the model achieves excellent classification accuracy even when there are only 10 labelled samples in each category, which can effectively improve the classification performance of Transformer under small-scale labelled samples.