Feedback-Enhanced Few-Shot Transformer Learning for Small-Sized Hyperspectral Image Classification

Abstract

In hyperspectral image (HSI) classification, few-shot learning (FSL) methods achieve satisfactory performance with limited samples. However, the labeled and unlabeled data are not utilized effectively, and the convolution operations restrict the spectral signature extraction for classification in FSL. In this letter, a feedback-enhanced few-shot transformer network (FFTN) is proposed for the small-sized HSI classification. FFTN consists of two feedback learning processes, which are designed to train a learnable classifier on the source domain and transfer it to the target domain. The first process is meta-learning with reinforced feedback training on the source domain, which trains a classifier to learn classification and improves the distinguishing ability of misclassified classes by reinforcement. The second process is target-learning with transductive feedback training on the target domain, where the classifier is transferred from the source to the target domain and learns the distribution of unlabeled samples. The transformer component in FFTN also merges the spatial and spectral attention to extract accurate and representative features. That is, FFTN not only transfers knowledge between different domains by feedback few-shot learning, but also reinforces the feature extraction and exploits the distribution of unlabeled samples to obtain the optimal generalization performance. The results of the comparison experiment on public HSI data sets demonstrate that FFTN provides a competitive classification performance with small-sized samples.