Attention Multisource Fusion-Based Deep Few-Shot Learning for Hyperspectral Image Classification

Abstract

Recently, deep learning-based methods outperform others in hyperspectral image (HSI) classification. However, deep learning methods require sufficient labeled samples to improve performance, which is unfeasible in practice. The training labels are usually limited in HSIs that need to be classified (namely target domain), while other available labels in multisource HSIs (namely source domain) are not utilized effectively. To mitigate these issues, an attention multisource fusion method of few-shot learning (AMF-FSL) is proposed for small-sized HSI classification. AMF-FSL is an implementation of few-shot learning (FSL) in the meta-learning field, which can transfer the learned ability of classification from multiple source data to target data. The process of learning to classify in AMF-FSL is not restricted by the traditional requirement of the same distribution between the source and target domains, which can learn from the source domain and apply it to a different distribution in the target domain. Moreover, the multisource domain adaption in AMF-FSL has the capacity of extracting features from fused homogeneous and heterogeneous data in the source domain, which can improve the generalization of the classification model in the cross domains. Specifically, the multisource domain adaption contains three modules, namely the target-based class alignment, domain attention assignment, and multisource data fusion, which are responsible for aligning the class space, paying band-level attention, and merging the distributions of homogeneous and heterogeneous data in the source domain. The experimental results demonstrate the effectiveness of the multisource domain adaption and the superiority of AMF-FSL over other state-of-the-art methods in small-sized HSI classification