Adaptive spectral-spatial feature fusion network for hyperspectral image classification using limited training samples

Abstract

Recently, the excellent power of spectral-spatial feature representation of convolutional neural network (CNN) has gained widespread attention for hyperspectral image (HSI) classification. Nevertheless, the practical performance of CNN-based models in HSI classification is ordinarily limited by the available amount of the training samples. In this article, we investigate the limitations of current CNN-based methods for HSI feature (spectral and spatial) extraction and utilization. For spectral features, the distant inter-band relationships are often neglected. Therefore, a novel spectral band non-localization (SBNL) operation is proposed to enable the non-local spectral inter-band correlations to be excavated by convolutional kernels with limited receptive fields. For spatial features, the extracted spatial multiscale features conventionally isolated in different channels. Subsequently, we develop a novel multiscale-share inception block (MSIB) to exploit the cross-relationships among the multiscale features. More significantly, to better take advantage of the complementary information of spectral and spatial features, a plug-and-play adaptive feature fusion (AFF) module is introduced. Eventually, the adaptive spectral-spatial feature fusion network (AS2F2N) is introduced for HSI classification. Experimental results derived from three benchmark data sets exhibit that the proposed method outperforms previous state-of-the-art CNN-based methods under limited training samples situation. The codes of this work will be available at https://github.com/zhonghaocheng/ELSEVIER_IJAEOG_AS2F2N for the sake of reproducibility.