The real-time and stack fusion enhanced dual-channel network with attention modules for fast hyperspectral image classification

Abstract

Hyperspectral image (HSI) classification with convolutional neural network (CNN) exhibiting excellent performance, but the improvement of computationally efficient is yet a challenging research task. Therefore, two fusion strategies and two attention modules have been introduced to enhance for a dual-channel structure, which consists of 2-D convolution branch for pure spatial features and 3-D convolution branch for joint spectral-spatial features. The real-time fusion is applied to each convolution layer between two branches, which enhanced information interaction within the network. Meanwhile, the stacked fusion strategy with the adaptive weights is introduced to adjust the influence of the two branches on the final classification results. Furthermore, the spectral-spatial reconstruction attention module (SSRAM) and adaptive channel attention module (ACAM) are designed to enhance features representative capability for 3-D branch and 2-D branch, respectively. All approaches introduced above allow us to construct a dual-channel network consisting of small convolution kernels, which achieve HSI classification accurately and efficiently. Experimental results on three public HSI datasets show that the proposed network outperforms several state-of-the-art models in terms of classification accuracy and computational efficiency.