Siamese Spectral Attention With Channel Consistency for Hyperspectral Image Classification

Abstract

Abundant spectral features are the precious wealth of hyperspectral images (HSI). Nevertheless, well-designed spectral feature is still a challenge that affects the performance of the classifier, especially with insufficient number of training samples. To make up the poor discriminability of spectral feature, double-branch methods are proposed by fusing parallel spectral and spatial branches. However, this structure does nothing to improve the quality of spectral feature, which is regarded as the most valuable information for HSI information. In this article, we propose a siamese spectral attention network with channel consistency (SSACC) to focus on obtaining discriminative spectral features, thus improving the generalization ability of the classifier. Two kinds of HSI cubes with different patch sizes are generated as the input of SSACC. The two cubes are divided into top and bottom branches and then be fed into the siamese network to obtain the refined spectral features. Then, self-attention is conducted to interacting with each channel for the spectral features enhancement. Meanwhile, two attention maps are obtained to display the spectral structures of each branch. A channel consistency regularization is performed on the two attention maps by enforcing the two branches to possess similar spectral patterns when identifying the same centric pixel. Extensive experiments conducted on the three HSI datasets verify the superiority of the obtained spectral feature. Furthermore, the proposed method applying convolution only on the spectral domain outperforms the state-of-the-art double-branch methods which integrate the spectral and spatial features simultaneously.