Residual networks with multi-attention mechanism for hyperspectral image classification

Abstract

Residual network which can effectively overcome gradient disappearance in convolutional neural networks has been successfully applied to hyperspectral classification. Yet simply accumulating residual units does not improve the model performance. Besides, the lack of training samples seriously restricts the hyperspectral classification accuracy. In attempt to solve the abovementioned problems, residual networks with multi-attention mechanism (RNMA) is proposed. The spectral-spatial attention module (SSAM) in accordance with the structure characteristics of hyperspectral images is introduced into the RNMA method. The SSAM is embedded after the input which can strengthen bands relevant to ground object and spatial salient regions of ground object while weaken less informative features. Meanwhile, channel attention module is introduced into residual network unit to form residual network unit with channel attention module (RUCA). Based on RUCA, the features of the spectral dimension and the spatial dimension are sequentially extracted. Besides, sample augmentation method with spectral and spatial constraints is applied to expand sample size. Experiments results on three hyperspectral datasets show that the RNMA model outperforms than other methods in classification accuracy. The RNMA model achieved superior classification result compared with other state-of-the-art methods in the small training sample size.