Multi-scale spatial-spectral fusion based on multi-input fusion calculation and coordinate attention for hyperspectral image classification

Abstract

Recently, the deep learning method that integrates image features has gradually become a hot development trend in hyperspectral image classification. However, these studies did not fully consider the fusion of image features, and did not remove the interference to the classification process caused by the difference in the size of the objects. These factors hinder the further improvement of the classification effect. To eliminate these drawbacks, this paper proposes a more effective fusion scheme (MSF-MIF), which realizes the fusion from the perspective of location characteristics and channel characteristics through 3D convolution and spatial feature concatenation. In view of the size discrepancy of the objects to be classified, this method extracts features from several input patches of different scales and uses the novel calculation method proposed to fuse them, which minimizes the interference caused by size differences. In addition, this research also tried to quote the coordinate attention structure for the first time that combines spatial and spectral attention features to further improve the classification performance. Experimental results on three commonly used data sets prove that this framework has achieved a breakthrough in classification accuracy.