Abstract
In recent years, the deep learning-based hyperspectral image (HSI) classification method has achieved great success, and the convolutional neural network (CNN) method has achieved good classification performance in the HSI classification task. However, the convolutional operation only works with local neighborhoods, and is effective in extracting local features. It is difficult to capture interactive features over long distances, which affects the accuracy of classification to some extent. At the same time, the data from HSI have the characteristics of three-dimensionality, redundancy, and noise. To solve these problems, we propose a 3D self-attention multiscale feature fusion network (3DSA-MFN) that integrates 3D multi-head self-attention. 3DSA-MFN first uses different sized convolution kernels to extract multiscale features, samples the different granularities of the feature map, and effectively fuses the spatial and spectral features of the feature map. Then, we propose an improved 3D multi-head self-attention mechanism that provides local feature details for the self-attention branch, and fully exploits the context of the input matrix. To verify the performance of the proposed method, we compare it with six current methods on three public datasets. The experimental results show that the proposed 3DSA-MFN achieves competitive classification and highlights the HSI classification task.
Highlights
Received: 7 January 2022A hyperspectral image is a combination of imaging and spectroscopy to obtain highdimensional spatial and spectral information simultaneously
We compare the proposed 3DSA-MFN with a support vector machine (SVM) [71], a three-dimensional convolutional neural network (3D-CNN) [72], a spectral–spatial attention network (SSAN) [25], a spectral–spatial residual network (SSRN) [73], a hyperspectral image classification using the bidirectional encoder representation from transformers (HSI-BERT) [36], and a selfattention transformer network(SAT) [37]
In the 3D-CNN method, we randomly sample 20% of the data as the training set, the spatial size of the HSI cube is set to 11 × 11, the virtual sample augmentation method is used
Summary
Received: 7 January 2022A hyperspectral image is a combination of imaging and spectroscopy to obtain highdimensional spatial and spectral information simultaneously. Conventional machine learning classification methods were used to classify hyperspectral images [9,10,11,12,13,14,15,16], such as the K-nearest neighbor algorithm (KNN) [9], support vector machine (SVM) [10,11], and random forest (RF) [12], which are unable to automatically learn deep features and rely on prior expert knowledge, making effective feature extraction difficult for datasets with high-order nonlinear distributions. HSI classification methods based on deep learning have become increasingly popular. Because deep learning can extract deep abstract features effectively, it has gradually replaced the previous classification model with manually created features
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
