Attention-based multiscale deep learning with unsampled pixel utilization for hyperspectral image classification

Abstract

In this research, a deep learning approach for hyperspectral image (HSI) classification was developed, incorporating attention mechanisms, multiscale feature learning, and utilization of unsampled pixels. The proposed model, multiscale attention-based hybrid spectral network and UNet (MSA-HybridSN-U), was evaluated on three benchmark datasets: Indian Pines, University of Pavia, and University of Houston using Overall Accuracy (OA), Average Accuracy (AA), and Kappa index (K). The proposed model MSA-HybridSN-U had high accuracy on Indian Pines (IP), Pavia University (PU), and Houston University (HU) datasets. It achieved the highest average accuracy compared with other models like 3D CNN, HybridSN, M3D-DCNN, DBDA, and ACA-HybridSN and higher than 2D CNN and SVM. For the IP dataset, the OA and AA scores are both 99.71% and 99.65% respectively, indicating a high level of accuracy in classifying the samples. The Kappa statistic, which measures inter-annotator agreement, is also high at 0.997, suggesting that the method is consistent in its predictions. For the PU dataset, the results are even higher with OA and AA scores of 99.97% and 99.89% respectively, and a Kappa statistic of 0.999, indicating even higher accuracy and consistency. Finally, for the HU dataset, the results are similarly high with OA and AA scores of 99.47% and 99.92% respectively, and a Kappa statistic of 0.994. The use of attention mechanisms, multiscale features, and unsampled pixels has improved the classification performance of the model. The combination of spectral and spatial attention modules improved the accuracy the most, with the highest accuracy of 99.97% on the PU dataset. Utilizing unsampled pixels in the classification process resulted in a noticeable improvement in accuracy, particularly on the HU dataset with an improvement of 0.29%. The results show the effectiveness of combining multiscale features and attention modules in improving the accuracy of HSI classification.