Hyperspectral image classification based on local feature decoupling and hybrid attention SpectralFormer network

Abstract

ABSTRACT Classification of hyperspectral image (HSI) is an essential step in various remote sensing tasks. Currently, transformer-based models have made significant contributions to HSI classification due to their remarkable ability to model the relation among long-range sequence data. However, HSI data exhibit strong nonlinear coupling and strong correlation, leading to a decrease in the classification performance of transformer-based models. The SpectralFormer network has achieved excellent performance in HSI classification, but its cross-layer adaptive fusion mechanism cannot effectively highlight critical information, affecting the representation ability of spectral-spatial features. To address the above issues, this paper proposes an HSI classification method based on local feature decoupling and SpectralFormer with a hybrid attention module (HAMSF) network. Specifically, this paper first uses the histogram of oriented gradient algorithm to preprocess HSI coupled data and achieve preliminary nonlinear decoupling of HSI data. Secondly, the hybrid attention module is embedded in a cross-layer adaptive fusion mechanism, generating refined neighbouring information before each information fusion to improve the spectral-spatial feature representation ability. Finally, the processed data are input into the HAMSF network for the final decoupling and classification of HSI. The experiment is carried out on three benchmark hyperspectral datasets, and the experimental results confirm that the proposed method has better classification performance compared to several transformer-based and classical HSI classification methods.