Multiscale Dense Cross-Attention Mechanism with Covariance Pooling for Hyperspectral Image Scene Classification

Abstract

In recent years, learning algorithms based on deep convolution frameworks have gradually become the research hotspots in hyperspectral image classification tasks. However, in the classification process, high-dimensionality problems with large amounts of data and feature redundancy with interspectral correlation of hyperspectral images have not been solved efficiently. Therefore, this paper investigates data dimensionality reduction and feature extraction and proposes a novel multiscale dense cross-attention mechanism algorithm with covariance pooling (MDCA-CP) for hyperspectral image scene classification. The multisize convolution module can detect subtle changes in the hyperspectral images’ spatial and spectral dimensions between the pixels in the local areas and are suitable for extracting hyperspectral data with complex and diverse types of structures. For traditional algorithms that assign attention weights in a one-way manner, thus leading to the loss of feature information, the dense cross-attention mechanism proposed in this study can jointly distribute the attention weights horizontally and vertically to efficiently capture the most representative features. In addition, this study also uses covariance pooling to further extract the features of hyperspectral images from the second order. Experiments have been conducted on three well-known hyperspectral datasets, and the results thus obtained show that the MDCA-CP algorithm is superior compared to the other well-known methods.