Pyramidal Multiscale Convolutional Network With Polarized Self-Attention for Pixel-Wise Hyperspectral Image Classification

Abstract

In recent years, pixel-wise hyperspectral image (HSI) classification has received growing attention in the field of remote sensing. Plenty of spectral-spatial convolutional neural network (CNN) methods with diverse attention mechanisms have been proposed for HSI classification due to the attention mechanisms being able to provide more flexibility over standard convolutional blocks. However, it remains a challenge to effectively extract multi-scale features of high-resolution HSI in a real-world complex environment. In this paper, we propose a pyramidal multi-scale spectral-spatial convolutional network with polarized self-attention for pixel-wise HSI classification. It contains three stages: channel-wise feature extraction network, spatial-wise feature extraction network, and classification network, which are used to extract spectral features, extract spatial features, and generate classification results, respectively. Pyramidal convolutional blocks and polarized attention blocks are combined to extract spectral and spatial features of HSI. Furthermore, residual aggregation and one-shot aggregation are employed to better converge the network. Experimental results on several public HSI datasets demonstrate that the proposed network outperforms other related methods.