Resolution reconstruction classification: fully octave convolution network with pyramid attention mechanism for hyperspectral image classification

Abstract

ABSTRACT We propose a fully octave convolution network with a pyramid attention mechanism (FOctConvPA) for whole Hyperspectral Image (HSI) classification. Because of the spatial relevance between pixels, effective extraction of global context and long-distance information can help improve the classification performance. However, global information flow is usually accompanied by the loss of local information and details. Therefore, in pursuit of solving this problem effectively, firstly, the proposed method based on the fully convolution network (FCN) uses the whole image as input to reserve original global information, and octave convolution is adopted to reduce spatial computing redundancy and highlight high-frequency edge details. Secondly, the information passes through the encoder and decoder in a resolution lossless form, so the loss of detail can be avoided as far as possible. Thirdly, we utilize the pyramid attention (PA) mechanism to achieve the purpose of global information flowing while long-distance dependencies are captured on both spatial and spectral dimensions. Finally, the mainstream information passing through the decoder is fused with initial information and symmetrical shallow information to improve classification accuracy and restore category boundary and details. Furthermore, to avoid the conflict between limited graphics processing unit (GPU) memory and the randomness of image size, a novel resolution reconstruction classification paradigm is proposed for our network. Experiments on four real benchmark hyperspectral datasets demonstrate that the proposed method could achieve better classification performance compared with other state-of-the-art deep learning models.