Pyramidal and conditional convolution attention network for hyperspectral image classification using limited training samples

Abstract

ABSTRACT Hyperspectral image classification methods based on convolutional neural network (CNN) and attention mechanism have obtained satisfactory results. However, the currently applied convolution operations treat all inputs equally, and the multiscale information extraction of hyperspectral data is not sufficient. The used attention mechanism is also one-dimensional, two-dimensional, or a simple combination of them. Alongside these, the high-precision classification models rely on many training samples. Based on these, we propose a pyramidal and conditional convolution attention network (PCAN), which can fully and effectively capture the information of hyperspectral input data through convolution kernels with different weights and different scales. And for the first time, the attention of three-dimensional weights is applied to hyperspectral data, and satisfactory results are achieved on minimal training sets. Specifically, PCAN is composed of spectral branch and spatial branch. The spectral branch uses conditionally parameterized convolutions (CondConv) to customize the convolution kernels according to the input features. It directly generates three-dimensional weights to quantify the importance of the entire feature maps. The spatial branch uses the pyramidal convolution (PyConv), which includes convolution kernels of different spatial sizes and depths and can capture different levels of detailed information in hyperspectral images. At the same time, competition and cooperation between different channels are captured through gating adaptation and channel normalization. Experiments on three well-known hyperspectral data sets show that our model achieves higher accuracy and less training time than other advanced methods, especially with fewer training samples.