Learnable three-dimensional Gabor convolutional network with global affinity attention for hyperspectral image classification

Abstract

Benefiting from the development of hyperspectral imaging technology, hyperspectral image (HSI) classification has become a valuable direction in remote sensing image processing. Recently, researchers have found a connection between convolutional neural networks (CNNs) and Gabor filters. Therefore, some Gabor-based CNN methods have been proposed for HSI classification. However, most Gabor-based CNN methods still manually generate Gabor filters whose parameters are empirically set and remain unchanged during the CNN learning process. Moreover, these methods require patch cubes as network inputs. Such patch cubes may contain interference pixels, which will negatively affect the classification results. To address these problems, in this paper, we propose a learnable three-dimensional (3D) Gabor convolutional network with global affinity attention for HSI classification. More precisely, the learnable 3D Gabor convolution kernel is constructed by the 3D Gabor filter, which can be learned and updated during the training process. Furthermore, spatial and spectral global affinity attention modules are introduced to capture more discriminative features between spatial locations and spectral bands in the patch cube, thus alleviating the interfering pixels problem. Experimental results on three well-known HSI datasets (including two natural crop scenarios and one urban scenario) have demonstrated that the proposed network can achieve powerful classification performance and outperforms widely used machine-learning-based and deep-learning-based methods.