Swin transformer with multiscale 3D atrous convolution for hyperspectral image classification

Abstract

Hyperspectral image (HSI) classification has attracted significant interest among researchers owing to its diverse practical applications. Convolutional neural networks (CNNs) have been extensively utilized for HSI classification. However, the effectiveness of CNN-based approaches is constrained by the fixed size and structure of the convolutional kernels, as well as their incapacity to capture global features. Moreover, these networks are inadequate in modeling the sequential characteristics of data. Recently, a promising approach, window-based multi-head self-attention has emerged to address the limitations of CNNs and incorporate efficient sequence modeling capabilities. This paper introduces a novel method, multiscale 3D atrous convolution with a lightweight swin transformer (MACLST), that effectively combines the strengths of two networks to capture both local and global features at different scales in HSI classification. The MACLST is designed to process HSI cubes as input and employs a spectral–spatial features extraction module based on multiscale 3D atrous convolution. This module involves parallel branches of 3D layers with varying atrous rates, enabling the extraction of features at multiple scales and resolutions. The extracted spectral–spatial features are fused and passed to the lightweight Swin transformer module as linear embeddings. This module captures long-range dependencies and learns effective feature representations of HSI. To reduce computational complexity, the swin transformer module is simplified and consists of only two stages, offering a more efficient version of the original swin transformer. The proposed MACLST model is extensively evaluated on five widely used benchmark HSI datasets, and the experimental results validate its superiority over state-of-the-art approaches with an overall accuracy of 99.00%, 99.59%, 99.95%, 98.71%, and 94.98% on the Indian Pines, University of Pavia, Salinas Valley, Houston University 2013, and Houston University 2018 datasets, respectively.