Cascaded Convolution-Based Transformer With Densely Connected Mechanism for Spectral–Spatial Hyperspectral Image Classification

Abstract

Hyperspectral image (HSI) classification attempts to classify each pixel, which is an important means of obtaining land–cover knowledge. Hyperspectral images are cubic data with spectral–spatial knowledge and can generally be considered as sequential data alongside spectral dimension. Unlike convolutional neural networks (CNNs), which mainly focus on local relationship models in images, transformers have been shown to be a powerful structure for qualifying sequence data. However, it lacks the excellent ability of CNNs in establishing local relationships in images and cannot perform good generalization in case of insufficient data. In addition, the gradient disappearance problem hinders the convergence stability of deep learning networks as the layers get deeper. To address these problems, we propose a Cascaded Convolution-based Transformer with Densely Connected Mechanism (CDCformer) for hyperspectral image classification. First, we propose a cascaded convolution feature tokenization to extract spectral–spatial information, which will introduce some inductive bias properties of CNN into the transformer. In addition, we design a simple and effective densely connected transformer to enhance feature propagation and transfer memorable information from shallow to deep layers. It efficiently improves the performance of the transformer and extracts more discriminative spectral–spatial features from the HSI. Extensive experimental evaluation of three public hyperspectral data sets shows that CDCformer achieves competitive classification results.