Semantic and spatial‐spectral feature fusion transformer network for the classification of hyperspectral image

Abstract

AbstractRecently, transformer‐based networks have been introduced for the classification of hyperspectral image (HSI). Although transformer‐based methods can well capture spectral sequence information, their ability to fuse different types of information contained in HSI is still insufficient. To exploit rich spectral, spatial and semantic information in HSI, a novel semantic and spatial‐spectral feature fusion transformer (S3FFT) network is proposed in this study. In the proposed S3FFT method, spatial attention and efficient channel attention (ECA) modules are employed for the extraction of shallow spatial‐spectral features. Then, a transformer‐based module is designed to extract advanced fused features and to produce the pseudo‐label and class probability of each pixel for semantic feature extraction. Finally, the semantic, spatial and spectral features are combined by the transformer for classification. Compared with traditional deep learning methods and recently transformer‐based methods, the proposed S3FFT shows relatively better results on three HSI datasets.