Hyperspectral image change detection based on an improved multi-scale and spectral-wise transformer

Abstract

ABSTRACT Change detection (CD) has a significant application in the remote sensing field. Because of the popularity of hyperspectral image (HSI) and the application of deep learning methods, hyperspectral image change detection (HSI-CD) techniques have been greatly developed. Among them, convolutional neural network (CNN) has garnered the greatest interest in HSI-CD due to their superior feature learning capabilities. However, current CNN-based algorithms have trouble capturing spectral similarity and long-range dependency owing to their intrinsic structural restrictions. Recently, transformers have been shown to extract global dependency from nature images in an extremely efficient way. But it has some difficulties in handling high-dimensional data, such as HSI. To address these issues, we propose an improved multi-scale and spectral-wise transformer (MS-SWT). The proposed MS-SWT is capable of capturing spectral similarity and long-range dependence between bands to enhance the efficiency of the HSI-CD task. Furthermore, to maximize the utilization of spatial information, we present a multi-scale feature fusion module (MFFM) to extract and fuse different dimensions of spatial features. More importantly, a locality self-attention (LSA) module is employed to alleviate the problem of smoothing the distribution of attention scores due to the large number of spectral embeddings. Moreover, we design a channel self-supervised loss function that can capture intrinsic information from the spectral channels to further strengthen the robustness of model training when the training samples are scarce. Lastly, comprehensive experiments present the high performance of our MS-SWT on four bitemporal HSI datasets and demonstrate the superiority of MS-SWT over state-of-the-art approaches.