Learning Multiscale Temporal–Spatial–Spectral Features via a Multipath Convolutional LSTM Neural Network for Change Detection With Hyperspectral Images

Abstract

Change detection (CD) with hyperspectral images (HSIs) can be effectively performed using deep learning networks (DLNs) by taking advantage of HSIs for their abundant spectral and spatial information and the excellent performance of DLN in machine learning. By modeling the temporal dependence of multiscale representative features, more discriminative information reflecting land use and land cover (LULC) changes can be obtained by suppressing less correlated information while improving the robustness of pseudo-changes caused by imaging noises. However, preserving time-dependent multiscale representative features while extracting spatial–spectral features based on conventional DLN is difficult, mainly due to the structural limitation of conventional DLN. A multipath convolutional long short-term memory (LSTM) multipath convolutional long short-term memory neural network (MP-ConvLSTM) taking advantage of LSTM and convolutional neural network (CNN) through the designed parallel architecture to learn multilevel temporal dependencies of bitemporal HSIs, therefore, was proposed for extracting multiscale temporal–spatial–spectral features by combining hidden states from different paths of ConvLSTM in the present study. In the proposed MP-ConvLSTM, the efficient channel attention (ECA) module was introduced to refine features of different paths, and Siamese CNN was adopted to reduce HSIs’ dimensionality and extract preliminary features to build up an end-to-end trainable model for CD with HSIs. The validity of the MP-ConvLSTM was evaluated using the binary and multiclass CD datasets. The CD accuracy of the proposed MP-ConvLSTM was visually and statistically evaluated by different criteria and compared with those derived from several state-of-the-art (SOTA) CD algorithms. The experiments demonstrated that our proposed model not only outperformed those SOTA CD models but also exhibited better tradeoff between complexity and accuracy in general.