Change Detection In Hyperspectral Images Research Articles

Interactive Siamese spatial-Spectral cross-layer fusion transformer for hyperspectral image change detection

ABSTRACT Recently, methods based on Transformer have been widely used in the research field of hyperspectral image (HSI) change detection (CD). However, existing transformer-based CD research does not sufficiently utilize the spatial-spectral features of HSIs. In this article, we propose an interactive Siamese spatial-spectral cross-layer fusion Transformer (IS2CF-Former) network to improve the accuracy of HSI-CD. The proposed Siamese interactive module integrates the Siamese network with the Transformer structure, enhancing communication between bi-temporal images. We have made improvements to the cross-layer adaptive fusion (CAF) Transformer, where the cross-layer fusion module enhances the interaction between layers and the ability to capture local contextual features, concurrently reducing the model’s parameter count to mitigate the risk of overfitting. The CAF Transformer is applied to extract spatial and spectral features. Evaluating the detection performance of the proposed model on three bi-temporal HSIs through extensive experiments demonstrates superior accuracy compared to seven excellent CD methods.

Hyperspectral image change detection based on an improved multi-scale and spectral-wise transformer

ABSTRACT Change detection (CD) has a significant application in the remote sensing field. Because of the popularity of hyperspectral image (HSI) and the application of deep learning methods, hyperspectral image change detection (HSI-CD) techniques have been greatly developed. Among them, convolutional neural network (CNN) has garnered the greatest interest in HSI-CD due to their superior feature learning capabilities. However, current CNN-based algorithms have trouble capturing spectral similarity and long-range dependency owing to their intrinsic structural restrictions. Recently, transformers have been shown to extract global dependency from nature images in an extremely efficient way. But it has some difficulties in handling high-dimensional data, such as HSI. To address these issues, we propose an improved multi-scale and spectral-wise transformer (MS-SWT). The proposed MS-SWT is capable of capturing spectral similarity and long-range dependence between bands to enhance the efficiency of the HSI-CD task. Furthermore, to maximize the utilization of spatial information, we present a multi-scale feature fusion module (MFFM) to extract and fuse different dimensions of spatial features. More importantly, a locality self-attention (LSA) module is employed to alleviate the problem of smoothing the distribution of attention scores due to the large number of spectral embeddings. Moreover, we design a channel self-supervised loss function that can capture intrinsic information from the spectral channels to further strengthen the robustness of model training when the training samples are scarce. Lastly, comprehensive experiments present the high performance of our MS-SWT on four bitemporal HSI datasets and demonstrate the superiority of MS-SWT over state-of-the-art approaches.

Spectrum-Induced Transformer-Based Feature Learning for Multiple Change Detection in Hyperspectral Images

Spectrum-Induced Transformer-Based Feature Learning for Multiple Change Detection in Hyperspectral Images

Change Representation and Extraction in Stripes: Rethinking Unsupervised Hyperspectral Image Change Detection With an Untrained Network.

Deep learning-based hyperspectral image (HSI) change detection (CD) approaches have a strong ability to leverage spectral-spatial-temporal information through automatic feature extraction, and currently dominate in the research field. However, their efficiency and universality are limited by the dependency on labeled data. Although the newly applied untrained networks can avoid the need for labeled data, their feature volatility from the simple difference space easily leads to inaccurate CD results. Inspired by the interesting finding that salient changes appear as bright "stripes" in a new feature space, we propose a novel unsupervised CD method that represents and models changes in stripes for HSIs (named as StripeCD), which integrates optimization modeling into an untrained network. The StripeCD method constructs a new feature space that represents change features in stripes and models them in a novel optimization manner. It consists of three main parts: 1) dual-branch untrained convolutional network, which is utilized to extract deep difference features from bitemporal HSIs and combined with a two-stage channel selection strategy to emphasize the important channels that contribute to CD. 2) multiscale forward-backward segmentation framework, which is proposed for salient change representation. It transforms deep difference features into a new feature space by exploiting the structure information of ground objects and associates salient changes with the stripe-shaped change component. 3) stripe-shaped change extraction model, which characterizes the global sparsity and local discontinuity of salient changes. It explores the intrinsic properties of deep difference features and constructs model-based constraints to better identify changed regions in a controllable manner. The proposed StripeCD method outperformed the state-of-the-art unsupervised CD approaches on three widely used datasets. In addition, the proposed StripeCD method indicates the potential for further investigation of untrained networks in facilitating reliable CD.

Hyperspectral Image Change Detection Based on Gated Spectral–Spatial–Temporal Attention Network With Spectral Similarity Filtering

Hyperspectral Image Change Detection Based on Gated Spectral–Spatial–Temporal Attention Network With Spectral Similarity Filtering

Survey of hyperspectral image change detection method

Survey of hyperspectral image change detection method

Hybrid attention and bidirectional-gated network for hyperspectral image change detection

Hybrid attention and bidirectional-gated network for hyperspectral image change detection

Hypergraph Self-Supervised Learning based Joint Spectral-Spatial-Temporal Feature Representation for Hyperspectral Image Change Detection

Hypergraph Self-Supervised Learning based Joint Spectral-Spatial-Temporal Feature Representation for Hyperspectral Image Change Detection

End-to-End Hyperspectral Image Change Detection Based on Band Selection

End-to-End Hyperspectral Image Change Detection Based on Band Selection

Uncertainty-Aware Graph Self-Supervised Learning for Hyperspectral Image Change Detection

Uncertainty-Aware Graph Self-Supervised Learning for Hyperspectral Image Change Detection

A dual-branch siamese spatial-spectral transformer attention network for Hyperspectral Image Change Detection

The convolutional neural networks have recently gained widespread attention in Hyperspectral Image Change Detection (HSI-CD) due to their outstanding feature extraction ability. However, limited by the inherent network backbones, the convolutional neural networks (CNNs) fail to mine the sequence attributes and model the intricate relationships of spectral signatures. In contrast, transformers are proficient at learning sequence information owing to the powerful self-attention mechanisms. The two backbone structures exhibit complementary spatial and spectral feature extraction strengths, respectively. Inspired by this, we propose a dual-branch siamese spatial–spectral transformer attention network (DBS3TAN) for HSI-CD. The main idea is to fully exploit the advantages of CNNs and transformers for spatial and spectral feature extraction. More importantly, we devise the two key modules, i.e., the spatial attention module and the spatial–spectral transformer module. The former utilizes depthwise separable convolutions and attention mechanisms to emphasize the features of dual-temporal HSIs from the spatial perspective. The latter focuses on the sequence attributes of spectral signatures and mines the spatial characteristics from adjacent pixels. We employ the weighted contrastive loss function to separate the changed and unchanged pixels more reliably and set the random weight factors to balance the contributions of the two branches. Finally, the threshold values judgment is used to obtain the ultimate detection maps. We conduct extensive experiments to evaluate the DBS3TAN on three HSI datasets, demonstrating its superior performances than compared methods qualitatively and quantitatively. The source code will be available at https://github.com/zhangyiyan001/DBS3TAN.

Temporal difference-guided network for hyperspectral image change detection

ABSTRACT Recently, the research area of hyperspectral (HS) image change detection (CD) is popular with convolutional neural networks (CNNs) based methods. However, conventional CNNs-based CD algorithms commonly achieve detection by comparing the deep features extracted from the bi-temporal images at decision level, which often fails to take full advantage of the features extracted by the network at different levels. Moreover, there are inevitably substantial redundant features located in non-varying regions in bi-temporal images, which considerably impedes the training efficiency of CNNs-based methods. To solve these two problems, we propose a temporal difference-guided HS image CD network, called TDGN Specifically, the rich spectral features will be extracted from the bi-temporal images hierarchically, and then the differences between the two images at different levels of the network will be yielded by the elaborated convolutional gated recurrent unit block in the spatial dimension. Furthermore, the differences from these different levels will be fused for the final detection. More significantly, to boost the efficiency of the backbone network for feature extraction, the obtained difference at each level is also leveraged to generate variation weights to guide the feature extraction at the next stage. Finally, the proposed TDGN can make full use of the temporal difference obtained by the network at different levels while this information is further employed to facilitate the attention and extraction of change features by the network. Extensive experiments, implemented on four well-known HS data sets, demonstrate that the proposed TDGN yields an average overall accuracy of 98.67%, 96.74%, 99.36%, and 96.81% on these data sets, respectively, acquiring promising detection performance compared to state-of-the-art methods. The codes of this work will be available at https://github.com/zhonghaochen/TDGN_Master for the sake of reproducibility.

Spectral–Temporal Transformer for Hyperspectral Image Change Detection

Deep-Learning-based (DL-based) approaches have achieved remarkable performance in hyperspectral image (HSI) change detection (CD). Convolutional Neural Networks (CNNs) are often employed to capture fine spatial features, but they do not effectively exploit the spectral sequence information. Furthermore, existing Siamese-based networks ignore the interaction of change information during feature extraction. To address this issue, we propose a novel architecture, the Spectral–Temporal Transformer (STT), which processes the HSI CD task from a completely sequential perspective. The STT concatenates feature embeddings in spectral order, establishing a global spectrum–time-receptive field that can learn different representative features between two bands regardless of spectral or temporal distance, thereby strengthening the learning of temporal change information. Via the multi-head self-attention mechanism, the STT is capable of capturing spectral–temporal features that are weighted and enriched with discriminative sequence information, such as inter-spectral correlations, variations, and time dependency. We conducted experiments on three HSI datasets, demonstrating the competitive performance of our proposed method. Specifically, the overall accuracy of the STT outperforms the second-best method by 0.08%, 0.68%, and 0.99% on the Farmland, Hermiston, and River datasets, respectively.

Adaptive convolution kernel network for change detection in hyperspectral images.

Feature extraction is a key step in hyperspectral image change detection. However, many targets with great various sizes, such as narrow paths, wide rivers, and large tracts of cultivated land, can appear in a satellite remote sensing image at the same time, which will increase the difficulty of feature extraction. In addition, the phenomenon that the number of changed pixels is much less than unchanged pixels will lead to class imbalance and affect the accuracy of change detection. To address the above issues, based on the U-Net model, we propose an adaptive convolution kernel structure to replace the original convolution operations and design a weight loss function in the training stage. The adaptive convolution kernel contains two various kernel sizes and can automatically generate their corresponding weight feature map during training. Each output pixel obtains the corresponding convolution kernel combination according to the weight. This structure of automatically selecting the size of the convolution kernel can effectively adapt to different sizes of targets and extract multi-scale spatial features. The modified cross-entropy loss function solves the problem of class imbalance by increasing the weight of changed pixels. Study results on four datasets indicate that the proposed method performs better than most existing methods.

Local Information-Enhanced Graph-Transformer for Hyperspectral Image Change Detection With Limited Training Samples

Hyperspectral image (HSI) change detection is a challenging task that focuses on identifying the differences between multi-temporal HSIs. The recent advancement of convolutional neural network (CNN) has made great progress on HSIs change detection. However, due to the limited receptive field, most CNN based change detection models trained with sufficient labeled samples cannot flexibly model the global information that is essential for distinguishing complex objects, thereby achieving relatively-low performance. In this paper, we propose a dual-branch local information enhanced graph-transformer change detection network to fully exploit the local-global spectral-spatial features of the multi-temporal HSIs with limited training samples for change recognition. Specifically, the proposed network is composed of a cascaded of local information enhanced graph-transformer (LIEG) blocks, which jointly extracts local-global features by learning local information representation to enhance the information of graph-transformer. A novel graph-transformer is developed to model global spectral–spatial correlation between graph nodes, enabling the spectral information preservation of HSIs and accurate change detection of areas with various sizes. Extensive experiments have proved that our method achieves significant performance improvement than other state-of-the-art methods on four commonly used HSI datasets.

From Trained to Untrained: A Novel Change Detection Framework Using Randomly Initialized Models With Spatial–Channel Augmentation for Hyperspectral Images

Deep learning approaches have been extensively applied to change detection in hyperspectral images (HSIs). However, the majority of them encounter scarcity of training samples or rely on complex structures and learning strategies. Although untrained change detection models have been proved to be effective in relief above problems, they were constructed using regular convolutions and treated spatial locations and channels equally, which are insufficient to extract discriminative features and lead to limited accuracy. Given this, a novel untrained framework using randomly initialized models with spatial-channel augmentation (RICD) is proposed for HSI change detection in this paper. It consists of two major modules: 1) an enhanced feature extraction network using successive dilation-deformable feature extraction blocks, which can extract multiscale spatial-spectral features over unfixed sampling locations. It enlarges the field of view of convolutions and takes arbitrary neighborhood into consideration, which helps to increase the discriminativeness of the extracted features; 2) a change sensitive feature augmentation and comparison module integrating feature selection and spatial-channel augmentation strategies, which can exploit spatial context and channel importance. It magnifies difference between changed pixels and unchanged ones and emphasizes contribution of significant channels of the selected change sensitive features. Despite that convolution operations are included in RICD, all the weights are untrained and fixed once they are randomly initialized, indicating that the RICD can work in an unsupervised manner. Its performance is tested over three widely used hyperspectral datasets. Quantitative and qualitative comparisons with several state-of-the-art unsupervised methods reveal the effectiveness of the RICD method.

Intrinsic Decomposition Model-Guided Two-Stream Coupled Autoencoder for Unsupervised Hyperspectral Image Change Detection

Hyperspectral image change detection (HSI-CD) is one of the main research topics in remote sensing. Theoretically, the ground objects can be considered to have changed when their spectral features behave differently. However, in practical scenarios, this presupposition does not hold as the collected features are affected by many factors, such as illumination conditions, atmospheric effects, and topographic changes. Inspired by the intrinsic image decomposition, we propose a novel unsupervised deep learning framework based on two-stream coupled autoencoder (TSCA) to cope with bi-temporal co-registered HSI-CD. The network consists of two symmetric encoders and a decoder, which can jointly decompose bi-temporal images into abundance coefficients corresponding to the same set of spectral bases. As our network separates the component of spectral variation from multiple images, the extracted abundance features with inherent properties of materials can provide better performance for change detection. Moreover, to enforce alignment of the feature space, a reasonable consistency loss is devised to constrain the solution space, by cross-reconstruction in both branches. Experimental results demonstrate its superiority over the recently developed state of the arts.

Multiscale Diff-Changed Feature Fusion Network for Hyperspectral Image Change Detection

For hyperspectral image (HSI) change detection (CD), multiscale features are usually used to construct the detection models. However, the existing studies only consider the multiscale features containing changed and unchanged components, which is difficult to represent the subtle changes between bitemporal HSIs in each scale. To address this problem, we propose a multiscale diff-changed feature fusion network (MSDFFN) for HSI CD, which improves the ability of feature representation by learning the refined change components between bitemporal HSIs under different scales. In this network, a temporal feature encoder–decoder subnetwork, which combines a reduced inception (RI) module and a cross-layer attention module to highlight the significant features, is designed to extract the temporal features of HSIs. A bidirectional diff-changed feature representation (BDFR) module is proposed to learn the fine changed features of bitemporal HSIs at various scales to enhance the discriminative performance of the subtle change. A multiscale attention fusion (MSAF) module is developed to adaptively fuse the changed features of various scales. The proposed method can not only discover the subtle change in bitemporal HSIs but also improve the discriminating power for HSI CD. Experimental results on three HSI datasets show that MSDFFN outperforms a few state-of-the-art methods.

Depthwise Separable Convolutional Autoencoders for Hyperspectral Image Change Detection

Hyperspectral image change detection (HSI-CD) has recently become a research hotspot. Current methods rely heavily on a huge amount of training samples to perform the change detection tasks. While acquiring data from the same region of bi-temporal HSIs is extraordinarily time-consuming and laborious. Therefore, this letter proposes an unsupervised method based on three dimensional (3D) depthwise separable convolutional autoencoders (DSConvAE). First, the dual-branch symmetrical 3D DSConvAE is pre-trained with limited samples to obtain the optimal weights, which facilitates extracting discriminative spatial and spectral features subsequently. Second, we adopt the temporal-specific feature concatenation strategy to acquire comprehensive characteristics from bi-temporal HSIs. Third, the general autoencoders are employed at the end of the model to further explore the high-level and abstract feature vectors. Finally, we compare the mean square loss calculated from the spatial-spectral branches and apply threshold judgement to generate the ultimate detection maps. Experimental results on three public HSI datasets demonstrate that the proposed framework outperforms other comparative methods by significant improvements.

TriTF: A Triplet Transformer Framework Based on Parents and Brother Attention for Hyperspectral Image Change Detection

Hyperspectral image (HSI) change detection (CD) is a technique to accurately detect land cover changes by using HSIs with rich spatial-spectral information. In recent years, the HSI-CD methods based on convolutional neural networks (CNNs) have achieved great success because of their flexible and effective feature extraction ability. However, these methods often take the HSI patches as the input of the networks, which undoubtedly hinders the overall perception of the HSIs. Meanwhile, the valuable temporal information in HSIs is often underutilized. For this end, a triplet transformer framework (TriTF) based on parents-temporal attention and brother-spatial attention is proposed for HSI-CD. The proposed framework mainly contains the following three parts: 1) Transformer-based network backbone, which uses the self-attention to capture the correlation between arbitrarily two pixels in the same patch and extracts the global spatial correlation in the unit of encoded input patches; 2) parents-temporal attention (PTA) branch. Unlike the previous cross-temporal attention mechanisms of the “T1↔T2” mode which only consider the interaction between bi-temporal HSIs, this paper constructs a novel PTA of the “T1→T3←T2” mode which takes the difference-temporal image T3 as the core. The impact of bi-temporal HSIs on the land cover changes is more concerned in the PTA; 3) brother-spatial attention (BSA) branch. The most similar patch in the current training batch of each patch is defined as its brother patch. Furthermore, cross-spatial attention is applied to propagate the features of the brother patch to the current patch. Thus, the middle- and long-range dependencies can be utilized and the scope of feature propagation can be extended. In this paper, the experiments under low and high sampling rates are conducted and proved the outstanding change detection performance of the proposed TriTF when compared with abundant state-of-the-art (SOTA) CD algorithms. The source code of this paper will be released at https://github.com/zkylnnu/TriTF.