Unchanged Regions Research Articles

Iterative Optimization-Enhanced Contrastive Learning for Multimodal Change Detection

Multimodal change detection (MCD) harnesses multi-source remote sensing data to identify surface changes, thereby presenting prospects for applications within disaster management and environmental surveillance. Nonetheless, disparities in imaging mechanisms across various modalities impede the direct comparison of multimodal images. In response, numerous methodologies employing deep learning features have emerged to derive comparable features from such images. Nevertheless, several of these approaches depend on manually labeled samples, which are resource-intensive, and their accuracy in distinguishing changed and unchanged regions is not satisfactory. In addressing these challenges, a new MCD method based on iterative optimization-enhanced contrastive learning is proposed in this paper. With the participation of positive and negative samples in contrastive learning, the deep feature extraction network focuses on extracting the initial deep features of multimodal images. The common projection layer unifies the deep features of two images into the same feature space. Then, the iterative optimization module expands the differences between changed and unchanged areas, enhancing the quality of the deep features. The final change map is derived from the similarity measurements of these optimized features. Experiments conducted across four real-world multimodal datasets, benchmarked against eight well-established methodologies, incontrovertibly illustrate the superiority of our proposed approach.

Two-stage fusion strategy and residual shrinkage capsule network for SAR image change detection

ABSTRACT Acquiring precise change map for synthetic aperture radar (SAR) images poses challenges due to the absence of labelled datasets and the presence of speckle noise interference. This paper presents a two-stage fusion strategy coupled with capsule network classification detection method. Initially, to enhance the structure of change areas within the difference image (DI), we propose an energy ratio (ER) algorithm based on the mean ratio (MR). Subsequently, employing a two-stage fusion strategy integrating multiplication and discrete wavelet transform (DWT), we fuse the DIs generated by the three algorithms to effectively complement change information across distinct DIs. In addition, a saliency detection algorithm is introduced to mitigate noise interference and balance the number of samples. Hierarchical fuzzy c-means clustering (HFCM) is then used to pre-classify the salient fusion difference image (SFDI) and derive pseudo-labels. Finally, we apply the residual shrinkage capsule network (RSCN) to reclassify pre-classification results and obtain the change detection outcome, capable of extracting the multi-scale refinement features and adaptively learning discriminant information from changed and unchanged regions. Experimental evaluations on various SAR image datasets demonstrate the exceptional detection performance of the proposed method.

Accelerating Text-to-Image Editing via Cache-Enabled Sparse Diffusion Inference

Due to the recent success of diffusion models, text-to-image generation is becoming increasingly popular and achieves a wide range of applications. Among them, text-to-image editing, or continuous text-to-image generation, attracts lots of attention and can potentially improve the quality of generated images. It's common to see that users may want to slightly edit the generated image by making minor modifications to their input textual descriptions for several rounds of diffusion inference. However, such an image editing process suffers from the low inference efficiency of many existing diffusion models even using GPU accelerators. To solve this problem, we introduce Fast Image Semantically Edit (FISEdit), a cached-enabled sparse diffusion model inference engine for efficient text-to-image editing. The key intuition behind our approach is to utilize the semantic mapping between the minor modifications on the input text and the affected regions on the output image. For each text editing step, FISEdit can 1) automatically identify the affected image regions and 2) utilize the cached unchanged regions' feature map to accelerate the inference process. For the former, we measure the differences between cached and ad hoc feature maps given the modified textual description, extract the region with significant differences, and capture the affected region by masks. For the latter, we develop an efficient sparse diffusion inference engine that only computes the feature maps for the affected region while reusing the cached statistics for the rest of the image. Finally, extensive empirical results show that FISEdit can be 3.4 times and 4.4 times faster than existing methods on NVIDIA TITAN RTX and A100 GPUs respectively, and even generates more satisfactory images.

Edge-Bound Change Detection in Multisource Remote Sensing Images

Detecting changes in multisource heterogeneous images is a great challenge for unsupervised change detection methods. Image-translation-based methods, which transform two images to be homogeneous for comparison, have become a mainstream approach. However, most of them primarily rely on information from unchanged regions, resulting in networks that cannot fully capture the connection between two heterogeneous representations. Moreover, the lack of a priori information and sufficient training data makes the training vulnerable to the interference of changed pixels. In this paper, we propose an edge-oriented generative adversarial network (EO-GAN) for change detection that indirectly translates images using edge information, which serves as a core and stable link between heterogeneous representations. The EO-GAN is composed of an edge extraction network and a reconstructive network. During the training process, we ensure that the edges extracted from heterogeneous images are as similar as possible through supplemented data based on superpixel segmentation. Experimental results on both heterogeneous and homogeneous datasets demonstrate the effectiveness of our proposed method.

SAAN: Similarity-Aware Attention Flow Network for Change Detection With VHR Remote Sensing Images.

Change detection (CD) is a fundamental and important task for monitoring the land surface dynamics in the earth observation field. Existing deep learning-based CD methods typically extract bi-temporal image features using a weight-sharing Siamese encoder network and identify change regions using a decoder network. These CD methods, however, still perform far from satisfactorily as we observe that 1) deep encoder layers focus on irrelevant background regions; and 2) the models' confidence in the change regions is inconsistent at different decoder stages. The first problem is because deep encoder layers cannot effectively learn from imbalanced change categories using the sole output supervision, while the second problem is attributed to the lack of explicit semantic consistency preservation. To address these issues, we design a novel similarity-aware attention flow network (SAAN). SAAN incorporates a similarity-guided attention flow module with deeply supervised similarity optimization to achieve effective change detection. Specifically, we counter the first issue by explicitly guiding deep encoder layers to discover semantic relations from bi-temporal input images using deeply supervised similarity optimization. The extracted features are optimized to be semantically similar in the unchanged regions and dissimilar in the changing regions. The second drawback can be alleviated by the proposed similarity-guided attention flow module, which incorporates similarity-guided attention modules and attention flow mechanisms to guide the model to focus on discriminative channels and regions. We evaluated the effectiveness and generalization ability of the proposed method by conducting experiments on a wide range of CD tasks. The experimental results demonstrate that our method achieves excellent performance on several CD tasks, with discriminative features and semantic consistency preserved.

DCAT: Dual Cross-Attention-Based Transformer for Change Detection

Several transformer-based methods for change detection (CD) in remote sensing images have been proposed, with Siamese-based methods showing promising results due to their two-stream feature extraction structure. However, these methods ignore the potential of the cross-attention mechanism to improve change feature discrimination and thus, may limit the final performance. Additionally, using either high-frequency-like fast change or low-frequency-like slow change alone may not effectively represent complex bi-temporal features. Given these limitations, we have developed a new approach that utilizes the dual cross-attention-transformer (DCAT) method. This method mimics the visual change observation procedure of human beings and interacts with and merges bi-temporal features. Unlike traditional Siamese-based CD frameworks, the proposed method extracts multi-scale features and models patch-wise change relationships by connecting a series of hierarchically structured dual cross-attention blocks (DCAB). DCAB is based on a hybrid dual branch mixer that combines convolution and transformer to extract and fuse local and global features. It calculates two types of cross-attention features to effectively learn comprehensive cues with both low- and high-frequency information input from paired CD images. This helps enhance discrimination between the changed and unchanged regions during feature extraction. The feature pyramid fusion network is more lightweight than the encoder and produces powerful multi-scale change representations by aggregating features from different layers. Experiments on four CD datasets demonstrate the advantages of DCAT architecture over other state-of-the-art methods.

Learning Color Distributions from Bitemporal Remote Sensing Images to Update Existing Building Footprints

For most cities, municipal governments have constructed basic building footprint datasets that need to be updated regularly for the management and monitoring of urban development and ecology. Cities are capable of changing in a short period of time, and the area of change is variable; hence, automated methods for generating up-to-date building footprints are urgently needed. However, the labels of current buildings or changed areas are usually lacking, and the conditions for acquiring images from different periods are not perfectly consistent, which can severely limit deep learning methods when attempting to learn deep information about buildings. In addition, common update methods can ignore the strictly accurate historical labels of unchanged areas. To solve the above problem, we propose a new update algorithm to update the existing building database to the current state without manual relabeling. First, the difference between the data distributions of different time-phase images is reduced using the image color translation method. Then, a semantic segmentation model predicts the segmentation results of the images from the latest period, and, finally, a post-processing update strategy is applied to strictly retain the existing labels of unchanged regions to attain the updated results. We apply the proposed algorithm on the Wuhan University change detection dataset and the Beijing Huairou district land survey dataset to evaluate the effectiveness of the method in building surface and complex labeling scenarios in urban and suburban areas. The F1 scores of the updated results obtained for both datasets reach more than 96%, which proves the applicability of our proposed algorithm and its ability to efficiently and accurately extract building footprints in real-world scenarios.

Remote Sensing Image-Change Detection with Pre-Generation of Depthwise-Separable Change-Salient Maps

Remote sensing change detection (CD) identifies changes in each pixel of certain classes of interest from a set of aligned image pairs. It is challenging to accurately identify natural changes in feature categories due to unstructured and temporal changes. This research proposed an effective bi-temporal remote sensing CD comprising an encoder that could extract multiscale features, a decoder that focused on semantic alignment between temporal features, and a classification head. In the decoder, we constructed a new convolutional attention structure based on pre-generation of depthwise-separable change-salient maps (PDACN) that could reduce the attention of the network on unchanged regions and thus reduce the potential pseudo-variation in the data sources caused by semantic differences in illumination and subtle alignment differences. To demonstrate the effectiveness of the PDA attention structure, we designed a lightweight network structure for encoders under both convolution-based and transformer architectures. The experiments were conducted on a single-building CD dataset (LEVIR-CD) and a more complex multivariate change type dataset (SYSU-CD). The results showed that our PDA attention structure generated more discriminative change variance information while the entire network model obtained the best performance results with the same level of network model parameters in the transformer architecture. For LEVIR-CD, we achieved an intersection over union (IoU) of 0.8492 and an F1 score of 0.9185. For SYSU-CD, we obtained an IoU of 0.7028 and an F1 score of 0.8255. The experimental results showed that the method proposed in this paper was superior to some current state-of-the-art CD methods.

Automatic binary and ternary change detection in SAR images based on evolutionary multiobjective optimization

In most of the previous works, changed and unchanged regions are detected by analyzing the changes of backscattering coefficients for SAR images, which is termed as binary change detection. In fact, due to the increase and decrease of backscattering coefficients, the changed regions can be further analyzed as two kinds of changes, which is termed as ternary change detection. In this paper, a change detection method based on evolutionary multiobjective optimization is proposed to automatically perform binary and ternary change detection of multitemporal SAR images. First, the log-likelihood function of the Gaussian mixture model and the Bhattacharyya distance are designed as two objectives, respectively. In particular, a novel measurement method based on Bhattacharyya distance is designed for the ternary change detection task. Not only the separability between each two classes is maximized, but also the Bhattacharyya distance between two changed classes and unchanged class is kept closer to obtain a more balanced classification performance. Then a multiobjective optimization method based on non-dominated sorting is used to optimize these two objectives simultaneously. In the proposed approach, chromosome ranking and perturbation probability selection operators are designed to make high-quality solutions with a high probability of being exploited and improve the performance of the algorithm. In addition, a one-step local search strategy based on the expectation–maximization method is integrated into the proposed algorithm to accelerate the convergence. Experimental results on simulated and real-world datasets demonstrate the effectiveness and robustness of the proposed algorithm.

Edge-Guided Recurrent Convolutional Neural Network for Multitemporal Remote Sensing Image Building Change Detection

Building change detection is a very important application in the field of remote sensing. Recently, deep learning (DL) has been introduced to solve the change detection task and achieved good performance, mainly due to the capability of automatically learning deep features. However, the lack of using prior knowledge (e.g., edge structure information) leads to inaccurate detection results, especially in the areas of building boundaries. To solve this problem, an end-to-end DL method for building change detection, named by edge-guided recurrent convolutional neural network (EGRCNN), is proposed in this article. The main idea is to incorporate both discriminative information and edge structure prior in one framework to improve change detection results, especially to generate more accurate building boundaries. First, a siamese convolutional neural network is trained to simultaneously extract primary multilevel features from multitemporal images. Then, a difference analysis module (DAM) is introduced to further produce discriminative features, which is constructed based on the basic long short-term memory module. Finally, both the discriminative features and the estimated edge structure information are jointly exploited to predict building change map. On one hand, the proposed DAM helps to enhance the discrimination between the changed and unchanged regions. On the other hand, the prior edge information is used to push the predicted changed buildings to preserve the original structure, which can further improve the accuracy of building change detection. Experimental results demonstrate that the performance of the proposed method outperforms several state-of-the-art approaches, in terms of objective metrics and visual comparison results.

Change Detection in Multitemporal SAR Images Based on Slow Feature Analysis Combined With Improving Image Fusion Strategy

Change detection in multitemporal synthetic aperture radar (SAR) images has been an important research content in the field of remote sensing for a long time. In this article, based on the slow feature analysis (SFA) theory and the nonsubsampled contourlet transform (NSCT) algorithm, we propose a novel unsupervised change detection method called NSCT nonlocal means (NSCT-NLM). The powerful extraction to the changed information of SFA and the superior information fusion of NSCT are jointly adopted in this method. The main framework consists of the following parts. First, SFA and the log-ratio operator are used to generate difference images (DIs) independently. Then, the NSCT is used to fuse two DIs into a new higher quality DI. The newly fused DI combines the complementary information of the two kinds of original DI. Therefore, the contrast of the changed regions and unchanged regions is greatly enhanced, as well as the changed details are preserved more completely. Furthermore, an NLM filtering algorithm is employed to suppress the strong speckles in the fused DI. We use the fuzzy C-means algorithm to generate the final binary change map. The experiments are carried out on two public datasets and three real-world SAR datasets from different scenarios. The results demonstrate that the proposed method has higher detection accuracy by comparing with the reference methods.

Study on the Landcover Changes Based on GIS and RS Technologies: A Case Study of the Sanjiangyuan National Nature Reserve in the Hinterland Tibet Plateau, China

With global warming and increased human activity, more and more ground surface areas have been affected, which leads to the creation of many problems. To understand the ecological environment changes in a typical alpine region, information on landcovers changes in the study area from 2000 to 2020 was obtained by combining remote sensing, geographic information system, and traditional statistical analysis for the Sanjiangyuan National Nature Reserve on the Qinghai-Tibet Plateau, China. The results show that the changes of surface cover in Sanjiangyuan National Nature Reserve are relatively concentrated, mainly distributed in the sections of south, northwest, central, and northeast, with the area of the changed regions and the unchanged regions being 31,127.45 km2 and 271,941.37 km2, respectively. The transformation between the various types of ground landcover is mainly characterized by a shift from other types to grassland and from grassland to bare land. And the regions of change are mostly concentrated between 3500 - 4600 m in elevation.

Adaptive Contourlet Fusion Clustering for SAR Image Change Detection.

In this paper, a novel unsupervised change detection method called adaptive Contourlet fusion clustering based on adaptive Contourlet fusion and fast non-local clustering is proposed for multi-temporal synthetic aperture radar (SAR) images. A binary image indicating changed regions is generated by a novel fuzzy clustering algorithm from a Contourlet fused difference image. Contourlet fusion uses complementary information from different types of difference images. For unchanged regions, the details should be restrained while highlighted for changed regions. Different fusion rules are designed for low frequency band and high frequency directional bands of Contourlet coefficients. Then a fast non-local clustering algorithm (FNLC) is proposed to classify the fused image to generate changed and unchanged regions. In order to reduce the impact of noise while preserve details of changed regions, not only local but also non-local information are incorporated into the FNLC in a fuzzy way. Experiments on both small and large scale datasets demonstrate the state-of-the-art performance of the proposed method in real applications.

Feature-Guided Multitask Change Detection Network

Change detection is the discovery of changes in remote sensing images of the same region obtained at different times. Change detection algorithms based on deep neural networks have significant advantages over traditional algorithms on high-resolution images. State-of-the-art (SOTA) change detection methods require sufficient labeled data to achieve good results, but semantic change detection requires not only binary change masks but also “from-to” change information, so large quantities of change labels are difficult to obtain. Achieving better semantic change detection accuracy with a limited number of labels remains an open problem in the remote sensing field. In this paper, we propose a feature-guided multitask change detection network (MCDnet). Feature guidance is characterized by three steps: 1) a multitask learning network that uses Siamese encoders to learn segmentation and change detection features simultaneously to realize mutual guidance between tasks is designed, 2) a fine-grained feature fusion module to integrate and enhance change information under the guidance of symmetrical change features is constructed, and 3) a contrastive loss function based on the a priori knowledge that the features of the changed regions are different while those of the unchanged regions are the same is proposed. The experimental results show that MCDnet achieves SOTA results on three public change detection datasets, including WHU-CD (F1: 94.46\IoU: 89.50), LEVIR (F1: 92.11\IoU: 85.37) and SECOND (mIoU: 73.1\Sek: 22.8). In addition, it is surprising that MCDnet is comparable to the SOTA models while using only 20% of the full training data.

Robust Change Detection Using Channel-Wise co-Attention-Based Siamese Network With Contrastive Loss Function

Change detection methods aim to identify significantly changed areas in co-registered bitemporal images taken of the same area. Since not only do bitemporal images usually have different environmental conditions ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.,</i> different weather conditions, noises, and seasonal changes) but also changes irrelevant to the purpose of change detection ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g.,</i> road changes when detecting building change), which should not be detected as changed areas, change detection methods often suffer from the problem of pseudo-change detection. To alleviate this problem, we propose an encoder-decoder-based Siamese network with a channel-wise co-attention module that considers the channel-wise correlations between a feature map in one image and all feature maps in the other image. By comparing the feature map in one image with the revised feature map in the other image considering the correlations, we are able to reduce the differences between the feature maps when pseudo-changes exist, thereby rendering the proposed method more robust to pseudo-changes. In addition, we apply a contrastive loss function that encourages the pairs of feature maps corresponding to unchanged regions to be similar, which can help improve the performance of change detection. We verified the performance of the proposed method through experiments using datasets such as the change detection dataset (CDD) and building change detection dataset (BCDD). In the experiment, the proposed method achieved significantly improved performance compared with existing methods in terms of recall, precision, f1-score, and overall accuracy.

Radiometric Normalization for Cross-Sensor Optical Gaofen Images with Change Detection and Chi-Square Test

As the number of cross-sensor images increases continuously, the surface reflectance of these images is inconsistent at the same ground objects due to different revisit periods and swaths. The surface reflectance consistency between cross-sensor images determines the accuracy of change detection, classification, and land surface parameter inversion, which is the most widespread application. We proposed a relative radiometric normalization (RRN) method to improve the surface reflectance consistency based on the change detection and chi-square test. The main contribution was that a novel chi-square test automatically extracts the stably unchanged samples between the reference and subject images from the unchanged regions detected by the change-detection method. We used the cross-senor optical images of Gaofen-1 and Gaofen-2 to test this method and four metrics to quantitatively evaluate the RRN performance, including the Root Mean Square Error (RMSE), spectral angle cosine, structural similarity, and CIEDE2000 color difference. Four metrics demonstrate the effectiveness of our proposed RRN method, especially the reduced percentage of RMSE after normalization was more than 80%. Comparing the radiometric differences of five ground features, the surface reflectance curve of two Gaofen images showed more minor differences after normalization, and the RMSE was smaller than 50 with the reduced percentages of about 50–80%. Moreover, the unchanged feature regions are detected by the change-detection method from the bitemporal Sentinel-2 images, which can be used for RRN without detecting changes in subject images. In addition, extracting samples with the chi-square test can effectively improve the surface reflectance consistency.

Content-Invariant Dual Learning for Change Detection in Remote Sensing Images

With the introduction of artificial intelligence (AI), current deep learning-driven change detection methods generally regard changes as a type of specific land cover information and try to detect them simply by using existing semantic labeling models. In practice, the changes that occur on different land cover categories may appear completely different for remote sensing images, making it difficult to detect multiple categories of changes with an end-to-end model. In addition, training such networks requires a large amount of prelabeled references, which are labor-intensive and time-consuming. Motivated by this observation, we integrate dual learning algorithm and disentangled representation theory to develop a novel approach, named content-invariant dual learning (CiDL), for either supervised or unsupervised change detection in remote sensing images. In our framework, two opposite Y-shaped networks, each of which consists of two encoders and one decoder, are introduced to translate bitemporal images from their original domains to each others, where their intrinsic content features are retained, while their style features are consistent. By training our hybrid framework, even without references, this method learns a category-wise cross-domain translation to suppress the discrepancies in paired unchanged regions, and meanwhile highlight those in paired changed regions. The experimental results on two typical change detection data sets and the comparison with other state-of-the-art deep learning-driven methods verify the effectiveness and competitiveness of our proposed CiDL.

Commonality Autoencoder: Learning Common Features for Change Detection From Heterogeneous Images.

Change detection based on heterogeneous images, such as optical images and synthetic aperture radar images, is a challenging problem because of their huge appearance differences. To combat this problem, we propose an unsupervised change detection method that contains only a convolutional autoencoder (CAE) for feature extraction and the commonality autoencoder for commonalities exploration. The CAE can eliminate a large part of redundancies in two heterogeneous images and obtain more consistent feature representations. The proposed commonality autoencoder has the ability to discover common features of ground objects between two heterogeneous images by transforming one heterogeneous image representation into another. The unchanged regions with the same ground objects share much more common features than the changed regions. Therefore, the number of common features can indicate changed regions and unchanged regions, and then a difference map can be calculated. At last, the change detection result is generated by applying a segmentation algorithm to the difference map. In our method, the network parameters of the commonality autoencoder are learned by the relevance of unchanged regions instead of the labels. Our experimental results on five real data sets demonstrate the promising performance of the proposed framework compared with several existing approaches.

Multitemporal Hyperspectral Images Change Detection Based on Joint Unmixing and Information Coguidance Strategy

The richness of spectral information in multitemporal hyperspectral images (MHSIs) offers the possibility to effectively detect subtle changes and properties of grounds. However, severe spectral variabilities and inadequate spatial co-exploitation capabilities limit the performance of existing methods due to differences in acquisition times and conditions. Therefore, this article proposes a strategy of joint unmixing and multitemporal spatial information coguidance (JUC) to fully exploit the spatio-temporal-spectral features. First, a multitemporal joint unmixing method is used to achieve endmembers’ extraction and abundance estimation. Wherein the method adds spectral perturbed regularization when compared to the traditional unmixing strategy, making it robust to spectral variability. Second, we propose a multitemporal coguidance method that highlights the contrast between changed and unchanged regions and removes the noise by transferring the common structure information between the multitemporal first principal component map and the abundance difference maps. It will obtain an enhanced abundance difference maps and achieve effective combination of multitemporal spatial information. The final change result can be obtained by combining and thresholding these enhanced abundance difference maps. Experiments on some data sets demonstrate that the proposed algorithm is sufficiently valid and robust for multitemporal images, especially for data containing spectral variabilities and obvious structures.

An initialization friendly Gaussian mixture model based multi-objective clustering method for SAR images change detection

Speckle noise is a main obstacle for change detection tasks of synthetic aperture radar (SAR) images. However, change detection methods often focus on removing noise and ignore the importance of preserving details of SAR images, which results in a loss of classification accuracy. In order to alleviate the contradiction between removing noise and preserving details, a multi-objective change detection method based on a modified Gaussian mixture model (GMM) is proposed in our paper. In the framework of our multi-objective model, one objective is composed of an special GMM of being applied to distinguish changed and unchanged regions and preserve details. The another is a carefully crafted noise reduction operator for removing speckle noise, which is constructed as a penalty to refine the result of classification. The aforementioned conflicting objectives are optimized simultaneously, which is conducted by a common multi-objective evolutionary algorithm based on decomposition (MOEA/D). Then, a series of solutions are obtained, all of which represent a balance of detail preservation and noise reduction. Compared with three classical change detection methods, the proposed method is applied to four real datasets in experimental studies. Experimental results and theoretical analysis reveal its effectiveness.