Change Detection Based on Deep Siamese Convolutional Network for Optical Aerial Images

A large number of publications have incorporated deep learning in the process of remote sensing change detection. In these Deep Learning Change Detection (DLCD) publications, deep learning methods have demonstrated their superiority over conventional change detection methods. However, the theoretical underpinnings of why deep learning improves the performance of change detection remain unresolved. As of today, few in-depth reviews have investigated the mechanisms of DLCD. Without such a review, five critical questions remain unclear. Does DLCD provide improved information representation for change detection? If so, how? How to select an appropriate DLCD method and why? How much does each type of change benefits from DLCD in terms of its performance? What are the major limitations of existing DLCD methods and what are the prospects for DLCD? To address these five questions, we reviewed according to the following strategies. We grouped the DLCD information assemblages into the four unique dimensions of remote sensing: spectral, spatial, temporal, and multi-sensor. For the extraction of information in each dimension, the difference between DLCD and conventional change detection methods was compared. We proposed a taxonomy of existing DLCD methods by dividing them into two distinctive pools: separate and coupled models. Their advantages, limitations, applicability, and performance were thoroughly investigated and explicitly presented. We examined the variations in performance between DLCD and conventional change detection. We depicted two limitations of DLCD, i.e. training sample and hardware and software dilemmas. Based on these analyses, we identified directions for future developments. As a result of our review, we found that DLCD’s advantages over conventional change detection can be attributed to three factors: improved information representation; improved change detection methods; and performance enhancements. DLCD has to surpass the limitations with regard to training samples and computing infrastructure. We envision this review can boost developments of deep learning in change detection applications.

Using medical images to evaluate disease severity and change over time is a routine and important task in clinical decision making. Grading systems are often used, but are unreliable as domain experts disagree on disease severity category thresholds. These discrete categories also do not reflect the underlying continuous spectrum of disease severity. To address these issues, we developed a convolutional Siamese neural network approach to evaluate disease severity at single time points and change between longitudinal patient visits on a continuous spectrum. We demonstrate this in two medical imaging domains: retinopathy of prematurity (ROP) in retinal photographs and osteoarthritis in knee radiographs. Our patient cohorts consist of 4861 images from 870 patients in the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) cohort study and 10,012 images from 3021 patients in the Multicenter Osteoarthritis Study (MOST), both of which feature longitudinal imaging data. Multiple expert clinician raters ranked 100 retinal images and 100 knee radiographs from excluded test sets for severity of ROP and osteoarthritis, respectively. The Siamese neural network output for each image in comparison to a pool of normal reference images correlates with disease severity rank (ρ = 0.87 for ROP and ρ = 0.89 for osteoarthritis), both within and between the clinical grading categories. Thus, this output can represent the continuous spectrum of disease severity at any single time point. The difference in these outputs can be used to show change over time. Alternatively, paired images from the same patient at two time points can be directly compared using the Siamese neural network, resulting in an additional continuous measure of change between images. Importantly, our approach does not require manual localization of the pathology of interest and requires only a binary label for training (same versus different). The location of disease and site of change detected by the algorithm can be visualized using an occlusion sensitivity map-based approach. For a longitudinal binary change detection task, our Siamese neural networks achieve test set receiving operator characteristic area under the curves (AUCs) of up to 0.90 in evaluating ROP or knee osteoarthritis change, depending on the change detection strategy. The overall performance on this binary task is similar compared to a conventional convolutional deep-neural network trained for multi-class classification. Our results demonstrate that convolutional Siamese neural networks can be a powerful tool for evaluating the continuous spectrum of disease severity and change in medical imaging.

It is quite an important and challenging problem for change detection (CD) from heterogeneous remote sensing images. The images obtained from different sensors (i.e., synthetic aperture radar (SAR) & optical camera) characterize the distinct properties of objects. Thus, it is impossible to detect changes by direct comparison of heterogeneous images. In this article, a new unsupervised change detection (USCD) method is proposed based on image translation. The cycle-consistent adversarial networks (CycleGANs) are employed to learn the subimage to subimage mapping relation using the given pair (i.e., before and after the event) of heterogeneous images from which the changes will be detected. Then, we can translate one image (e.g., SAR) from its original feature space (e.g., SAR) to another space (e.g., optical). By doing this, the pair of images can be represented in a common feature space (e.g., optical). The pixels with close pattern values in the before-event image may have quite different values in the after-event image if the change happens on some ones. Thus, we can generate the difference map between the translated before-event image and the original after-event image. Then, the difference map is divided into changed and unchanged parts. However, these detection results are not very reliable. We will select some significantly changed and unchanged pixel pairs from the two parts with the clustering technique (i.e., <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula> -means). These selected pixel pairs are used to learn a binary classifier, and the other pixel pairs will be classified by this classifier to obtain the final CD results. Experimental results on different real datasets demonstrate the effectiveness of the proposed USCD method compared with several other related methods.

This paper presents a robust point cloud optimization algorithm based on spatial geometry relationship and Siamese Network. The proposed algorithm is designed to be improve the integrity and accuracy of stereo matching. In order to approach the prior corresponding pixels in image pairs, an epipolar line constraint is employed to fix the effective matching range in searching images. Then a Siamese Network is utilized to calculate the similarity of matching templates between reference image and searching images to approach the optimal matching pixels. At last the corresponding pixel pairs are used to compute the coordinate of object points by the Space Intersection method. Comparison studies and experimental results prove the high integrity and accuracy of the proposed algorithm in low-altitude remote sensing image point cloud optimization.

Change detection methods aim to identify significantly changed areas in co-registered bitemporal images taken of the same area. Since not only do bitemporal images usually have different environmental conditions ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.,</i> different weather conditions, noises, and seasonal changes) but also changes irrelevant to the purpose of change detection ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g.,</i> road changes when detecting building change), which should not be detected as changed areas, change detection methods often suffer from the problem of pseudo-change detection. To alleviate this problem, we propose an encoder-decoder-based Siamese network with a channel-wise co-attention module that considers the channel-wise correlations between a feature map in one image and all feature maps in the other image. By comparing the feature map in one image with the revised feature map in the other image considering the correlations, we are able to reduce the differences between the feature maps when pseudo-changes exist, thereby rendering the proposed method more robust to pseudo-changes. In addition, we apply a contrastive loss function that encourages the pairs of feature maps corresponding to unchanged regions to be similar, which can help improve the performance of change detection. We verified the performance of the proposed method through experiments using datasets such as the change detection dataset (CDD) and building change detection dataset (BCDD). In the experiment, the proposed method achieved significantly improved performance compared with existing methods in terms of recall, precision, f1-score, and overall accuracy.

Change detection based on deep learning is an important research direction in intelligent interpretation of remote sensing images. It has developed rapidly in recent years, but it is also a long-term challenge in remote sensing applications. This is mainly because the production of labeled data for training requires a lot of labor costs, and the currently available change detection labeled data is relatively small. While the complexity of high-resolution remote sensing imagery greatly increases the difficulty for deep learning models to learn robust and discriminative representations from scenes and objects, in this case, training deep learning models with a small amount of labeled data is still a huge challenge. To address this issue, this paper proposes a semi-supervised learning change detection method based on Generative Adversarial Networks (GAN). Compared with previous techniques, this paper combines a typical GAN framework with a Siamese network and applies it to change detection in remote sensing images. We introduce residual networks and atrous convolutions into Siamese networks, and employ a flow alignment module (FAM) to learn semantic flow between adjacent hierarchical feature maps. The connected discriminator formulates the training of the generator as a min-max optimization problem. Comprehensive quantitative and qualitative evaluations of multiple models show that our proposed method outperforms state-of-the-art change detection algorithms.

Change detection involves extracting the changed regions from images taken of the same place at different times. Potential applications are automatically updating of HD maps or identifying damages caused by natural disasters. However, conventional change detection methods merely detect changed regions without classifying them. In this paper, we propose a change detection method that can estimate the object class of a changed region. Our method extends a U-Net as a multi-task learning framework and estimates changed regions and semantic segmentation simultaneously. We propose using the pixel-wise classification probabilities of semantic segmentation for detecting changed regions rather than the conventional L2 norm-based difference of feature maps. In our experiments, we show that our method can improve change detection performance and estimate the classes of corresponding changed objects.

Change detection (CD) is a challenging task on high-resolution bitemporal remote sensing images. Many recent studies of CD have focused on designing fully convolutional Siamese network architectures. However, most of these methods initialize their encoders by random values or an ImageNet pretrained model, without any prior for the CD task, thus limiting the performance of the CD model. In this article, the novel supervised contrastive pretraining and fine-tuning CD (SCPFCD) framework, which is made up of two cascaded stages, is presented to train a CD network based on a pretrained encoder. In the first supervised contrastive pretraining stage, the encoder of the Siamese network is asked to solve a joint pretext task introduced by the proposed CDContrast pretraining method on labeled CD data. The proposed CDContrast pretraining method includes land contrastive learning (LCL), which is based on supervised contrastive learning, and proxy CD learning. The LCL focuses on learning the spatial relationships among the land cover from bitemporal images by solving a land contrast task, while the proxy CD learning performs a proxy CD task on the top of the upsampling projector to avoid local optima for the LCL and learn features for the CD. Then, in the second fine-tuning stage, the whole Siamese network initialized with the pretrained encoder is fine-tuned to perform the CD task in an end-to-end manner. The proposed SCPFCD framework was verified with three CD datasets of high-resolution remote sensing images. The extensive experimental results consistently show that the proposed framework can effectively improve the ability to extract change information for Siamese networks.

Although considerable success has been achieved in change detection on optical remote sensing images, accurate detection of specific changes is still challenging. Due to the diversity and complexity of the ground surface changes and the increasing demand for detecting changes that require high-level semantics, we have to resort to deep learning techniques to extract the intrinsic representations of changed areas. However, one key problem for developing deep learning metho for detecting specific change areas is the limitation of annotated data. In this paper, we collect a change detection dataset with 862 labeled image pairs, where the urban construction-related changes are labeled. Further, we propose a supervised change detection method based on a deep siamese semantic segmentation network to handle the proposed data effectively. The novelty of the method is that the proposed siamese network treats the change detection problem as a binary semantic segmentation task and learns to extract features from the image pairs directly. The siamese architecture as well as the elaborately designed semantic segmentation networks significantly improve the performance on change detection tasks. Experimental results demonstrate the promising performance of the proposed network compared to existing approaches.

Wireless capsule endoscopy video summarization (WCE-VS) is highly demanded for eliminating redundant frames with high similarity. Conventional WCE-VS methods extract various hand-crafted features as image representations. Researches show that such features only reflect the low-level characteristics of single frame and essentially are not effective to capture the semantic similarity between WCE frames. Motivated by the salient property of Siamese neural network (SNN) in mapping similar image pairs closer while mapping dissimilar image pairs apart in the feature space, a novel learning-based WCE-VS method is proposed in this paper. Specifically, with the availability of labelled similar and dissimilar pairs of WCE frames, SNN is trained with a contrastive loss function to extract high level semantic features. Furthermore, for similarity judgment, to avoid the challenge of manually setting optimal threshold in conventional methods, we creatively cast it into a supervised classification problem implemented by a linear SVM. Extensive experiments validate the effectiveness and efficiency of our proposed method.

Surface soil moisture (SSM) is a significant component in the modeling of climatological, hydrological and ecological systems. In this study, an approach is presented and validated for combining the water cloud and Change Detection models to invert surface soil moisture using ENVISAT ASAR Global Monitor mode images (ASAR-GM data). A total of 25 images were selected in HH polarization. We have used parameters of the water cloud model best suited for eliminating the influence of vegetation cover on ASAR-GM and MOD15A2 LAI data in the Jiangsu Province. In our algorithm, Leaf Area Index (LAI) was used to substitute Vegetation Water Content (VWC). Relative soil moisture was calculated using the Change Detection method which eliminates the influence of surface roughness. Due to the relatively high rainfall and efficient irrigation system in this area as observed from in-situ measurement data, the lowest relative soil moisture (degree of duration) is about 30%. Therefore, the formula for the Change Detection method should apply a gain of 0.7, and an offset of 0.3, and thereafter converting the results to volumetric soil moisture (m v ) in accordance with local soil parameters. Finally, the retrieved volumetric soil moisture was validated with in-situ data, determination coefficient (R2) is 0.348 and a root mean square error (RMSE) in 0.087 cm3/cm3 was recorded, based on the method involving the combination of water cloud and Change Detection models. Compared with the conventional Change Detection method, the R2 of 0.104 was increased and the RMSE of 0.014 cm3/cm3 was reduced, indicating the robustness of our proposed soil moisture retrieval algorithm.

In this paper, we present a novel convolutional neural network (CNN)-based model for change detection in synthetic aperture radar (SAR) images. Considering that change detection task takes image pairs as an input, we first explore multiple neural network architectures, which are specifically adapted to the change detection task. There are several ways in which patch pairs can be processed by the network and how information sharing can efficiently learn the semantic difference between the changed and unchanged pixels. For this reason, we then design a “Siamese samples” CNN, which treats patch pairs as indiscriminate samples to extract descriptors and then joins for their outputs. During training, the two patch features are extracted by the same network instead of separate sub-networks, while the joining neuron measures the distance between the two feature vectors. Due to “pseudo-labels” with high accuracy that is difficult to obtain, we modify a joint classifier based on the fuzzy c-means method into joint-similarity classifier as preclassification to obtain coarse “pseudo labels,” and discard sample selection. Thus, the preclassification labels with a low accuracy are used to fine-tune the network. Finally, a significantly improved change detection result can be obtained from the network. The proposed architecture provides a better trade-off in terms of speed and accuracy among its counterparts (Siamese, Pseudo-Siamese, and 2-Channel networks). The experiments on several real SAR data sets demonstrate the state-of-the-art performance of the proposed method compared with the advanced change detection methods.

The application of deep learning techniques may prove difficult when datasets are small. Recently, techniques such as one-shot learning, few-shot learning, and Siamese networks have been proposed to address this problem. In this paper, we propose the use a convolutional Siamese network (CSN) that learns a similarity metric that discriminates between plant species based on images of leaves. Once the CSN has learned the similarity function, its discriminatory power is generalized to classify not just new pictures of the species used during training but also entirely new species for which only a few images are available. This is achieved by exposing the network to pairs of similar and dissimilar observations and minimizing the Euclidean distance between similar pairs while simultaneously maximizing it between dissimilar pairs. We conducted experiments to study two different scenarios. In the first one, the CSN was trained and validated with datasets that comprise 5, 10, 15, 20, 25, and 30 pictures per species, extracted from the well-known Flavia dataset. Then, the trained model was tested with another dataset composed of 320 images (10 images per species) also from Flavia. The obtained accuracy was compared with the results of feeding the same training, validation, and testing datasets to a convolutional neural network (CNN) in order to determine if there is a threshold value t for dataset size that defines the intervals for which either the CSN or the CNN has better accuracy. In the second studied scenario, the accuracy of both the CSN and the CNN—both trained and validated with the same datasets extracted from Flavia—were compared when tested on a set of images of leaves of 20 Costa Rican tree species that are not represented in Flavia.

Cultivated land is a vital resource that underpins human survival and sustainable social development. With the widespread use of high-resolution remote sensing imagery, conventional change detection methods often suffer from limited accuracy due to pseudo-changes and insufficient feature representation when dealing with complex land structures and significant seasonal variations. To address the challenges of representing multi-scale structures, mitigating pseudo-change interference, and accurately delineating boundaries in cultivated land change detection, this study proposes a diffusion-guided change detection network—FarmChanger. The network is designed based on the principles of adaptive feature extraction and diffusion-inspired feature refinement. These components are further integrated through cross-feature guidance to enhance spatial details, forming an end-to-end detection framework. Comprehensive evaluations on the CLCD and Peixian benchmark datasets demonstrate that FarmChanger achieves comparable or superior performance to mainstream models across multiple evaluation metrics, verifying its high accuracy and robustness in cultivated land dynamic monitoring tasks.

Object-to-features vectorisation is a hard problem to solve for objects that can be hard to distinguish. Siamese and Triplet neural networks are one of the more recent tools used for such task. However, most networks used are very deep networks that prove to be hard to compute in the Internet of Things setting. In this paper, a computationally efficient neural network is proposed for real-time object-to-features vectorisation into a Euclidean metric space. We use L₂ distance to reflect feature vector similarity during both training and testing. In this way, feature vectors we develop can be easily classified using K-Nearest Neighbours classifier. Such approach can be used to train networks to vectorise such "problematic" objects like images of human faces, keypoint image patches, like keypoints on Arctic maps and surrounding marine areas.