3D Change Detection Research Articles

3D surface change detection of slope using 3D point cloud based on construction of double surface

Aiming at the complex geological conditions of slopes and the difficulty of manual investigation, this paper proposes a method of constructing double surface based on 3D point cloud data to estimate the depth of slope deformation and disaster analysis. Terrestrial Laser Scanner is utilized to collect the slope point cloud data. Through the construction of double surface to simulate the original slope, the deformation depth is estimated and the depth data are fused into the point cloud to realize the automatic identification of slope deformation disaster and the calculation of deformation depth, and provide the deformation depth map. The shortcomings of the traditional Multiscale Model and Model Point Cloud Comparison (M3C2) algorithm, which needs to capture multiple time periods of point cloud data for calculation can be overcome. And there is no need to recognize the segmentation of the slope point cloud. The study results show that the maximum relative error of slope deformation is 10.82% in slope deformation. The proposed method is suitable for detecting slope hazards in various scenarios, providing a basis for further analysis and the development of effective erosion control measures. It introduces a new technology for daily monitoring of slope deformations and can be extended to drone-based LiDAR applications.

Revolutionizing urban mapping: deep learning and data fusion strategies for accurate building footprint segmentation

In the dynamic urban landscape, understanding the distribution of buildings is paramount. Extracting and delineating building footprints from high-resolution images, captured by aerial platforms or satellites, is essential but challenging to accomplish manually, due to the abundance of high-resolution data. Automation becomes imperative, yet it introduces complexities related to handling diverse data sources and the computational demands of advanced algorithms. The innovative solution proposed in this paper addresses some intricate challenges occurring when integrating deep learning and data fusion on Earth Observed imagery. By merging RGB orthophotos with Digital Surface Models, deriving from the same aerial high-resolution surveys, an integrated consistent four-band dataset is generated. This unified approach, focused on the extraction of height information through stereoscopy utilizing a singular source, facilitates enhanced pixel-to-pixel data fusion. Employing DeepLabv3 algorithms, a state-of-the-art semantic segmentation network for multi-scale context, pixel-based segmentation on the integrated dataset was performed, excelling in capturing intricate details, particularly when enhanced by the additional height information deriving from the Digital Surface Models acquired over urban landscapes. Evaluation over a 21 km2 area in Turin, Italy, featuring diverse building frameworks, showcases how the proposed approach leads towards superior accuracy levels and building boundary refinement. Notably, the methodology discussed in the present article, significantly reduces training time compared to conventional approaches like U-Net, overcoming inherent challenges in high-resolution data automation. By establishing the effectiveness of leveraging DeepLabv3 algorithms on an integrated dataset for precise building footprint segmentation, the present contribution holds promise for applications in 3D modelling, Change detection and urban planning. An approach favouring the application of deep learning strategies on integrated high-resolution datasets can then guide decision-making processes facilitating urban management tasks.

A protocol for annotation of total body photography for machine learning to analyze skin phenotype and lesion classification.

Artificial Intelligence (AI) has proven effective in classifying skin cancers using dermoscopy images. In experimental settings, algorithms have outperformed expert dermatologists in classifying melanoma and keratinocyte cancers. However, clinical application is limited when algorithms are presented with 'untrained' or out-of-distribution lesion categories, often misclassifying benign lesions as malignant, or misclassifying malignant lesions as benign. Another limitation often raised is the lack of clinical context (e.g., medical history) used as input for the AI decision process. The increasing use of Total Body Photography (TBP) in clinical examinations presents new opportunities for AI to perform holistic analysis of the whole patient, rather than a single lesion. Currently there is a lack of existing literature or standards for image annotation of TBP, or on preserving patient privacy during the machine learning process. This protocol describes the methods for the acquisition of patient data, including TBP, medical history, and genetic risk factors, to create a comprehensive dataset for machine learning. 500 patients of various risk profiles will be recruited from two clinical sites (Australia and Spain), to undergo temporal total body imaging, complete surveys on sun behaviors and medical history, and provide a DNA sample. This patient-level metadata is applied to image datasets using DICOM labels. Anonymization and masking methods are applied to preserve patient privacy. A two-step annotation process is followed to label skin images for lesion detection and classification using deep learning models. Skin phenotype characteristics are extracted from images, including innate and facultative skin color, nevi distribution, and UV damage. Several algorithms will be developed relating to skin lesion detection, segmentation and classification, 3D mapping, change detection, and risk profiling. Simultaneously, explainable AI (XAI) methods will be incorporated to foster clinician and patient trust. Additionally, a publicly released dataset of anonymized annotated TBP images will be released for an international challenge to advance the development of new algorithms using this type of data. The anticipated results from this protocol are validated AI-based tools to provide holistic risk assessment for individual lesions, and risk stratification of patients to assist clinicians in monitoring for skin cancer.

3D change detection for cultural heritage monitoring: Two case studies of underground sculptural reliefs

The preservation of underground cultural heritage is a challenging goal. Difficult acquisition conditions, the need for uniformity of temporal data and the speed at which surfaces change are some of the issues to consider. Also, quantifiable three-dimensional data are often unavailable when assessing changes on sculptural reliefs because traditional records are two-dimensional. In this paper, we present a workflow to easily detect and measure 3D superficial changes in sculptural reliefs using the deviation analysis technique. We compared multitemporal surveys, testing a quantitative, replicable, and verifiable procedure in two case studies: the reliefs of Pakal's Tomb at Palenque, Mexico, and those of the Osimo Caves in Italy. Both are located underground and show similar superficial alterations such as material disaggregation, efflorescence, salt crusts, and biological colonization. The reliefs showed visible and quantifiable changes on the deviation maps. The procedure to detect changes can be implemented with models generated by structured light scanning as well as photogrammetry.

3D Change Detection Method for Exterior Wall of LNG Storage Tank Supported by Multi‐Source Spatial Data

AbstractChange detection methods for measuring deformation of the LNG storage tank's exterior wall cannot meet the requirements for extensive and comprehensive deformation analysis. The method proposed in this paper uses multi‐source spatial data to detect the deformation of the LNG storage tank's exterior wall. Exterior wall data collected simultaneously by terrestrial and airborne laser scanners, as well as close‐range and oblique photogrammetry, are pre‐processed separately and then registered to the global point clouds. The deformation state of exterior wall is represented by a thorough assessment approach that takes into account four change indices. The results suggest that the registration precision of global point clouds is 6 mm. The global ovality of the LNG storage tank is 0.053%, with a tilt change of 2.58° and a deviation of 27 mm in the direction of the azimuth of 321.98°, an average value of 0.07 mm for depression and protrusion deformation, and a surface flatness of 21.3 mm. In this case, the flatness is greater than the variation threshold of the standard specification, indicating that the tank may have structural uneven settlement or other deformation problems. The attraction of the proposed method resides in its capacity to efficiently and accurately obtain spatial positioning and color information of the LNG storage tank's exterior wall, thereby providing a robust foundation for precise judgments regarding the deformation state.

Change Detection Needs Change Information: Improving Deep 3-D Point Cloud Change Detection

Change Detection Needs Change Information: Improving Deep 3-D Point Cloud Change Detection

RESEARCH ON SELF-CROSS-TRANSFORMER MODEL OF POINT CLOUD CHANGE DETECTION

Abstract. With the vigorous development of the urban construction industry, engineering deformation or changes often occur during the construction process. To combat this phenomenon, it is necessary to detect changes in order to detect construction loopholes in time, ensure the integrity of the project and reduce labor costs. Or the inconvenience and injuriousness of the road. In the study of change detection in 3D point clouds, researchers have published various research methods on 3D point clouds. Directly based on but mostly based on traditional threshold distance methods (C2C, M3C2, M3C2-EP), and some are to convert 3D point clouds into DSM, which loses a lot of original information. Although deep learning is used in remote sensing methods, in terms of change detection of 3D point clouds, it is more converted into two-dimensional patches, and neural networks are rarely applied directly. We prefer that the network is given at the level of pixels or points. Variety. Therefore, in this article, our network builds a network for 3D point cloud change detection, and proposes a new module Cross transformer suitable for change detection. Simultaneously simulate tunneling data for change detection, and do test experiments with our network.

DC3DCD: Unsupervised learning for multiclass 3D point cloud change detection

In a constant evolving world, change detection is of prime importance to keep updated maps. To better sense areas with complex geometry (urban areas in particular), considering 3D data appears to be an interesting alternative to classical 2D images. In this context, 3D point clouds (PCs), whether obtained through LiDAR or photogrammetric techniques, provide valuable information. While recent studies showed the considerable benefit of using deep learning-based methods to detect and characterize changes into raw 3D PCs, these studies rely on large annotated training data to obtain accurate results. The collection of these annotations are tricky and time-consuming. The availability of unsupervised or weakly supervised approaches is then of prime interest. In this paper, we propose an unsupervised method, called DeepCluster 3D Change Detection (DC3DCD), to detect and categorize multiclass changes at point level. We classify our approach in the unsupervised family given the fact that we extract in a completely unsupervised way a number of clusters associated with potential changes. Let us precise that in the end of the process, the user has only to assign a label to each of these clusters to derive the final change map. Our method builds upon the DeepCluster approach, originally designed for image classification, to handle complex raw 3D PCs and perform change segmentation task. An assessment of the method on both simulated and real public dataset is provided. The proposed method allows to outperform fully-supervised traditional machine learning algorithm and to be competitive with fully-supervised deep learning networks applied on rasterization of 3D PCs with a mean of IoU over classes of change of 57.06% and 66.69% for the simulated and the real datasets, respectively. The code is available at https://github.com/idegelis/torch-points3d-dc3dcd.

Fixed photogrammetric systems for natural hazard monitoring with high spatio-temporal resolution

Abstract. In this publication we address the lack of technical expertise in the geoscience community in the design and construction of photogrammetric systems for monitoring natural hazards at high spatio-temporal resolution. Accordingly, we provide in-depth information on the components, assembly instructions, and programming codes required to build them, making them accessible to researchers from different disciplines who are interested in 3D change detection monitoring. Each system comprises five photographic modules and a wireless transmission system for real-time image transfer. As an alternative to lidar (light detection and ranging), high-end digital cameras offer a simpler and more cost-effective solution for the generation of 3D models, especially in fixed time-lapse monitoring systems. The acquired images, in combination with algorithms that allow the creation of improved 3D models, offer change detection performance comparable to lidar. We showcase the usefulness of our approach by presenting real-world applications in the field of geohazard monitoring. Our findings highlight the potential of our method to detect pre-failure deformation and identify rockfalls with a theoretical change detection threshold of only 3–4 cm, thereby demonstrating the potential to achieve similar accuracies to lidar but at a much lower cost. Furthermore, thanks to the higher data acquisition frequency, the results show how the overlap of events that leads to an erroneous interpretation of the behaviour of the active area is minimized, allowing, for example, more accurate correlations between weather conditions and rockfall activity.

Rapid 3D lidar change detection for geohazard identification using GPU-based alignment and M3C2 algorithms

Topographic change detection is increasingly being used to identify and monitor landslides and other geohazards in support of risk-informed decision-making. Expanding change detection from site specific to regional scales enables increasingly proactive asset management and contributes to improving the resilience of infrastructure to extreme events. It is widely known that change detection precision can be improved by applying three-dimensional algorithms, such as iterative closest point (ICP) and M3C2, directly to raw point clouds. However, this also increases the computational requirements compared to alternatives such as digital elevation model differencing. This study presents a novel graphics processing unit (GPU)-based implementation of the ICP-M3C2 workflow to address this limitation. In the proposed algorithm, point cloud data segments are automatically queued and served to the working GPU, which efficiently performs point cloud processing operations, while the central processing unit (CPU) performs data management operations in parallel. The developed method is estimated to be up to 54 times faster than CPU-based versions of the same algorithm. In this study, we present how the workflow has been applied to six regional-scale landslide identification and monitoring case studies in which landslides are causing the disruption of pipelines, highways, and rail corridors. Overall, in 2021 and 2022, over 17 500 linear km of change detection were processed using the demonstrated method.

Deep unsupervised learning for 3D ALS point clouds change detection

Change detection from traditional 2D optical images has limited capability to model the changes in the height or shape of objects. Change detection using 3D point cloud from photogrammetry or LiDAR surveying can fill this gap by providing critical depth information. While most existing machine learning based 3D point cloud change detection methods are supervised, they severely depend on the availability of annotated training data, which is in practice a critical point. To circumnavigate this dependence, we propose an unsupervised 3D point cloud change detection method mainly based on self-supervised learning using deep clustering and contrastive learning. The proposed method also relies on an adaptation of deep change vector analysis to 3D point cloud via nearest point comparison. Experiments conducted on an aerial LiDAR survey dataset show that the proposed method obtains higher performance in comparison to the traditional unsupervised methods, with a gain of about 9% in mean accuracy (to reach more than 85%). Thus, it appears to be a relevant choice in scenario where prior knowledge (labels) is not ensured. The code will be made available at https://github.com/IdeGelis/torch-points3d-SSL-DCVA.

Two-Dimensional and 3D Change Detection in Urban Area Using Very High-Resolution Satellite Data and Impact of Urbanization over LST and NDVI

Two-Dimensional and 3D Change Detection in Urban Area Using Very High-Resolution Satellite Data and Impact of Urbanization over LST and NDVI

SHREC 2023: Point cloud change detection for city scenes

Localization and navigation are the two most important tasks for mobile robots, which require an up-to-date and accurate map. However, to detect map changes from crowdsourced data is a challenging task, especially from billions of points collected by 3D acquisition devices. Collecting 3D data often requires expensive data acquisition equipment and there are limited data sources to evaluate point cloud change detection. To address these issues, in this Shape Retrieval Challenge (SHREC) track, we provide a city-scene dataset with real and synthesized data to detect 3D point cloud change. The dataset consists of 866 pairs of object changes from 78 city-scene 3D point clouds collected by LiDAR and 845 pairs of object changes from 100 city-scene 3D point clouds generated by a high-fidelity simulator.We compare three methods on this benchmark. Evaluation results show that data-driven methods are the current trend in 3D point cloud change detection. Besides, the siamese network architecture is helpful to detect changes in our dataset. We hope this benchmark and comparative evaluation results will further enrich and boost the research of point cloud change detection and its applications.

Siamese KPConv: 3D multiple change detection from raw point clouds using deep learning

This study is concerned with urban change detection and categorization in point clouds. In such situations, objects are mainly characterized by their vertical axis, and the use of native 3D data such as 3D Point Clouds (PCs) is, in general, preferred to rasterized versions because of significant loss of information implied by any rasterization process. Yet, for obvious practical reasons, most existing studies only focus on 2D images for change detection purpose. In this paper, we propose a method capable of performing change detection directly within 3D data. Despite recent deep learning developments in remote sensing, to the best of our knowledge there is no such method to tackle multi-class change segmentation that directly processes raw 3D PCs. Thereby, based on advances in deep learning for change detection in 2D images and for analysis of 3D point clouds, we propose a deep Siamese KPConv network that deals with raw 3D PCs to perform change detection and categorization in a single step. Experimental results are conducted on synthetic and real data of various kinds (LiDAR, multi-sensors). Tests performed on simulated low density LiDAR and multi-sensor datasets show that our proposed method can obtain up to 80% of mean of IoU over classes of changes, leading to an improvement ranging from 10% to 30% over the state-of-the-art. A similar range of improvements is attainable on real data. Then, we show that pre-training Siamese KPConv on simulated PCs allows us to greatly reduce (more than 3,000×) the annotations required on real data. This is a highly significant result to deal with practical scenarios. Finally, an adaptation of Siamese KPConv is realized to deal with change classification at PC scale. Our network overtakes the current state-of-the-art deep learning method by 23% and 15% of mean of IoU when assessed on synthetic and public Change3D datasets, respectively. The code is available at the following link: https://github.com/IdeGelis/torch-points3d-SiameseKPConv.

Inferring 3D change detection from bitemporal optical images

In recent years, change detection (CD) using deep learning (DL) algorithms has been a very active research topic in the field of remote sensing (RS). Nevertheless, the CD algorithms developed so far are mainly focused on generating two-dimensional (2D) change maps where the planimetric extent of the areas affected by changes is identified without providing any information on the corresponding elevation variations. The aim of this work is, hence, to establish the basis for the development of DL algorithms able to automatically generate an elevation (3D) CD map along with a standard 2D CD map, using only bitemporal optical images as input, and thus without the need to rely directly on elevation data during the inference phase. Specifically, our work proposes a novel network, capable of solving the 2D and 3D CD tasks simultaneously, and a modified version of the 3DCD dataset, a freely available dataset designed precisely for this twofold task. The proposed architecture consists of a Transformer network based on a semantic tokenizer: the MultiTask Bitemporal Images Transformer (MTBIT). Encouraging results, obtained on the modified version of the 3DCD dataset by comparing the proposed architecture with other networks specifically designed to solve the 2D CD task, are shown. In particular, MTBIT achieves a metric accuracy (represented by the changed root mean squared error) equal to 6.46 m – the best performance among the compared architectures – with a limited number of parameters (13,1 M). The code and the 3DCD dataset are available at https://sites.google.com/uniroma1.it/3dchangedetection/home-page.

Three Dimensional Change Detection Using Point Clouds: A Review

Change detection is an important step for the characterization of object dynamics at the earth’s surface. In multi-temporal point clouds, the main challenge is to detect true changes at different granularities in a scene subject to significant noise and occlusion. To better understand new research perspectives in this field, a deep review of recent advances in 3D change detection methods is needed. To this end, we present a comprehensive review of the state of the art of 3D change detection approaches, mainly those using 3D point clouds. We review standard methods and recent advances in the use of machine and deep learning for change detection. In addition, the paper presents a summary of 3D point cloud benchmark datasets from different sensors (aerial, mobile, and static), together with associated information. We also investigate representative evaluation metrics for this task. To finish, we present open questions and research perspectives. By reviewing the relevant papers in the field, we highlight the potential of bi- and multi-temporal point clouds for better monitoring analysis for various applications.

An Automatic Individual Tree 3D Change Detection Method for Allometric Parameters Estimation in Mixed Uneven-Aged Forest Stands from ALS Data

Forests play a central role in the management of the Earth’s climate. Airborne laser scanning (ALS) technologies facilitate the monitoring of large and impassable areas and can be used to monitor the 3D structure of forests. While the ALS-based forest measures have been studied in depth, 3D change detection in forests is still a subject of little attention in the literature due to the challenges introduced by comparing point cloud pairs. In this study, we propose an innovative methodology to (i) automatically perform a 3D change detection of forests on an individual tree level; (ii) estimate tree parameters with allometric equations; and (iii) perform an assessment of the aboveground biomass (AGB) variation over time. The area in which the tests were carried out was hit by an ice storm that occurred in the time interval between the two LiDAR acquisitions; furthermore, field measurements were carried out and used to validate the results. The single-tree segmentation of the point clouds was automatically performed with a local maxima algorithm to detect the treetop, and a decision tree method to define the individual crowns around the local maxima. The multitemporal comparison of the point clouds was based on the identification of single trees, which were matched when there was a correlation between the position of the treetops. For each tree, the DBH (diameter at breast height) and the AGB were also estimated using allometric equations. The results are promising and allowed us to identify the uprooted trees and estimate that about 40% of the AGB of the area under examination had been destroyed, with an RMSE over the estimation ranging between 4% and 21% in four scenarios.

Object-Based Change Detection Algorithm with a Spatial AI Stereo Camera.

This paper presents a real-time object-based 3D change detection method that is built around the concept of semantic object maps. The algorithm is able to maintain an object-oriented metric-semantic map of the environment and can detect object-level changes between consecutive patrol routes. The proposed 3D change detection method exploits the capabilities of the novel ZED 2 stereo camera, which integrates stereo vision and artificial intelligence (AI) to enable the development of spatial AI applications. To design the change detection algorithm and set its parameters, an extensive evaluation of the ZED 2 camera was carried out with respect to depth accuracy and consistency, visual tracking and relocalization accuracy and object detection performance. The outcomes of these findings are reported in the paper. Moreover, the utility of the proposed object-based 3D change detection is shown in real-world indoor and outdoor experiments.

Point cloud registration and change detection in urban environment using an onboard Lidar sensor and MLS reference data

This paper presents a new method for urban scene analysis, which comprises 3D point cloud registration and change detection through fusing Lidar point clouds with significantly different density characteristics. The introduced method is able to extract dynamic scene segments (traffic participants or urban renewal regions) and seasonal changes (vegetation regions) from instant 3D (i3D) measurements captured by a Rotating Multi-beam (RMB) Lidar sensor mounted onto the top of a moving vehicle. As reference data, we rely on a dense point cloud-based environment model provided by Mobile Laser Scanning (MLS) systems. The proposed approach is composed of new solutions for two main subtasks. First, a novel multimodal point cloud registration algorithm is introduced, which can improve the alignment of the sparse i3D measurements to the dense MLS data, where conventional point level registration or keypoint/segment matching strategies fail. Second, an efficient Markov Random Field-based change extraction step is implemented between the registered point clouds, which exploits that due to geometric considerations of mapping with the given sensor configuration, the essence of the problem can be solved quickly in the 2D range image domain without information loss. Experimental evaluation is conducted on a new Benchmark set that contains three different heavy traffic road sections in city center areas covering in total nearly 1 km long pathway sections. Test data consists of relevant industrial measurements provided by a state-of-the-art RMB scanner (with a point density of around 50-500 points/m2) and an up-to-date MLS system (more than 5000 points/m2). The clear advantages of the new method are quantitatively demonstrated against various reference techniques. In comparison to six different point cloud registration methods, the median value of point level distances is decreased by 1–2 orders of magnitude by the proposed approach. Regarding change detection, the new method outperforms the reference models either in F1-scores (by around 10–25%) or in computational complexity (10–1000 times faster).

3DCD: A NEW DATASET FOR 2D AND 3D CHANGE DETECTION USING DEEP LEARNING TECHNIQUES

Abstract. Change detection is one of the main topics in Earth Observation, due to its wide range of applications, varying from urban development monitoring to natural disaster management. Most of the recently developed change detection methodologies rely on the use of deep learning algorithms. These kinds of algorithms are generally focused on generating two-dimensional (2D) change maps, thus they are only able to detect horizontal changes in land use/land cover, not considering nor returning any information on the corresponding elevation changes. Our work proposes a step forward, creating and sharing a dataset where two optical images acquired in different epochs are provided together with both the related 2D change maps containing land use/land cover variations and the three-dimensional (3D) maps containing elevation changes. Particularly, our aim is to provide a dataset useful to address and possibly solve the change detection task in 3D. Indeed, the proposed dataset, on the one hand, can empower a further development of 2D change detection algorithms, and, on the other hand, can allow to develop algorithms able to provide 3D change detection maps from two optical images captured in different epochs, without the need to rely directly on elevation data as input. The proposed dataset is publicly available at the following link: https://bit.ly/3wDdo41.