3D Change Detection Research Articles

Intelligent surveillance of indoor environments based on computer vision and 3D point cloud fusion

A real-time detection algorithm for intelligent surveillance is presented. The system, based on 3D change detection with respect to a complex scene model, allows intruder monitoring and detection of added and missing objects, under different illumination conditions.The proposed system has two independent stages. First, a mapping application provides an accurate 3D wide model of the scene, using a view registration approach. This registration is based on computer vision and 3D point cloud. Fusion of visual features with 3D descriptors is used in order to identify corresponding points in two consecutive views. The matching of these two views is first estimated by a pre-alignment stage, based on the tilt movement of the sensor, later they are accurately aligned by an Iterative Closest Point variant (Levenberg–Marquardt ICP), which performance has been improved by a previous filter based on geometrical assumptions.The second stage provides accurate intruder and object detection by means of a 3D change detection approach, based on Octree volumetric representation, followed by a clusters analysis. The whole scene is continuously scanned, and every captured is compared with the corresponding part of the wide model thanks to the previous analysis of the sensor movement parameters. With this purpose a tilt-axis calibration method has been developed.Tests performed show the reliable performance of the system under real conditions and the improvements provided by each stage independently. Moreover, the main goal of this application has been enhanced, for reliable intruder detection by the tilting of the sensors using its built-in motor to increase the size of the monitored area.

3D-information fusion from very high resolution satellite sensors

Abstract. In this paper we show the pre-processing and potential for environmental applications of very high resolution (VHR) satellite stereo imagery like these from WorldView-2 or Pl´eiades with ground sampling distances (GSD) of half a metre to a metre. To process such data first a dense digital surface model (DSM) has to be generated. Afterwards from this a digital terrain model (DTM) representing the ground and a so called normalized digital elevation model (nDEM) representing off-ground objects are derived. Combining these elevation based data with a spectral classification allows detection and extraction of objects from the satellite scenes. Beside the object extraction also the DSM and DTM can directly be used for simulation and monitoring of environmental issues. Examples are the simulation of floodings, building-volume and people estimation, simulation of noise from roads, wave-propagation for cellphones, wind and light for estimating renewable energy sources, 3D change detection, earthquake preparedness and crisis relief, urban development and sprawl of informal settlements and much more. Also outside of urban areas volume information brings literally a new dimension to earth oberservation tasks like the volume estimations of forests and illegal logging, volume of (illegal) open pit mining activities, estimation of flooding or tsunami risks, dike planning, etc. In this paper we present the preprocessing from the original level-1 satellite data to digital surface models (DSMs), corresponding VHR ortho images and derived digital terrain models (DTMs). From these components we present how a monitoring and decision fusion based 3D change detection can be realized by using different acquisitions. The results are analyzed and assessed to derive quality parameters for the presented method. Finally the usability of 3D information fusion from VHR satellite imagery is discussed and evaluated.

Landslide detection and monitoring capability of boat-based mobile laser scanning along Dieppe coastal cliffs, Normandy

Integrated in a wide research assessing destabilizing and triggering factors to model cliff dynamic along the Dieppe’s shoreline in High Normandy, this study aims at testing boat-based mobile LiDAR capabilities by scanning 3D point clouds of the unstable coastal cliffs. Two acquisition campaigns were performed in September 2012 and September 2013, scanning (1) a 30-km-long shoreline and (2) the same test cliffs in different environmental conditions and device settings. The potentials of collected data for 3D modelling, change detection and landslide monitoring were afterward assessed. By scanning during favourable meteorological and marine conditions and close to the coast, mobile LiDAR devices are able to quickly scan a long shoreline with median point spacing up to 10 cm. The acquired data are then sufficiently detailed to map geomorphological features smaller than 0.5 m2. Furthermore, our capability to detect rockfalls and erosion deposits (>m3) is confirmed, since using the classical approach of computing differences between sequential acquisitions reveals many cliff collapses between Pourville and Quiberville and only sparse changes between Dieppe and Belleville-sur-Mer. These different change rates result from different rockfall susceptibilities. Finally, we also confirmed the capability of the boat-based mobile LiDAR technique to monitor single large changes, characterizing the Dieppe landslide geometry with two main active scarps, retrogression up to 40 m and about 100,000 m3 of eroded materials.

A Novel 3D Building Damage Detection Method Using Multiple Overlapping UAV Images

Abstract. In this paper, a novel approach is presented that applies multiple overlapping UAV imagesto building damage detection. Traditional building damage detection method focus on 2D changes detection (i.e., those only in image appearance), whereas the 2D information delivered by the images is often not sufficient and accurate when dealing with building damage detection. Therefore the detection of building damage in 3D feature of scenes is desired. The key idea of 3D building damage detection is the 3D Change Detection using 3D point cloud obtained from aerial images through Structure from motion (SFM) techniques. The approach of building damage detection discussed in this paper not only uses the height changes of 3D feature of scene but also utilizes the image's shape and texture feature. Therefore, this method fully combines the 2D and 3D information of the real world to detect the building damage. The results, tested through field study, demonstrate that this method is feasible and effective in building damage detection. It has also shown that the proposed method is easily applicable and suited well for rapid damage assessment after natural disasters.

3D change detection at street level using mobile laser scanning point clouds and terrestrial images

Automatic change detection and geo-database updating in the urban environment are difficult tasks. There has been much research on detecting changes with satellite and aerial images, but studies have rarely been performed at the street level, which is complex in its 3D geometry. Contemporary geo-databases include 3D street-level objects, which demand frequent data updating. Terrestrial images provides rich texture information for change detection, but the change detection with terrestrial images from different epochs sometimes faces problems with illumination changes, perspective distortions and unreliable 3D geometry caused by the lack of performance of automatic image matchers, while mobile laser scanning (MLS) data acquired from different epochs provides accurate 3D geometry for change detection, but is very expensive for periodical acquisition. This paper proposes a new method for change detection at street level by using combination of MLS point clouds and terrestrial images: the accurate but expensive MLS data acquired from an early epoch serves as the reference, and terrestrial images or photogrammetric images captured from an image-based mobile mapping system (MMS) at a later epoch are used to detect the geometrical changes between different epochs. The method will automatically mark the possible changes in each view, which provides a cost-efficient method for frequent data updating. The methodology is divided into several steps. In the first step, the point clouds are recorded by the MLS system and processed, with data cleaned and classified by semi-automatic means. In the second step, terrestrial images or mobile mapping images at a later epoch are taken and registered to the point cloud, and then point clouds are projected on each image by a weighted window based z-buffering method for view dependent 2D triangulation. In the next step, stereo pairs of the terrestrial images are rectified and re-projected between each other to check the geometrical consistency between point clouds and stereo images. Finally, an over-segmentation based graph cut optimization is carried out, taking into account the color, depth and class information to compute the changed area in the image space. The proposed method is invariant to light changes, robust to small co-registration errors between images and point clouds, and can be applied straightforwardly to 3D polyhedral models. This method can be used for 3D street data updating, city infrastructure management and damage monitoring in complex urban scenes.

Towards an Efficient City Inventory Management System for Urban Authorities in Developing Countries – The Case of 3D Change Detection

AbstractAs cities and urban areas grow and become more complex, the task of managing, updating and displaying three‐dimensional urban information can be challenging for city and urban authorities. The emergence of 3D city models has enabled the authorities to work with more reliable and competent data as 3D data (and maps) are proven to be more accurate and relatable in representing real world objects and scenarios. In South East Asia countries such as Malaysia and Singapore, urban development activities are dynamic causing city inventory information to change within short periods of time. The ability to detect and track the changes on building structures located in a city can avoid loss of revenue to the urban authorities as well as improving the efficiency of a city inventory in assisting the decision making process on management level. City inventory management system should be able to store and provide information especially related to the building structure that is needed by the local authority to manage a city efficiently. Current traditional methods utilized by most urban authorities in South East Asia region are inefficient and unable to cope with current complex scenarios in urban areas. This results in some valuable information for building management and information systems being discarded. This research presents a method to boost the efficiency of some aspect in city inventory for local authorities by introducing 3D change detection method on 3D city models. The proposed method is able to detect and analyse changes that occur on 3D buildings using the 3D segmentation approach. A hybrid semantic‐geometric 3D segmentation method that deals with segmentation of a 3D building based on its semantic value and surface characteristics, fitted by one of the predefined primitives is presented. The technique is designed to retain existing semantic and geometric information and ultimately enrich it by adding more specialized information. For future work, the proposed method will be implemented as part of the auto‐updating module that is able to detect any changes on 3D buildings such as storage and retrieval of semantic information of the changed structure, and automatically update the 3D city models for analysis.

The Scheme and the Preliminary Test of Object-Oriented Simultaneous 3D Geometric and Physical Change Detection Using GIS-guided Knowledge

Current methods of remotely sensed image change detection almost assume that the DEM of the surface objects do not change. However, for the geological disasters areas (such as: landslides, mudslides and avalanches, etc.), this assumption does not hold. And the traditional approach is being challenged. Thus, a new theory for change detection needs to be extended from two-dimensional (2D) to three-dimensional (3D) urgently. This paper aims to present an innovative scheme for change detection method, object-oriented simultaneous three-dimensional geometric and physical change detection (OOS3DGPCD) using GIS-guided knowledge. This aim will be reached by realizing the following specific objectives: a) to develop a set of automatic multi-feature matching and registration methods; b) to propose an approach for simultaneous detecting 3D geometric and physical attributes changes based on the object-oriented strategy; c) to develop a quality control method for OOS3DGPCD; d) to implement the newly proposed OOS3DGPCD method by designing algorithms and developing a prototype system. For aerial remotely sensed images of YingXiu, Wenchuan, preliminary experimental results of 3D change detection are shown so as to verify our approach.

Airborne photogrammetry and lidar for DSM extraction and 3D change detection over an urban area – a comparative study

A digital surface model (DSM) extracted from stereoscopic aerial images, acquired in March 2000, is compared with a DSM derived from airborne light detection and ranging (lidar) data collected in July 2009. Three densely built-up study areas in the city centre of Ghent, Belgium, are selected, each covering approximately 0.4 km2. The surface models, generated from the two different 3D acquisition methods, are compared qualitatively and quantitatively as to what extent they are suitable in modelling an urban environment, in particular for the 3D reconstruction of buildings. Then the data sets, which are acquired at two different epochs t 1 and t 2, are investigated as to what extent 3D (building) changes can be detected and modelled over the time interval. A difference model, generated by pixel-wise subtracting of both DSMs, indicates changes in elevation. Filters are proposed to differentiate ‘real’ building changes from false alarms provoked by model noise, outliers, vegetation, etc. A final 3D building change model maps all destructed and newly constructed buildings within the time interval t 2 – t 1. Based on the change model, the surface and volume of the building changes can be quantified.

Reducing the gap between Augmented Reality and 3D modeling with real-time depth imaging

Whereas 3D surface models are often used for augmented reality (e.g., for occlusion handling or model-based camera tracking), the creation and the use of such dense 3D models in augmented reality applications usually are two separated processes. The 3D surface models are often created in offline preparation steps, which makes it difficult to detect changes and to adapt the 3D model to these changes. This work presents a 3D change detection and model adjustment framework that combines AR techniques with real-time depth imaging to close the loop between dense 3D modeling and augmented reality. The proposed method detects the differences between a scene and a 3D model of the scene in real time. Then, the detected geometric differences are used to update the 3D model, thus bringing AR and 3D modeling closer together. The accuracy of the geometric difference detection depends on the depth measurement accuracy as well as on the accuracy of the intrinsic and extrinsic parameters. To evaluate the influence of these parameters, several experiments were conducted with simulated ground truth data. Furthermore, the evaluation shows the applicability of AR and depth image–based 3D modeling for model-based camera tracking.

Towards Automatic 3D Change Detection inside Urban Areas by Combining Height and Shape Information

Monitoring of urban areas using remote sensing data requires reliable change detection techniques. While most of the changes are optically visible and easily detectable by an expert user, automatic processes that remain valid even when different kinds of input data are considered, are quite difficult to develop. This paper provides new solutions for semi-automatic 3D change detection of buildings based on the joint use of height and spatial information. It is an attempt to build a reliable scheme for change detection able to process high as well as lower quality Digital Surface Models (DSMs). The subtraction of DSM, computed from stereo pairs acquired at different epochs, provide valuable information about 3D urban change. However, when at least one of the DSMs presents some artifacts, a simple DSM subtraction may result also in the detection of virtual changes. Several post-processing steps are proposed in this paper and adapted to different DSM qualities in order to quantify real changes. Shape features are introduced to describe the geometry of the detected changes and a Support Vector Machine (SVM) classifier is used to differentiate real from virtual changes. Evalua-tion of the proposed approach on object and pixel level in terms of completeness, correctness, overall accuracy, etc is performed, proving its efficiency and relatively high accuracy for different kind of stereo images and consequently different DSM qualities.

Towards Automatic 3D Change Detection inside Urban Areas by Combining Height and Shape Information

Towards Automatic 3D Change Detection inside Urban Areas by Combining Height and Shape Information

Post-earthquake assessment of building damage degree using LiDAR data and imagery

Various methods have been developed to detect and assess building’s damages due to earthquakes using remotely sensed data. After the launch of the high resolution sensors such as IKONOS and QuickBird, it becomes realistic to identify damages on the scale of individual building. However the low accuracy of the results has often led to the use of visual interpretation techniques. Moreover, it is very difficult to estimate the degree of building damage (e.g. slight damage, moderate damage, or severe damage) in detail using the existing methods. Therefore, a novel approach integrating LiDAR data and high resolution optical imagery is proposed for evaluating building damage degree quantitatively. The approach consists of two steps: 3D building model reconstruction and rooftop patch-oriented 3D change detection. Firstly, a method is proposed for automatically reconstructing 3D building models with precise geometric position and fine details, using pre-earthquake LiDAR data and high resolution imagery. Secondly, focusing on each rooftop patch of the 3D building models, the pre- and post-earthquake LiDAR points belonging to the patch are collected and compared to detect whether it was destroyed or not, and then the degree of building damage can be identified based on the ratio of the destroyed rooftop patches to all rooftop patches. The novelty of the proposed approach is to detect damages on the scale of building’s rooftop patch and realize quantitative estimation of building damage degree.

Automatic change detection of geo-spatial data from imagery

The problems and difficulty of current change detection techniques are presented. Then, according to whether image registration is done before change detection algorithms, the authors classify the change detection into two categories: the change detection after image registration and the change detection simultaneous with image registration. For the former, four topics including the change detection between new image and old image, the change detection between new image and old map, the change detection between new image/old image and old map, and the change detection between new multi-source images and old map/image are introduced. For the latter, three categories, i. e. the change detection between old DEM, DOM and new non-rectification image, the change detection between old DLG, DRG and new non-rectification image, and the 3D change detection between old 4D products and new multi-overlapped photos, are discussed.

ヘリコプター搭載レーザスキャナによるＧＩＳデータベース変化部抽出と３次元化

The necessity of maximizing urban space has been fueling studies on three-dimensional (3D) geographic information systems (GIS) . Yet 3D data acquisition still relies on conventional, labor-intensive and costly photogrammetric techniques, and the data often suffers from being out-dated. This study investigates the potential of an airborne scanning laser radar system for 3D data acquisition and change detection of a 3D GIS database. Although attitude control of the helicopter still needs significant improvement for practical applications, the system proved to have the potential capability of automatically acquiring 3D information and detecting changes of 3D urban features.