Dense Image Matching Point Clouds Research Articles

EVALUATION OF UAV-BORNE PHOTOGRAMMETRY AND LASER SCANNING FOR 3D TOPOGRAPHIC CHANGE ANALYSIS AT AN ACTIVE ROCK GLACIER

Abstract. Point clouds derived from UAV-borne laser scanning and UAV-borne photogrammetry provide new opportunities for 3D topographic monitoring in geographic research. The airborne acquisition strategy overcomes common challenges of ground-based techniques, such as limited spatial coverage or heterogeneous measurement distribution, and allows flexible repeated acquisitions at high temporal and spatial resolution. While UAV-borne 3D sensing techniques are expected to thereby enhance geographic monitoring, their specific potential for methods and algorithms of 3D change analysis is yet to be investigated. In our study, we assess point clouds originating from UAV-borne photogrammetry using dense image matching (DIM) and UAV-borne laser scanning (ULS) as input for 3D topographic change analysis at an active rock glacier in the Austrian Alps. We analyse surface change by using ULS and DIM point clouds of 2019 and 2021 as input for two state-of-the-art methods for pairwise surface change analysis: (1) The Multiscale Model to Model Cloud Comparison (M3C2) algorithm and (2) a recent M3C2-based approach (CD-PB M3C2) using plane correspondences to reduce the uncertainty of quantified change. We evaluate ULS-based and DIM-based change analysis regarding their performance in (1) achieving high spatial coverage of derived changes, (2) accurately quantifying magnitudes and uncertainty of change, and (3) detecting significant change (change magnitudes > associated uncertainty). As reference we use change quantified between two terrestrial laser scanning (TLS) surveys undertaken simultaneously with the ULS and DIM data acquisitions. Our study shows the improved spatial coverage of M3C2 achieved with point clouds acquired with UAVs (+ 60% of core points used for change analysis). For CD-PB M3C2, ULS and DIM point clouds enabled a spatially more uniform distribution of plane pairs used for change quantification and a slightly higher spatial coverage (+6% – +7% of core points used for change analysis) compared to the TLS reference. Magnitudes of M3C2 change were closer to the TLS reference for ULS-ULS (mean difference: 0.04 m; std. dev.: 0.05 m) compared to ULS-DIM (mean difference: 0.12 m; std. dev.: 0.08 m). Similar results were obtained for CD-PB M3C2 using ULS-ULS (mean difference: 0.02 m; std. dev.: 0.01 m) and ULS-DIM (mean difference: 0.06 m; std. dev.: 0.01 m). Moreover, magnitudes of change were above the associated uncertainty in 82% – 89% (M3C2) and 89% – 90% (CD-PB M3C2) of the area of change analysis. Our findings demonstrate the potential of ULS and DIM point clouds as input for accurate 3D topographic change analysis for the study at hand and can support the design and setup of 3D/4D Earth observation systems for rock glaciers and natural scenes with complex topography, such as landslides or debris covered glaciers.

Building Change Detection of Airborne Laser Scanning and Dense Image Matching Point Clouds using Height and Class Information

Abstract. Detecting changes is an important task to update databases and find irregularities in spatial data. Every couple of years, national mapping agencies (NMAs) acquire nation-wide point cloud data from Airborne Laser Scanning (ALS) as well as from Dense Image Matching (DIM) using aerial images. Besides deriving several other products such as Digital Elevation Models (DEMs) from them, those point clouds also offer the chance to detect changes between two points in time on a large scale. Buildings are an important object class in the context of change detection to update cadastre data. As detecting changes manually is very time consuming, the aim of this study is to provide reliable change detections for different building sizes in order to support NMAs in their task to update their databases. As datasets of different times may have varying point densities due to technological advancements or different sensors, we propose a raster-based approach, which is independent of the point density altogether. Within a raster cell, our approach considers the height distribution of all points for two points in time by exploiting the Jensen-Shannon distance to measure their similarity. Our proposed method outperforms simple threshold methods on detecting building changes with respect to the same or different point cloud types. In combination with our proposed class change detection approach, we achieve a change detection performance measured by the mean F1-Score of about 71% between two ALS and about 60% between ALS and DIM point clouds acquired at different times.

SEMANTIC SEGMENTATION OF BUILDING IN AIRBORNE IMAGES

Abstract. Building is a key component to the reconstructing of LoD3 city modelling. Compared to terrestrial view, airborne datasets have more occlusions at street level but can cover larger area in the urban areas. With the popularity of the Deep Learning, many tasks in the field of computer vision can be solved in easier and efficiency way. In this paper, we propose a method to apply deep neural networks to building façade segmentation. In particular, the FC-DenseNet and the DeepLabV3+ algorithms are used to segment the building from airborne images and get semantic information such as, wall, roof, balcony and opening area. The patch-wise segmentation is used in the training and testing process in order to get information at pixel level. Different typologies of input have been considered: beside the conventional 2D information (i.e. RGB image), we combined 2D information with 3D features extracted from dense image matching point clouds to improve the performance of the segmentation. Results show that FC-DenseNet trained with 2D and 3D features achieves the best result, IoU up to 64.41%, it increases 5.13% compared to the result of the same model trained without 3D features.

EXPLORING ALS AND DIM DATA FOR SEMANTIC SEGMENTATION USING CNNS

Abstract. Over the past years, the algorithms for dense image matching (DIM) to obtain point clouds from aerial images improved significantly. Consequently, DIM point clouds are now a good alternative to the established Airborne Laser Scanning (ALS) point clouds for remote sensing applications. In order to derive high-level applications such as digital terrain models or city models, each point within a point cloud must be assigned a class label. Usually, ALS and DIM are labelled with different classifiers due to their varying characteristics. In this work, we explore both point cloud types in a fully convolutional encoder-decoder network, which learns to classify ALS as well as DIM point clouds. As input, we project the point clouds onto a 2D image raster plane and calculate the minimal, average and maximal height values for each raster cell. The network then differentiates between the classes ground, non-ground, building and no data. We test our network in six training setups using only one point cloud type, both point clouds as well as several transfer-learning approaches. We quantitatively and qualitatively compare all results and discuss the advantages and disadvantages of all setups. The best network achieves an overall accuracy of 96 % in an ALS and 83 % in a DIM test set.

A CASE STUDY ON THROUGH-WATER DENSE IMAGE MATCHING

Abstract. In the last years, the tremendous progress in image processing and camera technology has reactivated the interest in photogrammetrybased surface mapping. With the advent of Dense Image Matching (DIM), the derivation of height values on a per-pixel basis became feasible, allowing the derivation of Digital Elevation Models (DEM) with a spatial resolution in the range of the ground sampling distance of the aerial images, which is often below 10 cm today. While mapping topography and vegetation constitutes the primary field of application for image based surface reconstruction, multi-spectral images also allow to see through the water surface to the bottom underneath provided sufficient water clarity. In this contribution, the feasibility of through-water dense image matching for mapping shallow water bathymetry using off-the-shelf software is evaluated. In a case study, the SURE software is applied to three different coastal and inland water bodies. After refraction correction, the DIM point clouds and the DEMs derived thereof are compared to concurrently acquired laser bathymetry data. The results confirm the general suitability of through-water dense image matching, but sufficient bottom texture and favorable environmental conditions (clear water, calm water surface) are a preconditions for achieving accurate results. Water depths of up to 5 m could be mapped with a mean deviation between laser and trough-water DIM in the dm-range. Image based water depth estimates, however, become unreliable in case of turbid or wavy water and poor bottom texture.

FILTERING PHOTOGRAMMETRIC POINT CLOUDS USING STANDARD LIDAR FILTERS TOWARDS DTM GENERATION

Abstract. Digital Terrain Models (DTMs) can be generated from point clouds acquired by laser scanning or photogrammetric dense matching. During the last two decades, much effort has been paid to developing robust filtering algorithms for the airborne laser scanning (ALS) data. With the point cloud quality from dense image matching (DIM) getting better and better, the research question that arises is whether those standard Lidar filters can be used to filter photogrammetric point clouds as well. Experiments are implemented to filter two dense matching point clouds with different noise levels. Results show that the standard Lidar filter is robust to random noise. However, artefacts and blunders in the DIM points often appear due to low contrast or poor texture in the images. Filtering will be erroneous in these locations. Filtering the DIM points pre-processed by a ranking filter will bring higher Type II error (i.e. non-ground points actually labelled as ground points) but much lower Type I error (i.e. bare ground points labelled as non-ground points). Finally, the potential DTM accuracy that can be achieved by DIM points is evaluated. Two DIM point clouds derived by Pix4Dmapper and SURE are compared. On grassland dense matching generates points higher than the true terrain surface, which will result in incorrectly elevated DTMs. The application of the ranking filter leads to a reduced bias in the DTM height, but a slightly increased noise level.

Deep convolutional neural networks for building extraction from orthoimages and dense image matching point clouds

Automatic extraction of buildings from remote sensing data is an attractive research topic, useful for several applications, such as cadastre and urban planning. This is mainly due to the inherent artifacts of the used data and the differences in viewpoint, surrounding environment, and complex shape and size of the buildings. This paper introduces an efficient deep learning framework based on convolutional neural networks (CNNs) toward building extraction from orthoimages. In contrast to conventional deep approaches in which the raw image data are fed as input to the deep neural network, in this paper the height information is exploited as an additional feature being derived from the application of a dense image matching algorithm. As test sites, several complex urban regions of various types of buildings, pixel resolutions and types of data are used, located in Vaihingen in Germany and in Perissa in Greece. Our method is evaluated using the rates of completeness, correctness, and quality and compared with conventional and other “shallow” learning paradigms such as support vector machines. Experimental results indicate that a combination of raw image data with height information, feeding as input to a deep CNN model, provides potentials in building detection in terms of robustness, flexibility, and efficiency.

AN AUTOMATIC FILTER ALGORITHM FOR DENSE IMAGE MATCHING POINT CLOUDS

Abstract. Although many filter algorithms have been presented over past decades, these algorithms are usually designed for the Lidar point clouds and can’t separate the ground points from the DIM (dense image matching, DIM) point clouds derived from the oblique aerial images owing to the high density and variation of the DIM point clouds completely. To solve this problem, a new automatic filter algorithm is developed on the basis of adaptive TIN models. At first, the differences between Lidar and DIM point clouds which influence the filtering results are analysed in this paper. To avoid the influences of the plants which can’t be penetrated by the DIM point clouds in the searching seed pointes process, the algorithm makes use of the facades of buildings to get ground points located on the roads as seed points and construct the initial TIN. Then a new densification strategy is applied to deal with the problem that the densification thresholds do not change as described in other methods in each iterative process. Finally, we use the DIM point clouds located in Potsdam produced by Photo-Scan to evaluate the method proposed in this paper. The experiment results show that the method proposed in this paper can not only separate the ground points from the DIM point clouds completely but also obtain the better filter results compared with TerraSolid. 1.

Photogrammetric DSM denoising

Abstract. Image matching techniques can nowadays provide very dense point clouds and they are often considered a valid alternative to LiDAR point cloud. However, photogrammetric point clouds are often characterized by a higher level of random noise compared to LiDAR data and by the presence of large outliers. These problems constitute a limitation in the practical use of photogrammetric data for many applications but an effective way to enhance the generated point cloud has still to be found. In this paper we concentrate on the restoration of Digital Surface Models (DSM), computed from dense image matching point clouds. A photogrammetric DSM, i.e. a 2.5D representation of the surface is still one of the major products derived from point clouds. Four different algorithms devoted to DSM denoising are presented: a standard median filter approach, a bilateral filter, a variational approach (TGV: Total Generalized Variation), as well as a newly developed algorithm, which is embedded into a Markov Random Field (MRF) framework and optimized through graph-cuts. The ability of each algorithm to recover the original DSM has been quantitatively evaluated. To do that, a synthetic DSM has been generated and different typologies of noise have been added to mimic the typical errors of photogrammetric DSMs. The evaluation reveals that standard filters like median and edge preserving smoothing through a bilateral filter approach cannot sufficiently remove typical errors occurring in a photogrammetric DSM. The TGV-based approach much better removes random noise, but large areas with outliers still remain. Our own method which explicitly models the degradation properties of those DSM outperforms the others in all aspects.