Dense Point Cloud Generation Research Articles

Implementation of a Real Estate Management System using Photogrammetry Method Allowing 3d Navigation of Buildings

The objective of this study is to evaluate how the use of virtual representations of buildings, created through photogrammetry, can improve the efficiency and effectiveness of the real estate management system. In this regard, investigations have been conducted with stakeholders in the real estate sector, such as real estate agencies and tenants, to gather valuable information on real estate management. For illustration purposes, we have chosen to model the NASPW gymnasium. We have implemented a real estate management approach based on the photogrammetry method. To achieve our objective, we describe each step of the process, from the preparation of the photogrammetric mission (planning and simulating the mission) to data acquisition, carried out using a Phantom 4 Pro drone for aerial images and a Nikon D300 digital camera for interior shots. The data processing phase is also crucial, and it was done using the Agisoft Metashape software. We provide detailed explanations of the different steps in this process, which include image quality estimation, image alignment, georeferencing, dense point cloud generation, 3D mesh creation, and orthophoto generation.

Modified U-Net based 3D reconstruction model to estimate volume from multi-view images of a solid object

ABSTRACT This paper proposes a modified U-Net based reconstruction model to automatically estimate the volume of solid objects. Initially, the captured multi-view images are pre-processed using improved adaptive weighted mean filter (IAWMF). Then, the pre-processed image is segmented using modified U-Net based learning mechanism to determine ROI. After segmentation, the key feature points are extracted using Harris corner detection approach. Further, the extracted feature keypoints are matched using Oriented FAST and Rotated BRIEF (ORB) approach and the similarity among two images is calculated using similarity detection algorithm. Then, the multiple image views from various angles are used to calculate the depth map using VGGNet-16 to attain the appropriate volume of the object. Finally, the 3D point clouds are reconstructed from multiple images using dense point cloud generation algorithm for volumetric analysis. Finally, the result shows that the proposed model provides better accuracy with less error than other 3D CAD model.

Real-time dense point cloud generation and digital model construction of surface environment based on UAV platform

Real-time dense point cloud generation and digital model construction of surface environment based on UAV platform

A New Multi-Criteria Tie Point Filtering Approach to Increase the Accuracy of UAV Photogrammetry Models

Extracting accurate tie points plays an essential role in the accuracy of image orientation in Unmanned Aerial Vehicle (UAV) photogrammetry. In this study, a Multi-Criteria Decision Making (MCDM) automatic filtering method is presented. Based on the quality features of a photogrammetric model, the proposed method works at the level of sparse point cloud to remove low-quality tie points for refining the orientation results. In the proposed algorithm, different factors that affect the quality of tie points are identified. The quality measures are then aggregated by applying MCDM methods and a competency score for each 3D tie point. These scores are employed in an automatic filtering approach that selects a subset of high-quality points which are then used to repeat the bundle adjustment. To evaluate the proposed algorithm, various internal and external studies were conducted on different datasets. The findings suggest that our method is both effective and reliable. In addition, in comparison to the existing filtering techniques, the proposed strategy increases the accuracy of bundle adjustment and dense point cloud generation by about 40% and 70%, respectively.

The comparison of photogrammetric and terrestrial laser scanning methods in the documentation of small cultural heritage object – case study

Nowadays, 3D models are commonly used to protect, manage and preserve cultural heritage objects and sites. Two methods are widely used: close-range photogrammetry and terrestrial laser scanning. Both those methods are burdened by the errors caused by the measurement system and physical properties of the object being under development. This investigation aimed to present the potential and limitations of using the photogrammetric and terrestrial laser scanning methods to document small cultural heritage objects. The object is a water pumping station building located in the Museum of King John III's Palace at Wilanów. It is situated near Lake Wilanowskie; hence the access to one wall was considerably limited. Data were acquired by the Terrestrial Laser Scanner Z+F 5006h and full-frame camera Canon 5D Mark II with the focal length - 17, 24, 28, 35 mm. The close-range images were processed in the Agisoft Metashape program, and the Structure-from-Motion with MultiView Stereo approach was utilised for the dense point cloud generation. The extended quality assessment analysis was conducted in the CloudCompare program, and the comparison between point clouds and cross-sections derived from the dataset was compared. The result of performed investigation shows that the point cloud generated from photos taken with a lens with a focal length 35 mm is characterised by the highest accuracy.

A Probabilistic Approach for Stereo 3D Point Cloud Reconstruction from Airborne Single-Channel Multi-Aspect SAR Image Sequences

We investigate the problem of obtaining dense 3D reconstruction from airborne multi-aspect synthetic aperture radar (SAR) image sequences. Dense 3D reconstructions of multi-view SAR images are vulnerable to anisotropic scatters. To address this issue, we propose a probabilistic 3D reconstruction method based on jointly estimating the pixel’s height and degree of anisotropy. Specifically, we propose a mixture distribution model for the stereo-matching results, where the degree of anisotropy is modeled as an underlying error source. Then, a Bayesian filtering method is proposed for dense 3D point cloud generation. For real-time applications, redundancy in multi-aspect observations is further exploited in a probabilistic manner to accelerate the stereo-reconstruction process. To verify the effectiveness and reliability of the proposed method, 3D point cloud generation is tested on Ku-band drone SAR data for a domestic airport area.

Las2DoD: Change Detection Based on Digital Elevation Models Derived from Dense Point Clouds with Spatially Varied Uncertainty

The advances of remote sensing techniques allow for the generation of dense point clouds to detect detailed surface changes up to centimeter/millimeter levels. However, there is still a need for an easy method to derive such surface changes based on digital elevation models generated from dense point clouds while taking into consideration spatial varied uncertainty. We present a straightforward method, Las2DoD, to quantify surface change directly from point clouds with spatially varied uncertainty. This method uses a cell-based Welch’s t-test to determine whether each cell of a surface experienced a significant elevation change based on the points measured within the cell. Las2DoD is coded in Python with a simple graphic user interface. It was applied in a case study to quantify hillslope erosion on two plots: one dominated by rill erosion, and the other by sheet erosion, in southeastern United States. The results from the rilled plot indicate that Las2DoD can estimate 90% of the total measured sediment, in comparison to 58% and 70% from two other commonly used methods. The Las2DOD-derived result is less accurate (65%) but still outperforms the other two methods (30% and 48%) for the plot dominated by sheet erosion. Las2DoD captures more low-magnitude changes and is particularly useful where surface changes are small but contribute significantly to the total surface change when summed.

TERRESTRIAL LASER SCANNING AND SATELLITE DATA IN CULTURAL HERITAGE BUILDING DOCUMENTATION

Abstract. Technological advances in the field of information acquisition have led to the development of various techniques regarding building documentation. Among the proposed methods, acquisition of data without being in direct physical contact with the features under investigation could provide valuable information especially in the case of buildings or areas presenting a high cultural value. Satellite or ground-based remote sensing techniques could contribute to the protection, conservation and restoration of cultural heritage buildings, as well as in the interpretation and monitoring of their surrounding area. The increasing interest in the generation of 3D facade models for documentation of the built environment has made laser scanning a valuable tool for 3D data collection. Through the generation of dense 3D point clouds, digitization of building facades could be achieved, offering data that could be used for further processing. Satellite imagery could also contribute to this direction, extending the monitoring possibilities of the buildings’ surrounding area or even providing information regarding change detection in large-scale cultural landscapes. This paper presents the study of a mansion house built in the middle of the 18th century in northwestern Greece, using terrestrial laser scanning techniques for facade documentation, as well as satellite imagery for monitoring and interpretation purposes. The scanning process included multiple external scans of the main facade of the building which were registered using artificial targets in order to form a single colored 3D model. Further process resulted in a model that offers measurement possibilities valuable to future plans and designs for preservation and restoration. Digital processing of satellite imagery provided the extraction of additional enhanced data regarding the physiognomy of the surrounding area.

SLAM-OR: Simultaneous Localization, Mapping and Object Recognition Using Video Sensors Data in Open Environments from the Sparse Points Cloud.

In this paper, we propose a novel approach that enables simultaneous localization, mapping (SLAM) and objects recognition using visual sensors data in open environments that is capable to work on sparse data point clouds. In the proposed algorithm the ORB-SLAM uses the current and previous monocular visual sensors video frame to determine observer position and to determine a cloud of points that represent objects in the environment, while the deep neural network uses the current frame to detect and recognize objects (OR). In the next step, the sparse point cloud returned from the SLAM algorithm is compared with the area recognized by the OR network. Because each point from the 3D map has its counterpart in the current frame, therefore the filtration of points matching the area recognized by the OR algorithm is performed. The clustering algorithm determines areas in which points are densely distributed in order to detect spatial positions of objects detected by OR. Then by using principal component analysis (PCA)—based heuristic we estimate bounding boxes of detected objects. The image processing pipeline that uses sparse point clouds generated by SLAM in order to determine positions of objects recognized by deep neural network and mentioned PCA heuristic are main novelties of our solution. In contrary to state-of-the-art approaches, our algorithm does not require any additional calculations like generation of dense point clouds for objects positioning, which highly simplifies the task. We have evaluated our research on large benchmark dataset using various state-of-the-art OR architectures (YOLO, MobileNet, RetinaNet) and clustering algorithms (DBSCAN and OPTICS) obtaining promising results. Both our source codes and evaluation data sets are available for download, so our results can be easily reproduced.

Digital Terrain Models Generated with Low-Cost UAV Photogrammetry: Methodology and Accuracy

Digital terrain model (DTM) generation is essential to recreating terrain morphology once the external elements are removed. Traditional survey methods are still used to collect accurate geographic data on the land surface. Given the emergence of unmanned aerial vehicles (UAVs) equipped with low-cost digital cameras and better photogrammetric methods for digital mapping, efficient approaches are necessary to allow rapid land surveys with high accuracy. This paper provides a review, complemented with the authors’ experience, regarding the UAV photogrammetric process and field survey parameters for DTM generation using popular commercial photogrammetric software to process images obtained with fixed-wing or multicopter UAVs. We analyzed the quality and accuracy of the DTMs based on four categories: (i) the UAV system (UAV platforms and camera); (ii) flight planning and image acquisition (flight altitude, image overlap, UAV speed, orientation of the flight line, camera configuration, and georeferencing); (iii) photogrammetric DTM generation (software, image alignment, dense point cloud generation, and ground filtering); (iv) geomorphology and land use/cover. For flat terrain, UAV photogrammetry provided a horizontal root mean square error (RMSE) between 1 to 3 × the ground sample distance (GSD) and a vertical RMSE between 1 to 4.5 × GSD, and, for complex topography, a horizontal RMSE between 1 to 7 × GSD and a vertical RMSE between 1.5 to 5 × GSD. Finally, we stress that UAV photogrammetry can provide DTMs with high accuracy when the photogrammetric process variables are optimized.

A Robust Deconvolution Method of Airborne LiDAR Waveforms for Dense Point Clouds Generation in Forest

The generation of dense and accurate point clouds from airborne light detection and ranging (LiDAR) waveform data is crucial to forest inventory. This work proposes a deconvolution method with: 1) an automatic stopping criterion to differentiate near-adjacent targets and 2) an iterative false subwaveform removal algorithm to remove outliers caused by noise. Synthetic waveforms with different overlap rates were processed using the proposed method, the Gaussian decomposition (GD) method, and the Richardson Lucy (RL) deconvolution method. Results showed that: 1) the number of subwaveforms detected by the proposed method is 9% higher than that of the RL and 20% higher than that of the GD when the overlap rates are larger than 0.6 and 2) the proposed method is of the smallest ground and peak distance errors. Results of the indoor experiment also show that the proposed method is superior in finding near targets meanwhile leading to small ground and peak distance error. Furthermore, the proposed method was tested by airborne waveforms from the Dagujia forest farm. The point cloud density acquired by the proposed method is 3% and 35% higher than that by the RL and GD method. Fewer outliers are produced by the proposed method. The number of individual trees extracted from the proposed point clouds is 22%, 51%, and 57% greater than those extracted from the RL, the GD, and the reference point clouds using the canopy height model-based method. Best individual tree extraction result is produced by the proposed method, especially for an area with small trees.

Accurate and dense point cloud generation for industrial Measurement via target-free photogrammetry

Industrial photogrammetry systems commonly require multiple coded targets to establish a global coordinate frame for relatively large objects. However, there are many industrial products that do not allow coded targets to be placed on their surfaces. We propose an accurate and dense point cloud generation approach for measuring large-sized, untextured objects. Unlike most existing industrial measurement methods, our photogrammetry approach is free of coded targets. We have three core contributions. First, a Rotation-Free Digital Image Correlation (RFDIC) method is proposed to improve the multi-view stereopsis matching precision. Second, based on the technique of structure from motion (SfM), a coarse-to-fine strategy is utilized to construct the multi-view geometry and accurately optimize camera poses. Third, the devices used are consumer-friendly and commonly available, i.e., only a digital projector and a camera are needed to obtain the dense points on the measured object surface. The experimental results show that the error of reconstructed length for a scale bar is less than 0.01 mm/m. Compared to the state-of-the-art commercial measurement system, the average error of point cloud reconstruction is about 0.055 mm, which meets the accuracy demand for industrial applications.

Image enhancement comparison to improve underwater cultural heritage survey

This work aims at presenting an underwater image application to obtain an improved 3D model of cultural assets. In 2016, more than 500 images were acquired by a GoPro Camera with a low resolution of 72 dpi and focal length of 3 mm, without flash and are now used to reconstruct the 3D model of some amphoras of a Roman shipwreck found in Albenga (Italy). We have applied state-of-art image enhancement techniques, such as ACE, CLAHE, LAB and SP algorithms, to improve the quality of underwater images affected by low contrast, poor visibility conditions, not uniform lighting, colour variations, noise and blur effect. The visual quality has been evaluated through quantitative metrics, like average luminance, information entropy, average gradient of image, UCIQE and UIQM. Then, our efforts have been devoted to the dense 3D point cloud generation using a SfM-MVS software. In particular, the 3D reconstruction results are in line with the metric evaluation: in fact, the more accurate 3D objects are obtained from that enhanced dataset with the highest measured image quality.

Suitability of Automatic Photogrammetric Reconstruction Configurations for Small Archaeological Remains.

Three-dimensional (3D) reconstruction is a useful technique for the documentation, characterization, and evaluation of small archeological objects. In this research, a comparison among different photogrammetric setups that use different lenses (macro and standard zoom) and dense point cloud generation calibration processes for real specific objects of archaeological interest with different textures, geometries, and materials is raised using an automated data collection. The data acquisition protocol is carried out from a platform with control points referenced with a metrology absolute arm to accurately define a common spatial reference system. The photogrammetric reconstruction is performed considering a camera pre-calibration as well as a self-calibration. The latter is common for most data acquisition situations in archaeology. The results for the different lenses and calibration processes are compared based on a robust statistical analysis, which entails the estimation of both standard Gaussian and non-parametric estimators, to assess the accuracy potential of different configurations. As a result, 95% of the reconstructed points show geometric discrepancies lower than 0.85 mm for the most unfavorable case and less than 0.35 mm for the other cases.

OPEN-SOURCE IMAGE-BASED 3D RECONSTRUCTION PIPELINES: REVIEW, COMPARISON AND EVALUATION

Abstract. State-of-the-art automated image orientation (Structure from Motion) and dense image matching (Multiple View Stereo) methods commonly used to produce 3D information from 2D images can generate 3D results – such as point cloud or meshes – of varying geometric and visual quality. Pipelines are generally robust and reliable enough, mostly capable to process even large sets of unordered images, yet the final results often lack completeness and accuracy, especially while dealing with real-world cases where objects are typically characterized by complex geometries and textureless surfaces and obstacles or occluded areas may also occur. In this study we investigate three of the available commonly used open-source solutions, namely COLMAP, OpenMVG+OpenMVS and AliceVision, evaluating their results under diverse large scale scenarios. Comparisons and critical evaluation on the image orientation and dense point cloud generation algorithms is performed with respect to the corresponding ground truth data. The presented FBK-3DOM datasets are available for research purposes.

Smartphone-Based Photogrammetry for the 3D Modeling of a Geomorphological Structure

The geomatic survey in the speleological field is one of the main activities that allows for the adding of both a scientific and popular value to cave exploration, and it is of fundamental importance for a detailed knowledge of the hypogean cavity. Today, the available instruments, such as laser scanners and metric cameras, allow us to quickly acquire data and obtain accurate three-dimensional models, but they are still expensive, require a careful planning phase of the survey, as well as some operator experience for their management. This work analyzes the performance of a smartphone device for a close-range photogrammetry approach for the extraction of accurate three-dimensional information of an underground cave. The image datasets that were acquired with a high-end smartphone were processed using the Structure from Motion (SfM)-based approach for dense point cloud generation: different image-matching algorithms implemented in a commercial and an open source software and in a smartphone application were tested. In order to assess the reachable accuracy of the proposed procedure, the achieved results were compared with a reference dense point cloud obtained with a professional camera or a terrestrial laser scanner. The approach has shown a good performance in terms of geometrical accuracies, computational time and applicability.

INVESTIGATIONS INTO THE QUALITY OF IMAGE-BASED POINT CLOUDS FROM UAV IMAGERY

Abstract. The quality of image-based point clouds generated from images of UAV aerial flights is subject to various influencing factors. In addition to the performance of the sensor used (a digital camera), the image data format (e.g. TIF or JPG) is another important quality parameter. At the UAV test field at the former Zollern colliery (Dortmund, Germany), set up by Bochum University of Applied Sciences, a medium-format camera from Phase One (IXU 1000) was used to capture UAV image data in RAW format. This investigation aims at evaluating the influence of the image data format on point clouds generated by a Dense Image Matching process. Furthermore, the effects of different data filters, which are part of the evaluation programs, were considered. The processing was carried out with two software packages from Agisoft and Pix4D on the basis of both generated TIF or JPG data sets. The point clouds generated are the basis for the investigation presented in this contribution. Point cloud comparisons with reference data from terrestrial laser scanning were performed on selected test areas representing object-typical surfaces (with varying surface structures). In addition to these area-based comparisons, selected linear objects (profiles) were evaluated between the different data sets. Furthermore, height point deviations from the dense point clouds were determined using check points. Differences in the results generated through the two software packages used could be detected. The reasons for these differences are filtering settings used for the generation of dense point clouds. It can also be assumed that there are differences in the algorithms for point cloud generation which are implemented in the two software packages. The slightly compressed JPG image data used for the point cloud generation did not show any significant changes in the quality of the examined point clouds compared to the uncompressed TIF data sets.

LANDSLIDE DEFORMATION MONITORING BY THREE-CAMERA IMAGING SYSTEM

Abstract. Landslide deformation is a critical issue for WuWanZai as it will affect the road safety and cause transportation problem. Since the relief of this area is about 400 meters with an area of tens of hectares, we use an unmanned helicopter equipped with a three-camera imaging system to acquire high spatial resolution images in order to measure detail terrain variation. The unmanned helicopter can fly according to terrain slope to obtain 1–3 cm spatial resolution images. The acquired three-camera images are stitched into one perspective image in advance to construct a large format virtual image with a frame size of 34 mm by 78 mm and a FOV of 53° × 97°. Integrating ground control points that were measure by static GNSS continuous observation, we conduct aerial triangulation and dense point cloud generation by PhotoscanPro. We have acquired six dataset of UAV images since April 20, 2018. Then, we have conducted cloud-to-cloud distance calculation, DSM elevation difference calculation, ortho-image change analysis, photogrammetric points and GNSS stations displacement analysis, etc. In the end, from photogrammetric point displacement analysis, we have detected 1.6 meters displacement around the fourth curve of WuWanZai due to a heavy rainfall occurred at June 20. Based on the cloud-to-cloud distance analysis and DSM elevation difference results, we have observed more than 5 meters of height difference at the landslide area due to another heavy rainfall happened at Oct. 23–24. Experimental results demonstrate that by using the proposed UAV and three-camera imaging system can effectively detect landslide deformation in high accuracy.

UAV-Based Digital Terrain Model Generation under Leaf-Off Conditions to Support Teak Plantations Inventories in Tropical Dry Forests. A Case of the Coastal Region of Ecuador.

Remote sensing is revolutionizing the way in which forests studies are conducted, and recent technological advances, such as Structure from Motion (SfM) photogrammetry from Unmanned Aerial Vehicle (UAV), are providing more efficient methods to assist in REDD (Reducing Emissions from Deforestation and forest Degradation) monitoring and forest sustainable management. The aim of this work was to develop and test a methodology based on SfM from UAV to generate high quality Digital Terrain Models (DTMs) on teak plantations (Tectona grandis Linn. F.) situated in the Coastal Region of Ecuador (dry tropical forest). UAV overlapping images were collected using a DJI Phantom 4 Advanced© quadcopter during the dry season (leaf-off phenological stage) over 58 teak square plots of 36 m side belonging to three different plantations located in the province of Guayas (Ecuador). A workflow consisting of SfM absolute image alignment based on field surveyed ground control points, very dense point cloud generation, ground points filtering and outlier removal, and DTM interpolation from labeled ground points, was accomplished. A very accurate Terrestrial Laser Scanning (TLS) derived ground points were employed as ground reference to estimate the UAV-SfM DTM vertical error in each reference plot. The plot-level obtained DTMs presented low vertical bias and random error (−3.1 cm and 11.9 cm on average, respectively), showing statistically significant greater error in those reference plots with basal area and estimated vegetation coverage above 15 m2/ha and 60%, respectively. To the best of the authors’ knowledge, this is the first study aimed at monitoring of teak plantations located in dry tropical forests from UAV images. It provides valuable information that recommends carrying out the UAV image capture during the leaf-off season to obtain UAV-SfM derived DTMs suitable to serve as ground reference in supporting teak plantations inventories.

A non-contact measurement method of ship block using image-based 3D reconstruction technology

Nowadays, the traditional measurement methods applied in ship sections/blocks are heavily relied on years of experiences and could hardly meet the shipbuilding requirements due to their complicated and time-consuming processes. Furthermore, along with the development of photogrammetry, the image-based three-dimensional (3D) reconstruction technology has emerged and become flourishing. In this paper, we propose a novel measurement method based on image-based 3D reconstruction technology to measure the construction deformation of ship blocks with high accuracy in a simple way. The scheme only needs photos to rebuild the 3D model of ship block. It includes the camera calibration, the image preprocessing, the feature points matching followed by the steps of dense point cloud generation, meshes partition and 3D reconstruction with texture mapping. In the step of feature points matching, the scale-invariant feature transform (SIFT) algorithm is used and optimized with the addition of the global vector to reduce the mismatch and enhance the reconstruction accuracy. The shipyard experiments with shooting photos of arbitrary steel parts proved that the proposed image-based 3D reconstruction technology is able to be applied to measure the ship blocks with satisfied accuracy and less time in shipbuilding industry or other industries with large-scale one-of-a-kind product as well.