Precise Point Cloud Research Articles

Building Maps Using Monocular Image-feeds from Windshield-mounted Cameras in a Simulator Environment

3-dimensional, accurate, and up-to-date maps are essential for vehicles with autonomous capabilities, whose functionality is made possible by machine learning-based algorithms. Since these solutions require a tremendous amount of data for parameter optimization, simulation-to-reality (Sim2Real) methods have been proven immensely useful for training data generation. For creating realistic models to be used for synthetic data generation, crowdsourcing techniques present a resource-efficient alternative. In this paper, we show that using the Carla simulation environment, a crowdsourcing model can be created that mimics a multi-agent data gathering and processing pipeline. We developed a solution that yields dense point clouds based on monocular images and location information gathered by individual data acquisition vehicles. Our method provides scene reconstructions using the robust Structure-from-Motion (SfM) solution of Colmap. Moreover, we introduce a solution for synthesizing dense ground truth point clouds originating from the Carla simulator using a simulated data acquisition pipeline. We compare the results of the Colmap reconstruction with the reference point cloud after aligning them using the iterative closest point algorithm. Our results show that a precise point cloud reconstruction was feasible with this crowdsourcing-based approach, with 54\% of the reconstructed points having an error under 0.05 m, and a weighted root mean square error of 0.0449 m for the entire point cloud.

Static and Dynamic Performance Evaluation of a Solid-State LiDAR for 3D Object Detection in Greenhouse Spray Applications

Highlights Comprehensive evaluation of the measurement accuracy of an inexpensive solid-state LiDAR for object detection. Development of an algorithm to acquire point clouds of objects with various shapes under both static and dynamic conditions. Utilization of pseudo-color images to assess the surfaces of regular-shaped cartons and irregular artificial plants. Proposal for integrating the solid-state LiDAR into variable-rate spray applications for greenhouses. Abstract. An effective variable-rate spraying system for greenhouses requires accurate canopy structure parameters of plants to ensure proper pesticide dosage adjustment. While conventional laser systems integrated into spray systems can provide precise point cloud data of plants, they still present a high expense. This study examines the performance of a recently introduced, cost-effective, and high-resolution solid-state LiDAR (Intel RealSense L515) in relation to its potential for greenhouse spray applications. Additionally, a specialized point cloud acquisition algorithm was developed for this solid-state LiDAR to obtain the geometrical parameters of objects. To assess the LiDAR sensor's suitability for greenhouse spray applications, the performance of the LiDAR sensor and the algorithm was evaluated using five different sized regular-shaped cartons and three artificial plants with complex geometry. Various factors were analyzed, such as the horizontal distances between objects and the LiDAR sensor (0.5, 1.0, 1.5, 2.0, 2.5, and 3.0 m), the tilt angle of the LiDAR sensor relative to the ground (45°, 60°, and 75°), the height of the LiDAR sensor from the ground (ranging from 0.3 to 0.8 m with 0.5 m distance intervals), and the forward speed of the LiDAR sensor (0.1, 0.3, 0.6, and 0.9 m s-1). The findings revealed that the optimal detection distance for this LiDAR sensor is 1.0 m. Increasing or decreasing the detection distance of the object relative to the LiDAR sensor diminished the measurement accuracy. The accuracy of the derived geometrical variables was affected by the height and tilt angle of the LiDAR sensor. Nevertheless, the geometrical parameters obtained from the solid-state LiDAR showed a favorable correspondence with the results of manual measurements. The highest root mean square error (RMSE) and coefficient of variation (CV) for the overall test were 14.3 mm and 14.3% in the X (length) direction, 14.3 mm and 14.3% in the Y (width) direction, and 10.8 mm and 10.8% in the Z (height) direction, respectively. The contour Edge Similarity Score for objects measured using the solid-state LiDAR and images obtained with an RGB camera exceeded 0.90. These findings suggest that the proposed solid-state LiDAR and the specifically designed algorithm could be effectively adapted to acquire the geometrical parameters of objects and to develop precise variable-rate spraying systems for greenhouse applications. Keywords: Canopy structure measurements, Point cloud, Precision agriculture, Precision spray technology, Variable-rate spraying systems.

A Method Based on Improved iForest for Trunk Extraction and Denoising of Individual Street Trees

Currently, the street tree resource survey using Mobile laser scanning (MLS) represents a hot spot around the world. Refined trunk extraction is an essential step for 3D reconstruction of street trees. However, due to scanning errors and the effects of occlusion by various types of features in the urban environment, street tree point cloud data processing has the problem of excessive noise. For the noise points that are difficult to remove using statistical methods in close proximity to the tree trunk, we propose an adaptive trunk extraction and denoising method for street trees based on an improved iForest (Isolation Forest) algorithm. Firstly, to extract the individual tree trunk points, the trunk and the crown are distinguished from the individual tree point cloud through point cloud slicing. Next, the iForest algorithm is improved by conducting automatic calculation of the contamination and further used to denoise the tree trunk point cloud. Finally, the method is validated with five datasets of different scenes. The results indicate that our method is robust and effective in extracting and denoising tree trunks. Compared with the traditional Statistical Outlier Removal (SOR) filter and Radius filter denoising methods, the denoising accuracy of the proposed method can be improved by approximately 30% for noise points close to tree trunks. Compared to iForest, the proposed method automatically calculates the contamination, improving the automation of the algorithm. Our method can provide more precise trunk point clouds for 3D reconstruction of street trees.

CAOM: Change-aware online 3D mapping with heterogeneous multi-beam and push-broom LiDAR point clouds

A High-Definition (HD) map is an essential component not only for autonomous vehicles but also for surveyors in favor of city management. Moreover, the HD map must be up-to-date by reflecting environmental changes continuously. The development of LiDAR Simultaneous Localization And Mapping (SLAM) techniques has provided an effective way to collect precise point clouds for the construction of HD maps, while traditional methods are time-consuming and labor-intensive on account of the complex street scene. Furthermore, life-long 3D mapping is quite necessary and challenging nowadays. In this paper, we introduce a novel change-aware 3D online mapping framework CAOM,11Code at https://github.com/DCSI2022/CAOM. using point clouds collected by Multi-Beam LiDAR (MBL) and Push-Broom LiDAR (PBL) for rapid city map update and change detection. Concerning the prior map with higher precision, heterogenous fusion is performed to rectify the SLAM drift, based on the View-Traced Clouds (ViTC) derived according to their view in common. Meanwhile, corresponding virtual range images are integrated with distance images to detect the 3D changes between the current map and the historical reference map offline. The detection results are later fused in a probabilistic way along with the homogeneous fusion process where a resilient graph consisting of diverse factors is established to maintain the local and global consistency of the final point cloud map. Compared with 3D-CSTM, the real-world experiments have shown an average 0.5 m and max 1.5 m improvement in the ATE (Absolute Translation Error) of the trajectory generated by our system. Quantitative measurements on the individual point accuracy are also conducted to verify the performance of 3D mapping with up to 16% improvement in comparison with respect to the 3D-CSTM method. The effects of the disparity detection and probabilistic fusion can be revealed from the small objects distinguished in the point cloud map as well as > 95% overall accuracy.

Filtracja chmur punktów za pomocą dopasowania danych 2D-3D

Precision is a key feature for the development of 3D measurement systems. Time-of-flight cameras used for such measurements create point clouds containing a lot of noise, which may not be useful for further analysis. In our research to solve this problem, we propose a new method for precise point cloud filtering. We use 2D information from a telecentric lens camera to remove outlier points from 3D measurements recorded with a Time-of-Flight camera. The use of a telecentric camera allows us to obtain the most precise information about the contour of an object, which allows us to accurately filter the object reconstruction in 3D.

Exploring the potential of 3D scanning in Industry 4.0: An overview

A 3D scanner is a non-contact, non-destructive digital device that uses a light line/laser to accurately capture the shape of a physical object into Computer-Aided Design (CAD) data. It generates a point cloud or a set of data points in a coordinate system that accurately depicts a physical object's size and shape. The urge of human beings to build and recreate 3D objects is an essential requirement in the context of Industry 4.0. In Industry 4.0, 3D scanners are helpful for designing, assessing the minor features of any product, capturing freeform, and providing precise point clouds for complicated geometry and curved surfaces. Today advanced technologies are being introduced in industries, and 3D scanning technology is one of the latest emerging technologies. The situation has progressed significantly, and the ever-emerging world of 3D technology still disrupts a variety of sectors. This paper discusses 3D Scanning, its working process, and its adoption for reverse engineering and Industry 4.0 culture. Finally, this paper covers essential features, traits, and applications of 3D Scanning for Industry 4.0. Using different methods, 3D Scanning can collect information about an item, about an object's highness, breadth, and depth. 3D scanners are available on the market from high-end possibilities. They have different resolutions, and now they provide details of the scan to superimpose colour on an object, but the industry wants them to be developed for other quality attributes.

CO-REGISTRATION OF TLS POINT CLOUDS WITH SCAN-PATCHES AND BIM-FACES

Abstract. To geometrically compare point clouds with Building Information Modeling (BIM) models, they must be transformed to the same coordinate system. The determination of the transformation parameters is called registration. If another model is used instead of control points, the term co-registration has recently been established. The plane-based (relative) registration of point clouds from terrestrial laser scanning (TLS) is successfully used in practice with high accuracy and quality assurance. Based on this, we present a special co-registration approach, that makes use of planes from BIM models. Most buildings have a large number of planar faces, so it is obvious to use faces from BIM. We present intermediate research results of an approach that uses the local planes (patches) of the TLS point clouds also to determine the transformation parameters for the registration to BIM. The aim is to avoid the need of ground control points. The approach requires a (partially) existing BIM model in which the highly precise point cloud is to be registered. In the long term, the procedure is developed for automated construction progress documentation.

Artificial vision system for the automatic inspection of preforms

The preforming consists of processing the reinforcement to obtain a geometry of the dry fiber, close to that of the final product before being impregnated by the resin during the injection process. During the process, the fibers are manipulated to reach the desired shape are subjected to tensile, shear and torsion stresees that can cause changes in the orientation of the fibers, misalignments,or other defects such as wrinkles or fiber breakage that can affect the performance of the preform . In this work a solution for the automation of the quality control of the preforming process by artificial vision is investigated.. Two approaches are proposed, 3D vision for the geometric and superficial inspection of the fibers and machine learning aproach based on Deep Learning (DL) to detect irregularities in the orientation. The solution based on 3D vision automatically detects defects based on the comparison of a theoretical 3D model and a 3D reconstruction of the real part. A structured light system is used to generate dense and precise point clouds. Once aligned, the dissimilarities between both surfaces are analyzed. The solution is complemented by a 2D vision system based on DL that classifies the irregularities in the orientations of the fabric. The model is trained with images of classified fiber orientations to automatically identify areas of fiber whose orientation does not correspond to the expected one. The combination of both technologies allows to give a complete solution to the automated quality inspection of preforms for manufacturing or defects.

DF-Net: Dynamic and Folding Network for 3D Point Cloud Completion

The development of 3D sensors encourages researchers to process point cloud data directly. Point cloud data requires less memory but conveys more detailed 3D shape information. However, because of occlusion, sensing distance and other reasons, sensors usually cannot get a complete 3D shape. In this paper, we propose a Dynamic and Folding Network (DF-Net) to address the precise point cloud completion problem. Existing completion networks generate the overall shape of a point cloud from an incomplete point cloud. In this paper, we complete the missing part based on the existing part instead. We use a dynamic graph network to better extract local features of points in the neighborhood points. A FoldBlock is used to refine the prediction of the missing part. We validate our method with two benchmarks, ShapeNet-13 and ShapeNet-55. Both qualitative and quantitative experimental results show the proposed method achieves improvement over some state-of-the-art methods. Code is available at https://github.com/yiqisetian/DF-Net.

OC-SLAM: Steadily Tracking and Mapping in Dynamic Environments

Visual Simultaneous Localization and Mapping (SLAM) system is mainly used in real-time localization and mapping tasks of robots in various complex environments, while traditional monocular vision algorithms are struggling to cope with weak texture and dynamic scenes. To solve these problems, this work presents an object detection and clustering assisted SLAM algorithm (OC-SLAM), which adopts a faster object detection algorithm to add semantic information to the image and conducts geometrical constraint on the dynamic keypoints in the prediction box to optimize the camera pose. It also uses RGB-D camera to perform dense point cloud reconstruction with the dynamic objects rejected, and facilitates European clustering of dense point clouds to jointly eliminate dynamic features combining with object detection algorithm. Experiments in the TUM dataset indicate that OC-SLAM enhances the localization accuracy of the SLAM system in the dynamic environments compared with original algorithm and it has shown impressive performance in the localizition and can build a more precise dense point cloud map in dynamic scenes.

3D Reconstruction of Coastal Cliffs from Fixed-Wing and Multi-Rotor UAS: Impact of SfM-MVS Processing Parameters, Image Redundancy and Acquisition Geometry

Monitoring the dynamics of coastal cliffs is fundamental for the safety of communities, buildings, utilities, and infrastructures located near the coastline. Structure-from-Motion and Multi View Stereo (SfM-MVS) photogrammetry based on Unmanned Aerial Systems (UAS) is a flexible and cost-effective surveying technique for generating a dense 3D point cloud of the whole cliff face (from bottom to top), with high spatial and temporal resolution. In this paper, in order to generate a reproducible, reliable, precise, accurate, and dense point cloud of the cliff face, a comprehensive analysis of the SfM-MVS processing parameters, image redundancy and acquisition geometry was performed. Using two different UAS, a fixed-wing and a multi-rotor, two flight missions were executed with the aim of reconstructing the geometry of an almost vertical cliff located at the central Portuguese coast. The results indicated that optimizing the processing parameters of Agisoft Metashape can improve the 3D accuracy of the point cloud up to 2 cm. Regarding the image acquisition geometry, the high off-nadir (90°) dataset taken by the multi-rotor generated a denser and more accurate point cloud, with lesser data gaps, than that generated by the low off-nadir dataset (3°) taken by the fixed wing. Yet, it was found that reducing properly the high overlap of the image dataset acquired by the multi-rotor drone permits to get an optimal image dataset, allowing to speed up the processing time without compromising the accuracy and density of the generated point cloud. The analysis and results presented in this paper improve the knowledge required for the 3D reconstruction of coastal cliffs by UAS, providing new insights into the technical aspects needed for optimizing the monitoring surveys.

A SPECTRALLY IMPROVED POINT CLOUD CLASSIFICATION METHOD FOR MULTISPECTRAL LIDAR

Abstract. Precise point cloud classification can enhance lidar performance in various applications, such as land cover mapping, forestry management and autonomous driving. The development of multispectral lidar improves classification performance with rich spectral information. However, the employment of spectral information for classification is still underdeveloped. Therefore, we proposed a spectrally improved classification method for multispectral LiDAR. We conducted spectral improvement in two aspects: (1) we improved the eigenentropy-based neighbourhood selection by spectral angle match (SAM) to reform the more reliable neighbour; (2) we utilized both geometric and spectral features and compare the contributions of these features. A three-wavelength multispectral lidar and a complex indoor experimental scene were used for demonstration. The results indicate the effectiveness of our proposed spectrally improved method and the promising potential of spectral information on lidar classification.

RELATION NETWORK FOR FULL-WAVEFORMS LIDAR CLASSIFICATION

Abstract. LiDAR data are widely used in various domains related to geosciences (flow, erosion, rock deformations, etc.), computer graphics (3D reconstruction) or earth observation (detection of trees, roads, buildings, etc.). Because of the unstructured nature of remaining 3D points and because of the cost of acquisition, the LiDAR data processing is still challenging (few learning data, difficult spatial neighboring relationships, etc.). In practice, one can directly analyze the 3D points using feature extraction and then classify the points via machine learning techniques (Brodu, Lague, 2012, Niemeyer et al., 2014, Mallet et al., 2011). In addition, recent neural network developments have allowed precise point cloud segmentation, especially using the seminal pointnet network and its extensions (Qi et al., 2017a, Riegler et al., 2017). Other authors rather prefer to rasterize / voxelize the point cloud and use more conventional computers vision strategies to analyze structures (Lodha et al., 2006). In a recent work, we demonstrated that Digital Elevation Models (DEM) is reductive of the vertical component complexity describing objects in urban environments (Guiotte et al., 2020). These results highlighted the necessity to preserve the 3D structure of the point cloud as long as possible in the processing. In this paper, we therefore rely on ortho-waveforms to compute a land cover map. Ortho-waveforms are directly computed from the waveforms in a regular 3D grid. This method provides volumes somehow “similar” to hyperspectral data where each pixel is here associated with one ortho-waveform. Then, we exploit efficient neural networks adapted to the classification of hyperspectral data when few samples are available. Our results, obtained on the 2018 Data Fusion Contest dataset (DFC), demonstrate the efficiency of the approach.

Tidal Creek Extraction from Airborne LiDAR Data Using Ground Filtering Techniques

As elongated indentations or valleys in a wetland caused by tidal currents, tidal creeks act as drainage pathways and promote tidal flat evolution. Determining their geometric information is essential for topographical research of tidal flats. The airborne light detection and ranging (LiDAR) system has been the most efficient surveying technique in tidal topography because it can directly acquire precise geo-referenced point clouds for wide areas. Existing tidal creek extraction methods using airborne LiDAR data have limitations such as excessive user intervention, lack of adaptability to various shapes and sizes of tidal creeks, and decreased precision due to conversion to the digital elevation model. This study aims to overcome these limitations and effectively extract various types of tidal creeks by utilizing ground filtering which is a technique to filter off-ground objects (such as buildings, trees, etc.) in land LiDAR surveys. To derive a suitable method for tidal creek extraction, three verified ground filtering techniques, adaptive triangulated irregular network, gLiDAR, and cloth simulation filtering (CSF), were selected and tested using LiDAR point data. We modified the application procedure and optimized their parameters to enable tidal creek extraction. Our results confirmed that CSF can extract various tidal creeks with minimal user intervention. Finally, we calculated their depths and generated a tidal creek map.

An Object-Based Bidirectional Method for Integrated Building Extraction and Change Detection between Multimodal Point Clouds

Building extraction and change detection are two important tasks in the remote sensing domain. Change detection between airborne laser scanning data and photogrammetric data is vulnerable to dense matching errors, mis-alignment errors and data gaps. This paper proposes an unsupervised object-based method for integrated building extraction and change detection. Firstly, terrain, roofs and vegetation are extracted from the precise laser point cloud, based on “bottom-up” segmentation and clustering. Secondly, change detection is performed in an object-based bidirectional manner: Heightened buildings and demolished buildings are detected by taking the laser scanning data as reference, while newly-built buildings are detected by taking the dense matching data as reference. Experiments on two urban data sets demonstrate its effectiveness and robustness. The object-based change detection achieves a recall rate of 92.31% and a precision rate of 88.89% for the Rotterdam dataset; it achieves a recall rate of 85.71% and a precision rate of 100% for the Enschede dataset. It can not only extract unchanged building footprints, but also assign heightened or demolished labels to the changed buildings.

Precise iterative closest point algorithm for RGB-D data registration with noise and outliers

In this paper, we propose a precise Iterative Closest Point (ICP) algorithm which is a RGB information supplemented registration method based on correntropy measurement, which can overcome noise and outliers to complete precise point cloud registration. Firstly, in order to achieve accurate point set registration of single structure geometry point cloud with a large number of planes information rather than geometry information, we add the RGB information to our initial method as information compensation, which greatly ensure our algorithm could achieve accurate registration. Secondly, due to the noise and outliers existed in RGB-D data can easily cause deviations in registration, in order to achieve precise registration in RGB-D data, we introduced the correntropy measurement method into the initial model. Experimental results on our scene dataset demonstrate the proposed algorithm can achieve precise point set registration in RGB-D scene data well.

Indoor 3D Reconstruction Using Camera, IMU and Ultrasonic Sensors

The recent advances in sensing and display technologies have been transforming our living environments drastically. In this paper, a new technique is introduced to accurately reconstruct indoor environments in three-dimensions using a mobile platform. The system incorporates 4 ultrasonic sensors scanner system, an HD web camera as well as an inertial measurement unit (IMU). The whole platform is mountable on mobile facilities, such as a wheelchair. The proposed mapping approach took advantage of the precision of the 3D point clouds produced by the ultrasonic sensors system despite their scarcity to help build a more definite 3D scene. Using a robust iterative algorithm, it combined the structure from motion generated 3D point clouds with the ultrasonic sensors and IMU generated 3D point clouds to derive a much more precise point cloud using the depth measurements from the ultrasonic sensors. Because of their ability to recognize features of objects in the targeted scene, the ultrasonic generated point clouds performed feature extraction on the consecutive point cloud to ensure a perfect alignment. The range measured by ultrasonic sensors contributed to the depth correction of the generated 3D images (the 3D scenes). Experiments revealed that the system generated not only dense but precise 3D maps of the environments. The results showed that the designed 3D modeling platform is able to help in assistive living environment for self-navigation, obstacle alert, and other driving assisting tasks.

Real-Time Inspection System for Ballast Railway Fasteners Based on Point Cloud Deep Learning

Rail fasteners are the most numerous components in railways and they should be inspected periodically. Manual inspection is currently a common solution, which is laborious and low-efficient. Some automatic inspection approaches are proposed. But for ballast railway fasteners inspection, debris, especially ballast along tracks may cover the fasteners, which is still a tricky problem. In this paper, a real-time inspection system for ballast railway fasteners based on point cloud deep learning is developed. Dense and precise point cloud of fastener is obtained from the structured light sensors in the system. The point cloud of fastener is segmented into different parts to avoid the interference of debris on fasteners. A ballast fastener point cloud semantic segmentation dataset is created based on automatic annotation method. Several deep learning point cloud segmentation models are tested in this dataset and PointNet++ is selected to be deployed in the real-time deep learning module of the system. Field tests on ballast railways show excellent accuracy and efficiency of this system.

High-Resolution Representation for Mobile Mapping Data in Curved Regular Grid Model.

As mobile mapping systems become a mature technology, there are many applications for the process of the measured data. One interesting application is the use of driving simulators that can be used to analyze the data of tire vibration or vehicle simulations. In previous research, we presented our proposed method that can create a precise three-dimensional point cloud model of road surface regions and trajectory points. Our data sets were obtained by a vehicle-mounted mobile mapping system (MMS). The collected data were converted into point cloud data and color images. In this paper, we utilize the previous results as input data and present a solution that can generate an elevation grid for building an OpenCRG model. The OpenCRG project was originally developed to describe road surface elevation data, and also defined an open file format. As it can be difficult to generate a regular grid from point cloud directly, the road surface is first divided into straight lines, circular arcs, and and clothoids. Secondly, a non-regular grid which contains the elevation of road surface points is created for each road surface segment. Then, a regular grid is generated by accurately interpolating the elevation values from the non-regular grid. Finally, the curved regular grid (CRG) model files are created based on the above procedures, and can be visualized by OpenCRG tools. The experimental results on real-world data show that the proposed approach provided a very-high-resolution road surface elevation model.

Automated Road Curb Break Lines Extraction from Mobile LiDAR Point Clouds

In last decades, Mobile Light Detection And Ranging (LiDAR) systems were revealed to be an efficient and reliable method to collect dense and precise point clouds. The challenge now faced by researchers is the automatic object extraction from those point clouds, such as the curb break lines, which are essential to road rehabilitation projects and autonomous driving. Throughout this work, an efficient method to extract road curb break lines from mobile LiDAR point clouds is presented. The proposed method was based on the system working principles instead of an algorithmic application over the cloud as a mass of points. The point cloud was decomposed in the original sensor scan profiles. Then, a GPS epoch versus trajectory distance was used to eliminate most non-ground points. Finally, through a vertical monotone chain decomposition, candidate point arrays were created and the curb break lines are formed. The proposed method was shown to be able to avoid the occlusion effect caused by undergrowth. The method allows for distinguishing between right and left curbs and works on curved curbs. Both top and bottom tridimensional break lines were extracted. When compared with a reference manual method, in the tested dataset, the proposed method allowed for a decrease in the curb break lines extraction time from 25 min to less than 30 s. The extraction method provided completeness and correctness rates above 95% and 97%, respectively, and a quality value higher than 93%.