High-quality Point Research Articles

In Situ Calibration and Trajectory Enhancement of UAV and Backpack LiDAR Systems for Fine-Resolution Forest Inventory

Forest inventory has been relying on labor-intensive manual measurements. Using remote sensing modalities for forest inventory has gained increasing attention in the last few decades. However, tools for deriving accurate tree-level metrics are limited. This paper investigates the feasibility of using LiDAR units onboard uncrewed aerial vehicle (UAV) and Backpack mobile mapping systems (MMSs) equipped with an integrated Global Navigation Satellite System/Inertial Navigation System (GNSS/INS) to provide high-quality point clouds for accurate, fine-resolution forest inventory. To improve the quality of the acquired point clouds, a system-driven strategy for mounting parameters estimation and trajectory enhancement using terrain patches and tree trunks is proposed. By minimizing observed discrepancies among conjugate features captured at different timestamps from multiple tracks by single/multiple systems, while considering the absolute and relative positional/rotational information provided by the GNSS/INS trajectory, system calibration parameters and trajectory information can be refined. Furthermore, some forest inventory metrics, such as tree trunk radius and orientation, are derived in the process. To evaluate the performance of the proposed strategy, three UAV and two Backpack datasets covering young and mature plantations were used in this study. Through sequential system calibration and trajectory enhancement, the spatial accuracy of the UAV point clouds improved from 20 cm to 5 cm. For the Backpack datasets, when the initial trajectory was of reasonable quality, conducting trajectory enhancement significantly improved the relative alignment of the point cloud from 30 cm to 3 cm, and an absolute accuracy at the 10 cm level can be achieved. For a lower-quality trajectory, the initial 1 m misalignment of the Backpack point cloud was reduced to 6 cm through trajectory enhancement. However, to derive products with accurate absolute accuracy, UAV point cloud is required as a reference in the trajectory enhancement process of the Backpack dataset.

A Simple Point-based Iso-Scallop Tool Path Planning Method for Noisy Point Clouds with High Robustness and Controlled Errors

With the rapid advances in 3D scanning and vision technologies, point clouds have seen a significant increase in their application to CNC machining. As an important way to generate tool paths from point clouds, point-based direct planning methods are, however, now restricted to high-quality point clouds. Most existing methods would have robustness issues if a noisy point cloud is given due to their reliance on the noise-sensitive estimation of differential properties from point clouds. Error control under noisy point clouds is also rarely considered. This paper presents a robust and error-controlled point-based iso-scallop tool path planning method for CNC machining with ball-nosed cutters. It casts the problem of searching for scallop/CL points to another problem of finding circle-sphere intersection points and the maximum/minimum among them. Both circle-sphere intersections and max/min comparisons are easy to implement and robust. It has also been theoretically proved that the intersections and comparisons can ensure an error bound of 2ϵ on the final results if two conditions called ϵ-sampling and ϵ-covering are met. The effectiveness of the method has been demonstrated by a series of examples and comparisons.

MOBILE LASER SCANNING WITH LOW-COST NAVIGATION SENSORS: COMPENSATING FOR LOW-GRADE IMU WITH DUAL-GNSS AND TIGHTLY-COUPLED LIDAR

Abstract. Kinematic laser scanning enables efficient and accurate acquisition of 3D data, specifically point clouds. Transforming the laser ranging measurements from the scanner coordinate system to a georeferenced coordinate system requires integration with auxiliary navigation systems, in this case using GNSS (global navigation satellite system) and an IMU (inertial measurement unit). GNSS and IMU data are commonly fused via Kalman filter, and the resulting trajectory is used for georeferencing. Errors in the trajectory propagate through the georeferencing and cause discrepancies in the point cloud. To mitigate this, we present a holistic integration method incorporating GNSS, IMU and LiDAR measurements in a single adjustment, which can be seamlessly adapted to a setup with two GNSS receivers and antennas. This tight coupling of LiDAR and IMU together with the dual-GNSS setup allows the use of an ultra low-cost IMU while still achieving high-quality point clouds. We demonstrate this methodology on two datsets, where we discuss in particular the boresight calibration of such a system, and the impact of the LiDAR measurements and the dual-GNSS set-up on the trajectory and the point cloud.

AFLI-Calib: Robust LiDAR-IMU extrinsic self-calibration based on adaptive frame length LiDAR odometry

As an effective complement to common laser scanning systems, the portable laser scanning system can acquire point clouds flexibly and quickly. Calibration between Light detection and ranging (LiDAR) sensors and inertial measurement units (IMU) is the prerequisite for laser scanning systems to obtain high-quality point clouds. Related methods have been proposed in the last two decades, where the global navigation satellite system (GNSS) or high-precision calibration fields are commonly used. However, the extrinsic self-calibration of LiDAR-IMU is challenging, due to the large distortion in single-frame point cloud caused by rapid motion and the position errors of IMU integration which drift quickly. At the same time, the highly dynamic motion patterns of portable devices and the changes in the scanned scene structure are not well considered in existing LiDAR odometry methods. To take better advantage of the characteristics of non-repetitive scanning LiDAR sensor, this paper proposes AFLI-Calib, which utilizes adaptive frame length LiDAR odometry to perform the extrinsic self-calibration of LiDAR-IMU. Unlike LiDAR odometry methods with a fixed frame length, the LiDAR frame length is dynamically adjusted according to the motion state of sensors and the matching stability of scenes. The single-frame point cloud is registered to the map through a linear-based continuous-time model, eliminating the motion distortion correction in advance. For further optimization of trajectory and extrinsic parameters, IMU raw measurements and LiDAR observations are involved in the multi-constraint optimization, through tightly-coupled IMU pre-integration constraints, LiDAR point-to-plane constraints, and prior constraints. The method is fully validated using self-collected calibration data of indoor and outdoor scenes and different motion modes. Experiments show that on the test data, the translation parameter accuracy of the method is 0.041 m, which is 56.3% higher than the state-of-the-art method. The standard deviation is significantly reduced, with translation deviation (0.017 m, 0.024 m, 0.022 m) and rotation deviation (0.17°, 0.25°, 0.15°), which verifies the robustness of our method. The average RMSE of distances to the reference point cloud acquired by the terrestrial laser scanning system (TLS) is 0.042 m, showing a high accuracy calibration result. Comparative experiments with the fixed frame length LiDAR odometry method and classical “correction-then-registration” motion distortion model further verify the superiority and effectiveness of the proposed adaptive frame length LiDAR odometry.

Progress and perspectives of point cloud intelligence

ABSTRACT With the rapid development of reality capture methods, such as laser scanning and oblique photogrammetry, point cloud data have become the third most important data source, after vector maps and imagery. Point cloud data also play an increasingly important role in scientific research and engineering in the fields of Earth science, spatial cognition, and smart cities. However, how to acquire high-quality three-dimensional (3D) geospatial information from point clouds has become a scientific frontier, for which there is an urgent demand in the fields of surveying and mapping, as well as geoscience applications. To address the challenges mentioned above, point cloud intelligence came into being. This paper summarizes the state-of-the-art of point cloud intelligence, with regard to acquisition equipment, intelligent processing, scientific research, and engineering applications. For this purpose, we refer to a recent project on the hybrid georeferencing of images and LiDAR data for high-quality point cloud collection, as well as a current benchmark for the semantic segmentation of high-resolution 3D point clouds. These projects were conducted at the Institute for Photogrammetry, the University of Stuttgart, which was initially headed by the late Prof. Ackermann. Finally, the development prospects of point cloud intelligence are summarized.

Hybrid simplification algorithm for unorganized point cloud based on two-level fuzzy decision making

Traditional point cloud simplification algorithms have specific application scenarios. If these well-known methods can be accurately applied to different feature regions of a point cloud, high-quality point cloud feature preservation can be realized. In this paper, a hybrid point cloud simplification method, based on two-level fuzzy decision making, is proposed. Specifically, the number of peak points and bin intervals of each dimension density histogram of the point cloud are counted, and the cluster number and initial centers of fuzzy c-means (FCM) clustering method are determined, hence the autonomous FCM clustering method of point clouds is realized. Based on the expert linguistic inference rules of type-1 fuzzy system, its output is used to accurately identify clusters as flat, transitional and drastic type. According to the attributes of the clusters, predefined thresholds or algorithms are applied in each one, achieving high quality point cloud simplification. The experimental results show that the algorithm can effectively determine the number of clusters, improve the speed of FCM clustering algorithm and achieve feature preserving point cloud simplification, after removing 50–90 % of the points.

PU-MFA: Point Cloud Up-Sampling via Multi-Scale Features Attention

Recently, research using point clouds has been increasing with the development of 3D scanner technology. According to this trend, the demand for high-quality point clouds is increasing, but there is still a problem with the high cost of obtaining high-quality point clouds. Therefore, with the recent remarkable development of deep learning, point cloud up-sampling research, which uses deep learning to generate high-quality point clouds from low-quality point clouds, is one of the fields attracting considerable attention. This paper proposes a new point cloud up-sampling method called Point cloud Up-sampling via Multi-scale Features Attention (PU-MFA). Inspired by prior studies that reported good performance at generating high-quality dense point set using the multi-scale features or attention mechanisms, PU-MFA merges the two through a U-Net structure. In addition, PU-MFA adaptively uses multi-scale features to refine the global features effectively. The PU-MFA was compared with other state-of-the-art methods in various evaluation metrics through various experiments using the PU-GAN dataset, which is a synthetic point cloud dataset, and the KITTI dataset, which is the real-scanned point cloud dataset. In various experimental results, PU-MFA showed superior performance of generating high-quality dense point set in quantitative and qualitative evaluation compared to other state-of-the-art methods, proving the effectiveness of the proposed method. The attention map of PU-MFA was also visualized to show the effect of multi-scale features.

Detection of discontinuity pattern and orientation for tunnel faces by using 2D and 3D image analysis techniques

This study proposes a new measurement system based on Structure from Motion (SfM) photogrammetry to evaluate tunnel faces. The main goal is to determine the tunnel face discontinuity patterns and orientations for the tunnel face stability evaluation. A 3D point cloud model is generated from overlapping images of the tunnel face 3D polystyrene model through SfM photogrammetry. The discontinuity patterns and orientations are determined through facet extraction of the K-D tree plugin in CloudCompare. The same sets of overlapping images were analyzed in four different quality settings (low, medium, high, and ultra-high) with various sets of point cloud number that control the accuracy of the discontinuity's measurement. In this study, the high-quality setting shows the consistent measurement of the discontinuity patterns and orientations compared with the real 3D polystyrene tunnel face model. Validation of the pattern results from CloudCompare was carried out using JudGeo. Cluster analysis shows two sets of discontinuity with means of dip and dip direction of 82°/164° and 84°/307°, respectively. Both orientations were manually verified with a geological compass. The high-quality point cloud and manual compass measurement of the orientation highlight the similarity of the concentration for both discontinuity sets for the replica tunnel face. This tunnel face proposed method has the advantages of evaluating inaccessible areas and avoiding professional bias.

The Application of Laser-Scanning-Based BIM Technology in Large Steel Structure Engineering for Environmental Protection

The rise of heteromorphic architecture brings great challenges to engineering design, blanking, construction, completion testing, and maintenance. The 3D laser scanning measurement technology can quickly achieve the “copy” measurement of the target, especially suitable for the digitization of complex structure and the accurate construction of true 3D model and it takes care of other things which are necessary environment point of view. In this paper, we are considering the steel structure inspection and curtain wall blanking in the construction of the Grand Theater in the “Shangqiu Cultural and Art Center” as a case study. First, the laser scanning technology has been introduced to complete the data acquisition of steel structure objects and to screen the other details of the structure. Then, the high-quality point cloud is obtained through multistation splicing, filtering, denoising and smoothing, compression, and simplification. Through field comparison, the point cloud accuracy reached an acceptable level. Through field comparison, it can be seen that the difference between the designed building model combining laser technology and BIM technology is distributed within the range of ±(1–5)mm, and the mean square error is ±3.5 mm, and the deviation of numerical simulation is small, which meets the building requirements. Therefore, this method can effectively detect steel structure, and the BIM model can be updated according to the measured point cloud data. We provide accurate data reference for curtain wall material and installation.

In-field pose estimation of grape clusters with combined point cloud segmentation and geometric analysis

In-field object detection and pose estimation are challenging tasks in industrial harvesting scenarios. This study investigates a method for object detection and 6D pose estimation using a depth camera to prevent harvesting robots from collisions while harvesting grape clusters. First, the Mask Region Convolutional Neural Network (Mask R-CNN) is deployed to segment 2D images and output binary mask images of grape clusters. Second, the grape cluster point cloud is segmented based on the binary mask image and the mapping relationship between the image and point cloud to provide a high-quality point cloud through pre-processing. Third, the optimal cutting point is located by constructing the grape peduncle region of interest (RoI). Finally, the peduncle surface is fit based on the locally weighted scatterplot smoothing (LOWESS) algorithm, and the pose of the peduncle is estimated using geometric methods. The detection results from 238 test images show that the mean Precision (mP) is 86.0%, the mean Recall (mR) is 79.9%, the F1-score is 0.828, and mean Intersection Over Union (mIOU) for instance segmentation is 87.9%. The pose estimation results from 172 grape peduncles yield an error angle of 22.22 degrees ± 17.96 degrees in pose estimation. The detection and pose estimations for each grape cluster require approximately 1.786 s. The demonstrated performance of the proposed method indicates it can be applied to collision-free grape harvesting using robots in unstructured environments.

Point cloud semantic segmentation of complex railway environments using deep learning

Safety of transportation networks is of utmost importance for our society. With the emergency of digitalization, the railway sector is accelerating the automation in inventory and inspection procedures. Mobile mapping systems allow capturing three-dimensional point clouds of the infrastructure in short periods of time. In this paper, a deep learning methodology for semantic segmentation of railway infrastructures is presented. The methodology segments both linear and punctual elements from railway infrastructure, and it is tested in four scenarios: i) 90 km-long railway; ii) 2 km-long low-quality point clouds; iii) 400 m-long high-quality point clouds; iv) 1.4 km-long railway recoded with aerial mapping system. The longest one is used for training and testing, obtaining mean accuracy greater than 90%. The other scenarios are used only for testing, and qualitative results are discussed, proving that the method can be applied to new scenarios that significantly differ in terms of data quality and resolution.

PIMesh: An automatic point cloud and unstructured mesh generation algorithm for meshless methods and finite element analysis-with applications in surgical simulations.

We propose a point cloud and mesh generation algorithm, particle injection mesh generator (PIMesh), that can be used to generate optimized high-quality point clouds and unstructured meshes for domains in any shape with minimum (or even no) user intervention. The domains can be scanned images in OBJ format in 2D and 3D or just a line drawing in 2D. Mesh grading can also be easily controlled. The PIMesh is robust and easy to be implemented and is useful for a variety of applications, ranging from generating point clouds for meshless methods, mesh generation for finite element methods, computer graphics applications and surgical simulators. The core idea of the PIMesh is that a mesh domain is considered as an "airtight container" into which particles are "injected" at one or multiple selected interior points. The motion of the particles is controlled by a pseudo-molecular dynamics (PMD) formulation with a pairwise purely repelling "force" moderated by an absolute velocity dependent drag force. The particles repel each other and occupy the whole domain somewhat like blowing up a balloon. When the container is full of particles and the motion is stopped (the particles can be considered as a point cloud), a Delaunay triangulation algorithm is employed to link the particles together to generate an unstructured mesh. The performance of the PIMesh and the comparison with other unstructured mesh generation approaches are demonstrated through generating node distributions and meshes for several 2D and 3D object domains including a scanned image of bones and others.

MIXED REALITY VISUALIZATION OF POINT CLOUDS FOR SUPPORTING TERRESTRIAL LASER SCANNING

Abstract. 3D point clouds from terrestrial laser scanners (TLS) are used in a variety of fields and applications. To acquire high-quality point clouds that have enough point density, small scanning errors, and no lack of points in important regions, appropriate scan planning, including determination of scanner positions and scan conditions, is required. Currently, planning is supported by knowledge and experience of skilled workers, and it is difficult to ensure the quality of acquired point clouds. In this study, we propose a system for visualization of point clouds to support the acquisition of high-quality point clouds using TLS. The system allows the user to see and check the quality of scanned TLS point clouds and unscanned regions intuitively by superimposing the point clouds onto the real world using a mixed reality (MR) device. In addition, the system supports finding the next best scanner position for additional laser scans based on predicted scan quality visualization to acquire higher-quality points or fill the unscanned regions.

CorAl: Introspection for robust radar and lidar perception in diverse environments using differential entropy

Robust perception is an essential component to enable long-term operation of mobile robots. It depends on failure resilience through reliable sensor data and pre-processing, as well as failure awareness through introspection, for example the ability to self-assess localization performance. This paper presents CorAl: a principled, intuitive, and generalizable method to measure the quality of alignment between pairs of point clouds, which learns to detect alignment errors in a self-supervised manner. CorAl compares the differential entropy in the point clouds separately with the entropy in their union to account for entropy inherent to the scene. By making use of dual entropy measurements, we obtain a quality metric that is highly sensitive to small alignment errors and still generalizes well to unseen environments. In this work, we extend our previous work on lidar-only CorAl to radar data by proposing a two-step filtering technique that produces high-quality point clouds from noisy radar scans. Thus, we target robust perception in two ways: by introducing a method that introspectively assesses alignment quality, and by applying it to an inherently robust sensor modality. We show that our filtering technique combined with CorAl can be applied to the problem of alignment classification, and that it detects small alignment errors in urban settings with up to 98% accuracy, and with up to 96% if trained only in a different environment. Our lidar and radar experiments demonstrate that CorAl outperforms previous methods both on the ETH lidar benchmark, which includes several indoor and outdoor environments, and the large-scale Oxford and MulRan radar data sets for urban traffic scenarios. The results also demonstrate that CorAl generalizes very well across substantially different environments without the need of retraining.

Free-Boundary Conformal Parameterization of Point Clouds

With the advancement in 3D scanning technology, there has been a surge of interest in the use of point clouds in science and engineering. To facilitate the computations and analyses of point clouds, prior works have considered parameterizing them onto some simple planar domains with a fixed boundary shape such as a unit circle or a rectangle. However, the geometry of the fixed shape may lead to some undesirable distortion in the parameterization. It is therefore more natural to consider free-boundary conformal parameterizations of point clouds, which minimize the local geometric distortion of the mapping without constraining the overall shape. In this work, we develop a free-boundary conformal parameterization method for disk-type point clouds, which involves a novel approximation scheme of the point cloud Laplacian with accumulated cotangent weights together with a special treatment at the boundary points. With the aid of the free-boundary conformal parameterization, high-quality point cloud meshing can be easily achieved. Furthermore, we show that using the idea of conformal welding in complex analysis, the point cloud conformal parameterization can be computed in a divide-and-conquer manner. Experimental results are presented to demonstrate the effectiveness of the proposed method.

Smoothly varying projective transformation for line segment matching

Line segment matching is important in applications that require recovering the 3D structure of objects (e.g., manmade objects in street-level scenarios). However, differentiating between true and false line matches is generally difficult without strong geometric constraints for line segments. Hence, additional constraints are forced to be used, sacrificing many true line matches. This study proposes a robust line segment matching method based on global projective transformation modeling. We develop a non-parametric motion regression formulation with a specially designed direct linear transformation-based cost function that reformulates the piecewise smoothly varying projective transformations as a global continuous model from highly noisy point matches. The resultant model can effectively approximate the real underlying image transformation and derive high-quality point matches. We apply the computed model and high-quality point matches to a point-correspondence-based line matching pipeline, which provides sufficient strict geometric constraints for first generating the pair-to-pair matches and then distilling the line-to-line matches. Extensive experiments conducted on two challenging line matching datasets show that the proposed method can obtain considerable correct line segment matches, outperforming the comparison methods in mean F-score by at least 15.5% on the benchmark dataset and 16.9% on the local dataset. Code is available at https://github.com/geovsion/SLEM.

Point Cloud Video Streaming: Challenges and Solutions

Volumetric video (or hologram video), the medium for representing natural content in VR/AR/MR, is presumably the next generation of video technology and a typical use case for 5G and beyond wireless communications. To realize volumetric video applications, efficient volumetric video streaming is in critical demand. This article responds to the challenges of and proposes solutions to wireless transmission systems of point cloud video, which is the most popular and favored way to represent volumetric media and significantly differs from the other types of videos. In particular, we first introduce point cloud video technology and its applications, and then discuss the challenges of and solutions to point cloud video streaming, including encoding, tiling, viewing angle prediction, decoding, quality assessment and transmission optimization. Furthermore, we explain a prototype of a MPEG DASH-based point cloud video streaming system as a preliminary study, along with more simulation results to verify its performance. Finally, we identify future research directions for providing high-quality point cloud video streaming.

A Multiple Level-of-Detail 3D Data Transmission Approach for Low-Latency Remote Visualisation in Teleoperation Tasks

In robotic teleoperation, the knowledge of the state of the remote environment in real time is paramount. Advances in the development of highly accurate 3D cameras able to provide high-quality point clouds appear to be a feasible solution for generating live, up-to-date virtual environments. Unfortunately, the exceptional accuracy and high density of these data represent a burden for communications requiring a large bandwidth affecting setups where the local and remote systems are particularly geographically distant. This paper presents a multiple level-of-detail (LoD) compression strategy for 3D data based on tree-like codification structures capable of compressing a single data frame at multiple resolutions using dynamically configured parameters. The level of compression (resolution) of objects is prioritised based on: (i) placement on the scene; and (ii) the type of object. For the former, classical point cloud fitting and segmentation techniques are implemented; for the latter, user-defined prioritisation is considered. The results obtained are compared using a single LoD (whole-scene) compression technique previously proposed by the authors. Results showed a considerable improvement to the transmitted data size and updated frame rate while maintaining low distortion after decompression.

Comparative Analysis of Different Mobile LiDAR Mapping Systems for Ditch Line Characterization

Maintenance of roadside ditches is important to avoid localized flooding and premature failure of pavements. Scheduling effective preventative maintenance requires a reasonably detailed mapping of the ditch profile to identify areas in need of excavation to remove long-term sediment accumulation. This study utilizes high-resolution, high-quality point clouds collected by mobile LiDAR mapping systems (MLMS) for mapping roadside ditches and performing hydrological analyses. The performance of alternative MLMS units, including an unmanned aerial vehicle, an unmanned ground vehicle, a portable backpack system along with its vehicle-mounted version, a medium-grade wheel-based system, and a high-grade wheel-based system, is evaluated. Point clouds from all the MLMS units are in agreement within the ±3 cm range for solid surfaces and ±7 cm range for vegetated areas along the vertical direction. The portable backpack system that could be carried by a surveyor or mounted on a vehicle is found to be the most cost-effective method for mapping roadside ditches, followed by the medium-grade wheel-based system. Furthermore, a framework for ditch line characterization is proposed and tested using datasets acquired by the medium-grade wheel-based and vehicle-mounted portable systems over a state highway. An existing ground-filtering approach—cloth simulation—is modified to handle variations in point density of mobile LiDAR data. Hydrological analyses, including flow direction and flow accumulation, are applied to extract the drainage network from the digital terrain model (DTM). Cross-sectional/longitudinal profiles of the ditch are automatically extracted from the LiDAR data and visualized in 3D point clouds and 2D images. The slope derived from the LiDAR data turned out to be very close to the highway cross slope design standards of 2% on driving lanes, 4% on shoulders, and a 6-by-1 slope for ditch lines.

Point cloud denoising using non-local collaborative projections

Point cloud is important for object detection and recognition. The main challenge of point cloud denoising is to preserve the geometric structures. Several state-of-the-art point cloud denoising methods focus only on analyzing local geometric information, which is sensitive to noise and outliers. In this paper, we propose a novel point cloud denoising algorithm based on the characteristics of non-local self-similarity. First, we present an adaptive curvature threshold to select the edge points and tune their corresponding normals, which can preserve the sharp details. Then we propose a structure-aware descriptor called projective height vector to capture the local height variations by normal height projection and the most similar non-local projective height vectors are grouped into a height matrix to enhance the structure representation. Moreover, the proposed structure descriptor is invariant with rigid transformation. Finally, an improved weighted nuclear norm minimization is proposed to optimize the height matrix and reconstruct a high-quality point cloud. Rather than treating each singular value independently, each component in our proposed weight definition connects with the most important components to preserve the major structural information. Experiments on synthetic and scanned point cloud datasets demonstrate that our algorithm outperforms state-of-the-art methods in terms of reconstruction accuracy and structure preservation.