Alignment Of Point Clouds Research Articles

A tracker pose optimization method for robotic measuring system based on spatial distance constraints

In large-scale metrology (LSM), the transformation of the tracker base frame (TBF) is a predominant method to enlarge the field of view (FOV) of the tracking sensor for full-field 3D measurements. Nevertheless, such a process will introduce cumulative errors and significantly diminish the global point cloud alignment accuracy. To address this problem, we propose a novel tracker pose optimization method for TBF transformation. A pose graph optimization (PGO) model based on spatial distance constraints is implemented to improve the tracker pose accuracy. We also adopt a robust coefficient and a damping factor to simplify the experimental process and stabilize the convergence results. Simulations and experiments on high-speed train surfaces are conducted to validate our method’s accuracy and effectiveness. The results indicate that our optimization method outperforms two existing methods in spatial positioning accuracy and point cloud alignment accuracy, which showcases its practical applicability and superiority in manufacturing scenarios.

Fusing LiDAR and Photogrammetry for Accurate 3D Data: A Hybrid Approach

The fusion of LiDAR and photogrammetry point clouds is a necessary advancement in 3D-modeling, enabling more comprehensive and accurate representations of physical environments. The main contribution of this paper is the development of an innovative fusion system that combines classical algorithms, such as Structure from Motion (SfM), with advanced machine learning techniques, like Coherent Point Drift (CPD) and Feature-Metric Registration (FMR), to improve point cloud alignment and fusion. Experimental results, using a custom dataset of real-world scenes, demonstrate that the hybrid fusion method achieves an average error of less than 5% in the measurements of small reconstructed objects, with large objects showing less than 2% deviation from real sizes. The fusion process significantly improved structural continuity, reducing artifacts like edge misalignments. The k-nearest neighbors (kNN) analysis showed high reconstruction accuracy for the hybrid approach, demonstrating that the hybrid fusion system, particularly when combining machine learning-based refinement with traditional alignment methods, provides a notable advancement in both geometric accuracy and computational efficiency for real-time 3D-modeling applications.

Analysis of 3D Modeling Using LiDAR Velodyne for Tree Inventory Using the LIO-SAM Method

Abstract LiDAR technology has become increasingly popular for tree inventory, particularly when combined with techniques like LiDAR Inertial Odometry and Mapping, or LIO-SAM. LIO-SAM method of 3D modeling analysis for tree inventory using LiDAR is the main topic of this study. Planting trees is one way to lower the amount of carbon in the atmosphere, but doing so requires precise assessments of the trees’ architecture. LiDAR technology overcomes the spatial resolution constraints of satellite photography to enable very accurate three-dimensional modeling of real things. Since tiny LiDAR devices like TLS and Backpack are quite costly, this method was created as a workaround. The Ublox F9R GNSS receiver, Pixhawk orange IMU sensor, and Velodyne vlp 16 LiDAR sensor are the sensors that are used. Accurate point cloud creation is achieved via the LIO-SAM technique, which is subsequently converted into a 3D model of the trees. The precise measurement of tree dimensions required for numerous environmental and forest management applications is made possible by this research’s use of LiDAR. The outcomes demonstrate that the LIO-SAM approach offers excellent 3D modeling accuracy with a minimal average point cloud alignment error. The findings of georeferencing have an RMSE of less than one meter. Furthermore, a robust association has been shown between ground-truth data and 3D modeling outcomes obtained using this approach. The findings of this study can help with efforts to mitigate climate change and improve the management of forests.

LIO-SAM++: A Lidar-Inertial Semantic SLAM with Association Optimization and Keyframe Selection.

Current lidar-inertial SLAM algorithms mainly rely on the geometric features of the lidar for point cloud alignment. The issue of incorrect feature association arises because the matching process is susceptible to influences such as dynamic objects, occlusion, and environmental changes. To address this issue, we present a lidar-inertial SLAM system based on the LIO-SAM framework, combining semantic and geometric constraints for association optimization and keyframe selection. Specifically, we mitigate the impact of erroneous matching points on pose estimation by comparing the consistency of normal vectors in the surrounding region. Additionally, we incorporate semantic information to establish semantic constraints, further enhancing matching accuracy. Furthermore, we propose an adaptive selection strategy based on semantic differences between frames to improve the reliability of keyframe generation. Experimental results on the KITTI dataset indicate that, compared to other systems, the accuracy of the pose estimation has significantly improved.

Proposal of UAV-SLAM-Based 3D Point Cloud Map Generation Method for Orchards Measurements

This paper proposes a method for generating highly accurate point cloud maps of orchards using an unmanned aerial vehicle (UAV) equipped with light detection and ranging (LiDAR). The point cloud captured by the UAV-LiDAR was converted to a geographic coordinate system using a global navigation satellite system / inertial measurement unit (GNSS/IMU). The converted point cloud is then aligned with the simultaneous localization and mapping (SLAM) technique. As a result, a 3D model of an orchard is generated in a low-cost and easy-to-use manner for pesticide application with precision. The method of direct point cloud alignment with real-time kinematic-global navigation satellite system (RTK-GNSS) had a root mean square error (RMSE) of 42 cm between the predicted and true crop height values, primarily due to the effects of GNSS multipath and vibration of automated vehicles. Contrastingly, our method demonstrated better results, with RMSE of 5.43 cm and 2.14 cm in the vertical and horizontal axes, respectively. The proposed method for predicting crop location successfully achieved the required accuracy of less than 1 m with errors not exceeding 30 cm in the geographic coordinate system.

Full Transformer Framework for Robust Point Cloud Registration With Deep Information Interaction.

Point cloud registration is an essential technology in computer vision and robotics. Recently, transformer-based methods have achieved advanced performance in point cloud registration by utilizing the advantages of the transformer in order-invariance and modeling dependencies to aggregate information. However, they still suffer from indistinct feature extraction, sensitivity to noise, and outliers, owing to three major limitations: 1) the adoption of CNNs fails to model global relations due to their local receptive fields, resulting in extracted features susceptible to noise; 2) the shallow-wide architecture of transformers and the lack of positional information lead to indistinct feature extraction due to inefficient information interaction; and 3) the insufficient consideration of geometrical compatibility leads to the ambiguous identification of incorrect correspondences. To address the above-mentioned limitations, a novel full transformer network for point cloud registration is proposed, named the deep interaction transformer (DIT), which incorporates: 1) a point cloud structure extractor (PSE) to retrieve structural information and model global relations with the local feature integrator (LFI) and transformer encoders; 2) a deep-narrow point feature transformer (PFT) to facilitate deep information interaction across a pair of point clouds with positional information, such that transformers establish comprehensive associations and directly learn the relative position between points; and 3) a geometric matching-based correspondence confidence evaluation (GMCCE) method to measure spatial consistency and estimate correspondence confidence by the designed triangulated descriptor. Extensive experiments on the ModelNet40, ScanObjectNN, and 3DMatch datasets demonstrate that our method is capable of precisely aligning point clouds, consequently, achieving superior performance compared with state-of-the-art methods. The code is publicly available at https://github.com/CGuangyan-BIT/DIT.

WildScenes: A benchmark for 2D and 3D semantic segmentation in large-scale natural environments

Recent progress in semantic scene understanding has primarily been enabled by the availability of semantically annotated bi-modal (camera and LiDAR) datasets in urban environments. However, such annotated datasets are also needed for natural, unstructured environments to enable semantic perception for applications, including conservation, search and rescue, environment monitoring, and agricultural automation. Therefore, we introduce WildScenes, a bi-modal benchmark dataset consisting of multiple large-scale, sequential traversals in natural environments, including semantic annotations in high-resolution 2D images and dense 3D LiDAR point clouds, and accurate 6-DoF pose information. The data is (1) trajectory-centric with accurate localization and globally aligned point clouds, (2) calibrated and synchronized to support bi-modal training and inference, and (3) containing different natural environments over 6 months to support research on domain adaptation. Our 3D semantic labels are obtained via an efficient, automated process that transfers the human-annotated 2D labels from multiple views into 3D point cloud sequences, thus circumventing the need for expensive and time-consuming human annotation in 3D. We introduce benchmarks on 2D and 3D semantic segmentation and evaluate a variety of recent deep-learning techniques to demonstrate the challenges in semantic segmentation in natural environments. We propose train-val-test splits for standard benchmarks as well as domain adaptation benchmarks and utilize an automated split generation technique to ensure the balance of class label distributions. The WildScenes benchmark webpage is https://csiro-robotics.github.io/WildScenes , and the data is publicly available at https://data.csiro.au/collection/csiro:61541 .

Motion transformation solutions based on Euler angle perturbation model

This paper presents two novel solutions to compute motion transformation by effectively addressing the nonlinear constraints formed by observational data. These solutions have broad applicability in algorithms involving rotation calculation, such as autonomous positioning systems. Instead of the commonly used Lie algebra-based solutions, the two novel solutions are based on the Euler angle perturbation model, which circumvent the problem arising from the singularity of Euler angles. The first solution is a forward compositional algorithm, which emphasizes accelerated iterative processes. The second solution is an inverse compositional algorithm that aims to minimize computational complexity. Both algorithms offer accelerated iteration by employing efficient Jacobian computation and parameter updates. Furthermore, we have developed an odometry algorithm and a point cloud alignment program to assess the performance of these different solutions. The experimental results demonstrate that the two novel solutions achieve an equivalent global minimum while also reducing computational load.

Exploration on the Teaching Reform of Cultural and Creative Product Design Based on the Cultivation of Art and Labour Integration Talents

The teaching mode of art-industry integration talent cultivation is a project-driven design education mode based on the combination of art and skill constructed. The research in this paper explores the creative practice of cultural and creative product design, takes the effectiveness of the combination of art and craft and practice as the evaluation basis, and proposes a PCA point cloud alignment algorithm based on feature matching, so as to realize the innovation of teaching cultural and creative product design. The normalization of point cloud coordinates is achieved by PCA, and the current state prediction and covariance are obtained based on the normalized results. The results show that in the demand analysis of cultural and creative product design, the Kano value of students’ knowledge development demand reaches above 0.703. It shows that in the art-industry fusion talent cultivation mode, the effectiveness of practical teaching should be taken as the research acceptance standard, so as to explore a new mode and a new path about the cultivation of talents of cultural and creative design in colleges and universities.

基于扫描上下文和 NDT-ICP 融合方案的果园同步定位与绘图方法研究

Simultaneous localization and mapping (SLAM) is one of the key technologies for agricultural robots to build maps and localize in complex orchard environments and realize unmanned autonomous operations. Due to the complexity of the orchard environment, the single canopy feature and the diffuse reflection of light caused by the leaves, etc., the map construction process of the orchard environment leads to mismatch and increases the cumulative error of the map construction. Aiming at the above problems, this paper propose a navigation map construction method for orchard environment based on the fusion of Scan Context and NDT-ICP. The method firstly searches the Ring key quickly to get the candidate frames, and scores the similarity between the candidate frames and the current frame, and effectively detects the loopbacks by two-stage searching algorithm to reduce the false matches in the map of orchard environment. Meanwhile, a point cloud alignment method based on the fusion of normal distribution transform coarse alignment and iterative nearest point exact alignment is used to reduce the cumulative error of the orchard environment map. The results show that the improved algorithm compensates the drift of the point cloud map with higher mapping accuracy, better real-time performance, lower resource utilization, higher overlap between the trajectory estimation and the real trajectory, smoother loops, and a 4% reduction in CPU occupancy. In the complex orchard environment, the root mean square error and standard deviation of the trajectories of this paper's algorithm are 0.57 m and 0.19 m, which are 68% and 83% higher than those of the loop detection algorithms in the Lightweight Ground Optimized Lidar Trajectory Measurement and Multivariate Terrain Mapping (LeGO-LOAM), respectively. Accurate map construction and low drift pose estimation can be performed.The research algorithm effectively reduces the influence of mis-matching and large cumulative error in the process of map construction in the orchard environment, meets the demand for high-precision environmental mapping in the orchard environment, and provides technical support for promoting unmanned operation in the orchard environment.

GNSS/LiDAR/IMU Fusion Odometry Based on Tightly-Coupled Nonlinear Observer in Orchard

High-repetitive features in unstructured environments and frequent signal loss of the Global Navigation Satellite System (GNSS) severely limits the development of autonomous robot localization in orchard settings. To address this issue, we propose a LiDAR-based odometry pipeline GLIO, inspired by KISS-ICP and DLIO. GLIO is based on a nonlinear observer with strong global convergence, effectively fusing sensor data from GNSS, IMU, and LiDAR. This approach allows for many potentially interfering and inaccessible relative and absolute measurements, ensuring accurate and robust 6-degree-of-freedom motion estimation in orchard environments. In this framework, GNSS measurements are treated as absolute observation constraints. These measurements are tightly coupled in the prior optimization and scan-to-map stage. During the scan-to-map stage, a novel point-to-point ICP registration with no parameter adjustment is introduced to enhance the point cloud alignment accuracy and improve the robustness of the nonlinear observer. Furthermore, a GNSS health check mechanism, based on the robot’s moving distance, is employed to filter reliable GNSS measurements to prevent odometry crashed by sensor failure. Extensive experiments using multiple public benchmarks and self-collected datasets demonstrate that our approach is comparable to state-of-the-art algorithms and exhibits superior localization capabilities in unstructured environments, achieving an absolute translation error of 0.068 m and an absolute rotation error of 0.856°.

YOLOTree-Individual Tree Spatial Positioning and Crown Volume Calculation Using UAV-RGB Imagery and LiDAR Data

Individual tree canopy extraction plays an important role in downstream studies such as plant phenotyping, panoptic segmentation and growth monitoring. Canopy volume calculation is an essential part of these studies. However, existing volume calculation methods based on LiDAR or based on UAV-RGB imagery cannot balance accuracy and real-time performance. Thus, we propose a two-step individual tree volumetric modeling method: first, we use RGB remote sensing images to obtain the crown volume information, and then we use spatially aligned point cloud data to obtain the height information to automate the calculation of the crown volume. After introducing the point cloud information, our method outperforms the RGB image-only based method in 62.5% of the volumetric accuracy. The AbsoluteError of tree crown volume is decreased by 8.304. Compared with the traditional 2.5D volume calculation method using cloud point data only, the proposed method is decreased by 93.306. Our method also achieves fast extraction of vegetation over a large area. Moreover, the proposed YOLOTree model is more comprehensive than the existing YOLO series in tree detection, with 0.81% improvement in precision, and ranks second in the whole series for mAP50-95 metrics. We sample and open-source the TreeLD dataset to contribute to research migration.

A Registration Method Based on Planar Features Between BIM Model and Point Cloud

Abstract In the digitization process of the construction industry, it is frequently necessary to use BIM as a digital model carrier, and the registration between BIM and point cloud is a crucial step in BIM applications. Most existing 3D registration methods, such as the ICP algorithm, are capable of aligning point clouds. However, these classical methods are subject to the influence of numerous points contained within the point cloud, relying heavily on point-to-point correlations. Consequently, it is challenging to extend these algorithms to register with other forms of spatial information, such as BIM models, beyond the point cloud. Considering that planes are basic geometric elements in building BIM models, this paper proposes a novel method for aligning BIM models with point clouds by matching planar features. The method extracts planes from point clouds using an enhanced region growing algorithm, directly parsing the planar geometric information from BIM in IFC format. After completing the matching of plane groups, the optimal solution is calculated using a weighted least squares method. The experimental results indicate that the proposed method can achieve successful registration between the BIM model and point cloud, with a lower RMSE of 5.73mm compared to the registration method using RANSAC+ICP on the same dataset.

Advancing 3D point cloud understanding through deep transfer learning: A comprehensive survey

The 3D point cloud (3DPC) has significantly evolved and benefited from the advances of deep learning (DL). However, the latter faces various issues, including the lack of data or annotated data, the existence of a significant gap between training data and test data, and the requirement for high computational resources. To that end, deep transfer learning (DTL), which decreases dependency and costs by utilizing knowledge gained from a source data/task in training a target data/task, has been widely investigated. Numerous DTL frameworks have been suggested for aligning point clouds obtained from several scans of the same scene. Additionally, DA, which is a subset of DTL, has been modified to enhance the point cloud data’s quality by dealing with noise and missing points. Ultimately, fine-tuning and DA approaches have demonstrated their effectiveness in addressing the distinct difficulties inherent in point cloud data. This paper presents the first review shedding light on this aspect. it provides a comprehensive overview of the latest techniques for understanding 3DPC using DTL and domain adaptation (DA). Accordingly, DTL’s background is first presented along with the datasets and evaluation metrics. A well-defined taxonomy is introduced, and detailed comparisons are presented, considering different aspects such as different knowledge transfer strategies, and performance. The paper covers various applications, such as 3DPC object detection, semantic labeling, segmentation, classification, registration, downsampling/upsampling, and denoising. Furthermore, the article discusses the advantages and limitations of the presented frameworks, identifies open challenges, and suggests potential research directions.

A Hybrid Improved SAC-IA with a KD-ICP Algorithm for Local Point Cloud Alignment Optimization

To overcome incomplete point cloud data obtained from laser scanners scanning complex surfaces, multi-viewpoint cloud data needs to be aligned for use. A hybrid improved SAC-IA with a KD-ICP algorithm is proposed for local point cloud alignment optimization. The scanned point cloud data is preprocessed with statistical filtering, as well as uniform down-sampling. The sampling consistency initial alignment (SAC-IA) algorithm is improved by introducing a dissimilarity vector for point cloud initial alignment. In addition, the iterative closest point (ICP) algorithm is improved by incorporating bidirectional KD-tree to form the KD-ICP algorithm for fine point cloud alignment. Finally, the algorithms are compared in terms of runtime and alignment accuracy. The implementation of the algorithms is based on the Visual Studio 2013 software configurating point cloud library environment for testing experiments and practical experiments. The overall alignment method can be 40%~50% faster in terms of running speed. The improved SAC-IA algorithm provides better transformed poses, combined with the KD-ICP algorithm to select the corresponding nearest neighbor pairs, which improves the accuracy, as well as the applicability of the alignment.

LiDAR Point Cloud Super-Resolution Reconstruction Based on Point Cloud Weighted Fusion Algorithm of Improved RANSAC and Reciprocal Distance

This paper proposes a point-by-point weighted fusion algorithm based on an improved random sample consensus (RANSAC) and inverse distance weighting to address the issue of low-resolution point cloud data obtained from light detection and ranging (LiDAR) sensors and single technologies. By fusing low-resolution point clouds with higher-resolution point clouds at the data level, the algorithm generates high-resolution point clouds, achieving the super-resolution reconstruction of lidar point clouds. This method effectively reduces noise in the higher-resolution point clouds while preserving the structure of the low-resolution point clouds, ensuring that the semantic information of the generated high-resolution point clouds remains consistent with that of the low-resolution point clouds. Specifically, the algorithm constructs a K-d tree using the low-resolution point cloud to perform a nearest neighbor search, establishing the correspondence between the low-resolution and higher-resolution point clouds. Next, the improved RANSAC algorithm is employed for point cloud alignment, and inverse distance weighting is used for point-by-point weighted fusion, ultimately yielding the high-resolution point cloud. The experimental results demonstrate that the proposed point cloud super-resolution reconstruction method outperforms other methods across various metrics. Notably, it reduces the Chamfer Distance (CD) metric by 0.49 and 0.29 and improves the Precision metric by 7.75% and 4.47%, respectively, compared to two other methods.

A Review of Point Cloud Alignment Methods and Their Applications

Point cloud alignment is a key technique in the field of computer vision, which involves estimating the transformation between two-point clouds. With the development of optimization methods and deep learning techniques, the robustness and efficiency of point cloud alignment have been significantly improved. Recent studies have combined these two methods to further optimize the performance. Meanwhile, advances in 3D sensing and 3D reconstruction techniques have given rise to the new research field of cross-source point cloud alignment. This paper reviews recent advances in point cloud alignment, including both homologous and cross-source alignment techniques, and explores the interconnection of optimization and deep learning techniques. In addition, the paper reviews relevant benchmark datasets and explores their applications in different domains. Finally, future research directions for point cloud alignment are also described.

Robust colored point cloud alignment based on L*a*b* guided and Cauchy kernel

AbstractPrecision agriculture benefits from point set registration, which can monitor plant health and growth in real time, promote the precise application of fertilizers and pesticides, and provide technical support for achieving sustainable development of agriculture. In this work, we propose a robust point set registration method for precision agriculture based on L*a*b* color guidance, bidirectional search and Cauchy distribution. First, the L*a*b* color guidance is applied to establish accurate correspondences between agricultural RGB‐D data. Second, the bidirectional nearest neighbor search strategy between point sets improves the reliability of establishing correspondences and broadens the convergence domain of the algorithm. Third, Cauchy distribution is utilized as an energy function for noise suppression, which further improves the robustness of the algorithm in dealing with complex vegetation scenes. Finally, results of ablation and simulation experiments indicate that the proposed registration algorithm can achieve more accurate and robust alignment results than other classic and state‐of‐the‐art point cloud registration algorithms to achieve monitoring and comparison of plant growth.

Multiple organ segmentation framework for brain metastasis radiotherapy

Radiotherapy is a preferred treatment for brain metastases, which kills cancer cells via high doses of radiation meanwhile hardly avoiding damage to surrounding healthy cells. Therefore, the delineation of organs-at-risk (OARs) is vital in treatment planning to minimize radiation-induced toxicity. However, the following aspects make OAR delineation a challenging task: extremely imbalanced organ sizes, ambiguous boundaries, and complex anatomical structures. To alleviate these challenges, we imitate how specialized clinicians delineate OARs and present a novel cascaded multi-OAR segmentation framework, called OAR-SegNet. OAR-SegNet comprises two distinct levels of segmentation networks: an Anatomical-Prior-Guided network (APG-Net) and a Point-Cloud-Guided network (PCG-Net). Specifically, APG-Net handles segmentation for all organs, where multi-view segmentation modules and a deep prior loss are designed under the guidance of prior knowledge. After APG-Net, PCG-Net refines small organs through the mini-segmentation and the point-cloud alignment heads. The mini-segmentation head is further equipped with the deep prior feature. Extensive experiments were conducted to demonstrate the superior performance of the proposed method compared to other state-of-the-art medical segmentation methods.

Research on 3D point cloud alignment algorithm based on SHOT features.

To overcome the problem of the high initial position of the point cloud required by the traditional Iterative Closest Point (ICP) algorithm, in this paper, we propose a point cloud registration method based on normal vector and directional histogram features (SHOT). Firstly, a hybrid filtering method based on the voxel idea is proposed and verified using the measured point cloud data, and the noise removal rates of 97.5%, 97.8%, and 93.8% are obtained. Secondly, in terms of feature point extraction, the original algorithm is optimized, and the optimized algorithm can better extract the missing part of the point cloud. Finally, a fine alignment method based on normal vector and directional histogram features (SHOT) is proposed, and the improved algorithm is compared with the existing algorithm. Taking the Stanford University point cloud data and the self-measured point cloud data as examples, the plotted iteration-error plots can be concluded that the improved method can reduce the number of iterations by 40.23% and 37.62%, respectively.