Point Cloud Matching Research Articles

Non-Rigid Point Cloud Matching Based on Invariant Structure for Face Deformation

In this paper, we present a non-rigid point cloud matching method based on an invariant structure for face deformation. Our work is guided by the realistic needs of 3D face reconstruction and re-topology, which critically need support for calculating the correspondence between deformable models. Our paper makes three main contributions: First, we propose an approach to normalize the global structure features of expressive faces using texture space properties, which decreases the variation magnitude of facial landmarks. Second, we make a modification to the traditional shape context descriptor to solve the problem of regional cross-mismatch. Third, we collect a dataset with various expressions. Ablation studies and comparative experiments were conducted to investigate the performance of the above work. In face deformable cases, our method achieved 99.89% accuracy on our homemade face dataset, showing superior performance over some other popular algorithms. In this way, it can help modelers to build digital humans more easily based on the estimated correspondence of facial landmarks, saving a lot of manpower and time.

Dynamic detection of three-dimensional crop phenotypes based on a consumer-grade RGB-D camera.

Nondestructive detection of crop phenotypic traits in the field is very important for crop breeding. Ground-based mobile platforms equipped with sensors can efficiently and accurately obtain crop phenotypic traits. In this study, we propose a dynamic 3D data acquisition method in the field suitable for various crops by using a consumer-grade RGB-D camera installed on a ground-based movable platform, which can collect RGB images as well as depth images of crop canopy sequences dynamically. A scale-invariant feature transform (SIFT) operator was used to detect adjacent date frames acquired by the RGB-D camera to calculate the point cloud alignment coarse matching matrix and the displacement distance of adjacent images. The data frames used for point cloud matching were selected according to the calculated displacement distance. Then, the colored ICP (iterative closest point) algorithm was used to determine the fine matching matrix and generate point clouds of the crop row. The clustering method was applied to segment the point cloud of each plant from the crop row point cloud, and 3D phenotypic traits, including plant height, leaf area and projected area of individual plants, were measured. We compared the effects of LIDAR and image-based 3D reconstruction methods, and experiments were carried out on corn, tobacco, cottons and Bletilla striata in the seedling stage. The results show that the measurements of the plant height (R²= 0.9~0.96, RSME = 0.015~0.023 m), leaf area (R²= 0.8~0.86, RSME = 0.0011~0.0041 m 2 ) and projected area (R² = 0.96~0.99) have strong correlations with the manual measurement results. Additionally, 3D reconstruction results with different moving speeds and times throughout the day and in different scenes were also verified. The results show that the method can be applied to dynamic detection with a moving speed up to 0.6m/s and can achieve acceptable detection results in the daytime, as well as at night. Thus, the proposed method can improve the efficiency of individual crop 3D point cloud data extraction with acceptable accuracy, which is a feasible solution for crop seedling 3D phenotyping outdoors.

Monitoring Mining Surface Subsidence with Multi-Temporal Three-Dimensional Unmanned Aerial Vehicle Point Cloud

Long-term and high-intensity coal mining has led to the increasingly serious surface subsidence and environmental problems. Surface subsidence monitoring plays an important role in protecting the ecological environment of the mining area and the sustainable development of modern coal mines. The development of surveying technology has promoted the acquisition of high-resolution terrain data. The combination of an unmanned aerial vehicle (UAV) point cloud and the structure from motion (SfM) method has shown the potential of collecting multi-temporal high-resolution terrain data in complex or inaccessible environments. The difference of the DEM (DoD) is the main method to obtain the surface subsidence in mining areas. However, the obtained digital elevation model (DEM) needs to interpolate the point cloud into the grid, and this process may introduce errors in complex natural topographic environments. Therefore, a complete three-dimensional change analysis is required to quantify the surface change in complex natural terrain. In this study, we propose a quantitative analysis method of ground subsidence based on three-dimensional point cloud. Firstly, the Monte Carlo simulation statistical analysis was adopted to indirectly evaluate the performance of direct georeferencing photogrammetric products. After that, the operation of co-registration was carried out to register the multi-temporal UAV dense matching point cloud. Finally, the model-to-model cloud comparison (M3C2) algorithm was used to quantify the surface change and reveal the spatio-temporal characteristics of surface subsidence. In order to evaluate the proposed method, four periods of multi-temporal UAV photogrammetric data and a period of airborne LiDAR point cloud data were collected in the Yangquan mining area, China, from 2020 to 2022. The 3D precision map of a sparse point cloud generated by Monte Carlo simulation shows that the average precision in X, Y and Z directions is 44.80 mm, 45.22 and 63.60 mm, respectively. The standard deviation range of the M3C2 distance calculated by multi-temporal data in the stable area is 0.13–0.19, indicating the consistency of multi-temporal photogrammetric data of UAV. Compared with DoD, the dynamic moving basin obtained by the M3C2 algorithm based on the 3D point cloud obtained more real surface deformation distribution. This method has high potential in monitoring terrain change in remote areas, and can provide a reference for monitoring similar objects such as landslides.

Point Cloud Registration Using Multiattention Mechanism and Deep Hybrid Features

Due to some unfavorable factors, how to accurately register point clouds is still a challenging task. In this article, an effective point cloud registration network is proposed with multiple attention mechanism and deep hybrid features. For the features obtained with a graph neural network, three attention modules, namely the spatial attention module, channel attention module, and self-geometric attention module, are utilized to mine various areas of regional information. An attention-based feature fusion module, which consists of three consecutive residual blocks, is devised to fuse the features from the three attention modules. Moreover, the capability of the network for correctly matching point clouds is enhanced, by using deep hybrid features to guide the correspondence search and the calculation of matching confidence. Experimental results on several widely used datasets demonstrate the effectiveness of the proposed point cloud registration network.

Edge-Preserving Stereo Matching Using LiDAR Points and Image Line Features

This letter proposes a LiDAR and image line-guided stereo matching method (L2GSM), which combines sparse but high-accuracy LiDAR points and sharp object edges of images to generate accurate and fine-structure point clouds. After extracting depth discontinuity lines on the image by using LiDAR depth information, we propose a trilateral update of cost volume and depth discontinuity lines-aware semi-global matching (SGM) strategies to integrate LiDAR data and depth discontinuity lines into the dense matching algorithm. The experimental results for the indoor and aerial datasets show that our method significantly improves the results of the original SGM and outperforms two state-of-the-art LiDAR constraints’ SGM methods, especially in recovering the 3-D structure of low-textured and depth discontinuity regions. In addition, the 3-D point clouds generated by our proposed method outperform the LiDAR data and dense matching point clouds generated by Metashape and SURE aerial in terms of completeness and edge accuracy.

Research on Modeling Method of Digital Elevation Model for Dense Matching Point Cloud Images in Complex Environments

Research on Modeling Method of Digital Elevation Model for Dense Matching Point Cloud Images in Complex Environments

Learning a Task-Specific Descriptor for Robust Matching of 3D Point Clouds

Existing learning-based point feature descriptors are usually task-agnostic, which pursue describing the individual 3D point clouds as accurate as possible. However, the matching task aims at describing the corresponding points consistently across different 3D point clouds. Therefore these too accurate features may play a counterproductive role due to the inconsistent point feature representations of correspondences caused by the unpredictable noise, partiality, deformation, \etc, in the local geometry. In this paper, we propose to learn a robust task-specific feature descriptor to consistently describe the correct point correspondence under interference. Born with an Encoder and a Dynamic Fusion module, our method EDFNet develops from two aspects. First, we augment the matchability of correspondences by utilizing their repetitive local structure. To this end, a special encoder is designed to exploit two input point clouds jointly for each point descriptor. It not only captures the local geometry of each point in the current point cloud by convolution, but also exploits the repetitive structure from paired point cloud by Transformer. Second, we propose a dynamical fusion module to jointly use different scale features. There is an inevitable struggle between robustness and discriminativeness of the single scale feature. Specifically, the small scale feature is robust since little interference exists in this small receptive field. But it is not sufficiently discriminative as there are many repetitive local structures within a point cloud. Thus the resultant descriptors will lead to many incorrect matches. In contrast, the large scale feature is more discriminative by integrating more neighborhood information. ...

Practical semiautomatic global registration of multiple point clouds based on semidefinite programming

Multiple point clouds registration is a prerequisite step for geometry object observation from multiple viewpoints. Most algorithms depend on pose graph or relative poses, and they make the results ambiguous and inconsistent with the original point clouds. However, the existing algorithms based on original data can almost exclusively be applied to simulated data. In this paper, a practical semiautomatic global registration algorithm is proposed. A objective function is proposed; it utilizes correspondences between point cloud pairs directly to describe this problem, making the algorithm easy to interface with other point cloud matching algorithms. Moreover, it can be effectively approximated by semidefinite programming. Since no automatic algorithm can guarantee the correctness of the correspondences, a manual strategy is proposed to filter the correspondences and find the correlation between point clouds. Based on the correlation, irrelevant point clouds can be removed easily in the point cloud set or divide the set into multiple independent parts. Besides, a postprocessing strategy based on the correlation is proposed for refining the results. The qualitative and quantitative experimental results demonstrate that the approach achieves more accurate and robust performance than previous algorithms.

Indoor and outdoor multi-source 3D data fusion method for ancient buildings

Ancient buildings carry important information, such as ancient politics, economy, culture, customs. However, with the course of time, ancient buildings are often damaged to different degrees, so the restoration of ancient buildings is of great importance from the historical point of view. There are three commonly used non-contact measurement methods, including UAV-based oblique photogrammetry, terrestrial laser scanning, and close-range photogrammetry. These methods can provide integrated three-dimensional surveys of open spaces, indoor and outdoor surfaces for ancient buildings. Theoretically, the combined use of the three measurement methods can provide 3D (three-dimensional) data support for the protection and repair of ancient buildings. However, data from the three methods need to be fused urgently, because if the image data is not used, it will lead to a lack of real and intuitive texture information, and if only image matching point clouds are used, their accuracy will be lower than that of terrestrial laser scanning point clouds, and it will also lead to a lack of digital expression for components with high indoor historical value of ancient buildings. Therefore, in this paper, a data fusion method is proposed to achieve multi-source and multi-scale 3D data fusion of indoor and outdoor surfaces. It takes the terrestrial laser point cloud as the core, and based on fine component texture features and building outline features, respectively, the ground close-range image matching point cloud and UAV oblique image matching point cloud are registered with the terrestrial laser point cloud. This method unifies the data from three measurements in the point cloud and realizes the high-precision fusion of these three data. Based on the indoor and outdoor 3D full-element point cloud formed by the proposed method, it will constitute a visual point cloud model in producing plans, elevations, sections, orthophotos, and other elements for the study of ancient buildings.

Novel Method for Monitoring Mining Subsidence Featuring Co-Registration of UAV LiDAR Data and Photogrammetry

Addressing the problem that traditional methods cannot reliably monitor surface subsidence in coal mining, a novel method has been developed for monitoring subsidence in mining areas using time series unmanned aerial vehicle (UAV) photogrammetry in combination with LiDAR. A dynamic subsidence basin based on the differential digital elevation model (DEM) was constructed and accuracy of the proposed method was verified, with the uncertainty of the DEM of difference (DoD) being quantified via co-registration of a dense matching point cloud of the time series UAV data. The root mean square error calculated for the monitoring points on the subsidence DEM was typically between 0.2 m and 0.3 m with a minimum of 0.17 m. The relative error between the maximum subsidence value of the extracted profile line on the main section after fitting and the measured maximum subsidence value was not more than 20%, and the minimum value was 0.7%. The accuracy of the UAV based method was at the decimeter level, and high accuracy in monitoring the maximum subsidence value was attained, confirming that an innovative strategy for monitoring mining subsidence was realized.

S2-PCM: Super-Resolution Structural Point Cloud Matching for High-Accuracy Video-SAR Image Registration

In this paper, the super-resolution structural point cloud matching (S2-PCM) framework is proposed for video synthetic aperture radar (SAR) inter-frame registration, which consists of a feature recurrence super-resolution network (FRSR-Net), structural point cloud extraction network (SPCE-Net) and robust point matching network (RPM-Net). FRSR-Net is implemented by integrating the feature recurrence structure and residual dense block (RDB) for super-resolution enhancement, SPCE-Net is implemented by training a U-Net with data augmentation, and RPM-Net is applied for robust point cloud matching. Experimental results show that compared with the classical SIFT-like algorithms, S2-PCM achieves higher registration accuracy for video-SAR images under diverse evaluation metrics, such as mutual information (MI), normalized mutual information (NMI), entropy correlation coefficient (ECC), structural similarity (SSIM), etc. The proposed FRSR-Net can significantly improve the quality of video-SAR images and point cloud extraction accuracy. Combining FRSR-Net with S2-PCM, we can obtain higher inter-frame registration accuracy, which is crucial for moving target detection and shadow tracking.

RDC-SLAM: A Real-Time Distributed Cooperative SLAM System Based on 3D LiDAR

To improve the accuracy and efficiency of 3D LiDAR mapping, real-time cooperative SLAM has been considered to explore large and complex areas. To merge the individual maps from multiple robots, it is crucial to identify the common areas and obtain alternative matches between them. However, data transmission, especially in sparse networks with narrow bandwidth and limited range, is a challenging issue for the above problem. Since the distribution manner is suitable for limited communication, we proposed a common framework of 3D real-time distributed cooperative SLAM to fill the community gap. Assuming that each robot can communicate with others, the presented framework consists of four key modules: place recognition, relative pose estimation, distributed graph optimization, and communication. Meanwhile, we developed a complete real-time distributed cooperative SLAM system, called RDC-SLAM, by integrating state-of-the-art components into the framework. For computation and data transmission efficiency, descriptor-based registration is used instead of the conventional point cloud matching. An intensity-based descriptor is developed to perform the place recognition and obtain the alternative matches, while an eigenvalue-based segment descriptor is applied to further refine the relative pose estimations between these alternative matches. A distributed graph optimization method is utilized to obtain the maximum likelihood of multi-trajectory estimation. A communication protocol is also designed to associate data among robots that are easy to deploy and have low network requirements. The RDC-SLAM is validated by real-world experiments and exhibits superior performance concerning accuracy, computation efficiency, and data efficiency.

R-LIO: Rotating Lidar Inertial Odometry and Mapping

In this paper, we propose a novel simultaneous localization and mapping algorithm, R-LIO, which combines rotating multi-line lidar and inertial measurement unit. R-LIO can achieve real-time and high-precision pose estimation and map-building. R-LIO is mainly composed of four sequential modules, namely nonlinear motion distortion compensation module, frame-to-frame point cloud matching module based on normal distribution transformation by self-adaptive grid, frame-to-submap point cloud matching module based on line and surface feature, and loop closure detection module based on submap-to-submap point cloud matching. R-LIO is tested on public datasets and private datasets, and it is compared quantitatively and qualitatively to the four well-known methods. The test results show that R-LIO has a comparable localization accuracy to well-known algorithms as LIO-SAM, FAST-LIO2, and Faster-LIO in non-rotating lidar data. The standard algorithms cannot function normally with rotating lidar data. Compared with non-rotating lidar data, R-LIO can improve localization and mapping accuracy in rotating lidar data.

Test and Analysis of Vegetation Coverage in Open-Pit Phosphate Mining Area around Dianchi Lake Using UAV-VDVI.

This work aimed to detect the vegetation coverage and evaluate the benefits of afforestation and ecological protection. Unmanned aerial vehicle (UAV) aerial survey was adopted to obtain the images of tailings area at Ma’anshan near the Dianchi Lake estuary, so as to construct a high-resolution Digital Orthophoto Map (DOM) and high-density Dense Image Matching (DIM) point cloud. Firstly, the optimal scale was selected for segmentation by considering the terrain. Secondly, the visible-band difference vegetation index (VDVI) of the classified vegetation information of the tail mining area was determined from the index gray histogram, ground class error analysis, and the qualitative and quantitative analysis of the bimodal index. Then, the vegetation information was extracted by combining the random forest (RF) classification algorithm. Finally, the extracted two-dimensional (2D) vegetation information was mapped to the three-dimensional (3D) point cloud, and the redundant data was eliminated. Fractional vegetation cover (FVC) was counted in the way of surface to point and human–machine combination. The experimental results showed that the vegetation information extracted from the 2D image was mapped to the 3D point cloud in the form of surface to point, and the redundant bare ground information was eliminated. The statistical FVC was 36.06%. The field survey suggested that the vegetation information in the turf dam area adjacent to the open phosphate deposit accumulation area research area was sparse. Relevant measures should be taken in the subsequent mining to avoid ecological damage caused by expanded phosphate mining. In general, applying UAV measurement technology and related 2D and 3D products to detect the vegetation coverage in an open phosphate mine area was of practical significance and unique technical advantages.

Scale robust point matching‐Net: End‐to‐end scale point matching using Lie group

Point cloud matching is an important procedure in a variety of computer vision tasks. Traditional point cloud matching methods have made great progress, while neural network-based approaches are becoming a trend, powered by their strong capabilities of feature extraction. Existing point matching neural networks, however, mainly focus on the rigid transformation. More complex transformations should also be considered in many scenarios. In this regard, the authors extend the rigid registration to non-rigid cases and propose a network called the Scale Robust Point Matching (SRPM)-Net for scale point matching. This robust structure-preserving network is implemented by incorporating Lie group parametrisation. It is conducted by Lie group linearisation representation with the constraints of parameters under the corresponding basis of Lie algebra. SRPM-Net preserves the structure of the solution and avoids degeneration. The contributions of this paper lie in two aspects: Most importantly, SRPM-Net provides an extendable framework for handling complicated transformations. Secondly, it introduces a new feature learning module, which better preserves the shape structure by aggregating the high-dimensional feature and calculating the normal vector of point cloud surface automatically. Experimental results show that SRPM-Net is more robust and accurate than existing traditional and recent deep learning methods under various situations.

WPMAVM: Weighted plus-and-minus allowance variance minimization algorithm for solving matching distortion

• A weighted plus-and-minus allowance variance minimization (WPMAVM) algorithm is proposed. • WPMAVM takes the plus-and-minus allowance in the objective function of VMM into account. • The distance function is introduced to suppress the distortion caused by the abnormal allowance. • WPMAVM has better robustness to minus allowance and abnormal point clouds compared to ICP and VMM. • The proposed algorithm is suitable for calibration and measurement of complex curved surfaces. Point cloud matching is widely used in the calibration and measurement of complex curved surfaces, and the matching accuracy affects both the subsequently machined surface quality and measurement results. In the actual application, the existing iterative closest point (ICP) algorithm is prone to matching distortion when the measurement point clouds have inherent defects such as unclosed shape, uneven density and Gaussian noise . Despite this problem can be solved by the variance-minimization matching (VMM) algorithm, but the point clouds match distortedly both for ICP and VMM particularly when the workpieces behave with complex and unknown allowance distribution. For examples, the car bodywork is characterized by remarkable flexibility with local deformation, resulting in abnormal allowance, while the machined engine blade shows over- and less-cutting induced minus and abnormal allowances in high and low curvature areas, respectively. In this paper, a weighted plus-and-minus allowance variance minimization (WPMAVM) algorithm is proposed to overcome these problems. Specifically, the plus-and-minus allowance in the objective function of VMM algorithm is subdivided into the sum of plus-and-minus allowance variance minimization to suppress the distortion caused by the plus-and-minus allowance. The weighted term w , defined as the distance function from the point to the weighted plus-and-minus mean, is introduced to suppress the distortion caused by the abnormal allowance. The simulation and experimental comparisons on three typical workpieces show that the proposed algorithm has better robustness to minus allowance and abnormal point clouds compared to the existing ICP and VMM algorithms.

Research on motion distortion optimization algorithm of laser SLAM

Aiming at the problems of inaccurate laser point cloud matching, large reference pose error and insufficient utilization of sensor observation data during 2D laser SLAM positioning and mapping, which lead to the motion distortion of lidar, a piecewise linear interpolation based method is proposed. Fusion of radar motion distortion optimization strategy and PL-ICP algorithm. The optimization algorithm first uses the piecewise linear interpolation method to fuse the data of the wheel odometer and the lidar, solve the robot pose corresponding to each laser point in the current frame of laser data, then convert all the laser points to the same coordinate system according to the solved pose, and finally the laser data scan matching is performed by the PL-ICP algorithm. In the experiment, the ICP and PL-ICP algorithms were compared for the radar data, and the SLAM algorithm equipped with the motion distortion optimization module was used for localization and mapping. The experimental results show that the motion distortion optimization algorithm effectively removes the motion distortion of low-precision lidar and improves the accuracy of mobile robot positioning and mapping.

Markerless Point Cloud Matching Algorithm Based on 3D Feature Extraction

Since 3D scanners can only scan in a limited range and the scanning process is prone to occlusion and other problems, a complete 3D model cannot be obtained from a point cloud scanning result. Therefore, point cloud matching is necessary for most 3D scanning projects. We propose a point cloud registration algorithm based on a comprehensive approach, and the research contents include a new curvature-based point cloud feature extraction method, a three-dimensional spatial structure classifier of feature points, and an unmarked point cloud-matching algorithm based on three-dimensional feature extraction. The experiments are based on the simulation data and real data to verify the algorithm and evaluate the accuracy. The experimental results show that the matching accuracy reaches the millimetre level, and the fully automated and high-precision label-free point cloud matching based on 3D features is realized, which can provide innovative and breakthrough help for 3D reconstruction.

Learning Dense Features for Point Cloud Registration Using a Graph Attention Network

Point cloud registration is a fundamental task in many applications such as localization, mapping, tracking, and reconstruction. Successful registration relies on extracting robust and discriminative geometric features. Though existing learning-based methods require high computing capacity for processing a large number of raw points at the same time, computational capacity limitation is not an issue thanks to powerful parallel computing process using GPU. In this paper, we introduce a framework that efficiently and economically extracts dense features using a graph attention network for point cloud matching and registration (DFGAT). The detector of the DFGAT is responsible for finding highly reliable key points in large raw data sets. The descriptor of the DFGAT takes these keypoints combined with their neighbors to extract invariant density features in preparation for the matching. The graph attention network (GAT) uses the attention mechanism that enriches the relationships between point clouds. Finally, we consider this as an optimal transport problem and use the Sinkhorn algorithm to find positive and negative matches. We perform thorough tests on the KITTI dataset and evaluate the effectiveness of this approach. The results show that this method with the efficiently compact keypoint selection and description can achieve the best performance matching metrics and reach the highest success ratio of 99.88% registration in comparison with other state-of-the-art approaches.

A Meshless Numerical Modeling Method for Fractured Reservoirs Based on Extended Finite Volume Method

Summary In this paper, a meshless numerical modeling method named mesh-free discrete fracture model (MFDFM) of fractured reservoirs based on the newly developed extended finite volume method (EFVM) is proposed. First, matching and nonmatching point cloud generation algorithms are developed to discretize the reservoir domain with fracture networks, which avoid the gridding challenges of the reservoir domain in traditional mesh-based methods. Then, taking oil/water two-phase flow in fractured reservoirs as an example, MFDFM derives the EFVM discrete scheme of the governing equations, constructs various types of connections between matrix nodes and fracture nodes, and calculates the corresponding transmissibilities. Finally, the EFVM discrete scheme of the governing equations and the generalized finite difference discrete scheme of various boundary conditions form the global nonlinear equations, which do not increase the degree of nonlinearity compared with those in the traditional finite volume method (FVM)-based numerical simulator. The global equations can be solved by the existing nonlinear solver in the FVM-based reservoir numerical simulator by only adding the linear discrete equations of boundary conditions, which reduce the difficulty of forming a general purpose MFDFM-based fractured reservoir numerical simulator. Several numerical test cases are implemented to illustrate that the proposed MFDFM can achieve good computational performance under matching and nonmatching point clouds, and for heterogeneous reservoirs, complex fracture networks, complex boundary geometry, and complex boundary conditions, by comparing the computational results of MFDFM with embedded discrete fracture model (EDFM). Thus, MFDFM retains the computational performances of the traditional mesh-based methods and can avoid the difficulties of handling complex geometry and complex boundary conditions of the computational domain, which is the first meshless numerical framework to model fractured reservoirs in parallel with the mesh-based discrete fracture model (DFM) and EDFM.