Point Cloud Matching Research Articles

GETr: A Geometric Equivariant Transformer for Point Cloud Registration

AbstractAs a fundamental problem in computer vision, 3D point cloud registration (PCR) aims to seek the optimal transformation to align point cloud pairs. Meanwhile, the equivariance lies at the core of matching point clouds at arbitrary pose. In this paper, we propose GETr, a geometric equivariant transformer for PCR. By learning the point‐wise orientations, we decouple the coordinate to the pose of the point clouds, which is the key to achieve equivariance in our framework. Then we utilize attention mechanism to learn the geometric features for superpoints matching, the proposed novel self‐attention mechanism encodes the geometric information of point clouds. Finally, the coarse‐to‐fine manner is used to obtain high‐quality correspondence for registration. Extensive experiments on both indoor and outdoor benchmarks demonstrate that our method outperforms various existing state‐of‐the‐art methods.

Adaptive point cloud thinning method based on point cloud saliency factor

Abstract Point cloud thinning is an important data pre-processing method for saving computing resources and improving accuracy of point cloud matching. Aiming at the problems of insufficient feature extraction, poor environmental adaptation, and high computational complexity in existing methods, this study proposes a simplified method based on point cloud salient factors. This method first preprocesses the point cloud data, then calculates the significant factors of the point cloud, and uses the local curvature variance to perform adaptive region division. In the local area, the improved feature farthest point sampling (CIFPS) algorithm is used to classify the point cloud. Perform thinning and finally obtain a simplified point cloud. In order to verify the effectiveness of the method, we conducted a large number of experiments on our own experimental platform and public data sets, and compared it with several related point cloud simplification methods. Experimental results show that the average thinning time of our algorithm on 16-line radar data is 23.67ms, 32-line radar is 50.74ms, and 64-line radar is 87.28ms; the point cloud matching error is relative to that based on farthest point sampling and voxel Sampling and Laplacian sampling were reduced by 44.3%, 46.3%, and 23.7% respectively.

Extending geodetic networks for geo-monitoring by supervised point cloud matching

Abstract In this paper we propose a monitoring method that allows the inclusion of point clouds into a geodetic monitoring network. Consequently, network adjustment and a rigorous deformation analysis can be performed allowing consistent error propagation and trustful significance calculation. We introduce a supervised pipeline based on ICP-matching of small-scale laser scan patches. It is specially designed for geo-monitoring applications and allows to improve the spatial resolution as well as the network geometry of monitoring networks in inaccessible hazardous areas in the mountains, where the installation of a sufficient number of targets is difficult. We apply our method to two datasets. The first is a monitoring setup in the laboratory, where we establish the parameters for the supervised patch selection and demonstrate how the network geometry is improved. Second is the real case study of Mt. Hochvogel, where the proposed method helps to clearly improve the spatial resolution of deformation vectors.

Optimize data association of point cloud to improve the quality of mapping and positioning

Purpose Accurate mapping is crucial for the positioning and navigation of mobile robots. Recent advancements in algorithms and the accuracy of LiDAR sensors have led to a gradual improvement in map quality. However, challenges such as lag in closing loops and vignetting at map boundaries persist due to the discrete and sparse nature of raster map data. The purpose of this study is to reduce the error of map construction and improve the timeliness of closed loop. Design/methodology/approach In this letter, the authors introduce a method for dynamically adjusting point cloud distance constraints to optimize data association (ODA-d), effectively addressing these issues. The authors propose a dynamic threshold optimization method for matching point clouds to submaps during scan matching. Findings Large deviations in LiDAR sensor point cloud data, when incorporated into the submap, can result in irreparable errors in correlation matching and loop closure optimization. By implementing a data association framework with double constraints and dynamically adjusting the matching threshold, the authors significantly enhance submap quality. In addition, the authors introduce a dynamic fusion method that accounts for both submap size and the distance between submaps during the mapping process. ODA-d reduces errors between submaps and facilitates timely loop closure optimization. Originality/value The authors validate the localization accuracy of ODA-d by examining translation and rotation errors across three open data sets. Moreover, the authors compare the quality of map construction in a real-world environment, demonstrating the effectiveness of ODA-d.

Two-stage graph matching point cloud registration method based on graph attention network

Two-stage graph matching point cloud registration method based on graph attention network

Digital image correlation calculation method for RGB-D camera multi-view matching using variable template

Multi-view matching is an indispensable method for digital image correlation 3D reconstruction. However, as the camera viewing angle changes, traditional DIC methods can lead to the accumulation of matching errors and an increase in mismatches, thereby affecting the 3D reconstruction quality. This paper proposes a digital image correlation method for multi-view RGB-D images based on variable template matching. Initial positioning is achieved by dynamically adjusting the size and sampling direction of the variable template. Sub-pixel registration is then performed using the inverse compositional algorithm, followed by multi-view point cloud fusion using the Iterative Closest Point (ICP) registration method. Experimental results show that the iteration speed of point cloud registration is increased by 30%, and the cumulative error is reduced by 62.5%. These improvements demonstrate that the proposed method is suitable for multi-viewpoint matching scenarios, achieving excellent results in multi-view feature point matching and multi-view point cloud fusion, significantly enhancing 3D reconstruction accuracy and efficiency.

Graph-based adaptive weighted fusion SLAM using multimodal data in complex underground spaces

Accurate and robust simultaneous localization and mapping (SLAM) is essential for autonomous exploration, unmanned transportation, and emergency rescue operations in complex underground spaces. However, the demanding conditions of underground spaces, characterized by poor lighting, weak textures, and high dust levels, pose substantial challenges to SLAM. To address this issue, we propose a graph-based adaptive weighted fusion SLAM (AWF-SLAM) for autonomous robots to achieve accurate and robust SLAM in complex underground spaces. First, a contrast limited adaptive histogram equalization (CLAHE) that combined adaptive gamma correction with weighting distribution (AGCWD) in hue, saturation, and value (HSV) space is proposed to enhance the brightness and contrast of visual images in underground spaces. Then, the performance of each sensor is evaluated using a consistency check based on the Mahalanobis distance to select the optimal configuration for specific conditions. Subsequently, we elaborate an adaptive weighting function model, which leverages the residuals from point cloud matching and the inner point rate of image matching. This model fuses data from light detection and ranging (LiDAR), inertial measurement unit (IMU), and cameras dynamically, enhancing the flexibility of the fusion process. Finally, multiple primitive features are adaptively fused within the factor graph optimization, utilizing a sliding window approach. Extensive experiments were conducted to check the performance of AWF-SLAM using a self-designed mobile robot in underground parking lots, excavated subway tunnels, and complex underground coal mine spaces based on reference trajectories and reconstructions provided by state-of-the-art methods. Satisfactorily, the root mean square error (RMSE) of trajectory translation is only 0.17 m, and the mean relative robustness distance between the point cloud maps reconstructed by AWF-SLAM and the reference point cloud map is lower than 0.09 m. These results indicate a substantial improvement in the accuracy and robustness of SLAM in complex underground spaces.

FELC-SLAM: feature extraction and loop closure optimized lidar SLAM system

Abstract Simultaneous Localization and Mapping (SLAM) is one of the key technologies in robot navigation and autonomous driving, playing an important role in robot navigation. Due to the sparsity of LiDAR data and the singularity of point cloud features, accuracy loss of LiDAR SLAM can occur during point cloud matching and localization. In response to these issues, this paper proposes a LiDAR Measurement SLAM algorithm that integrates multi type geometric feature extraction and optimized point cloud registration algorithms. This article first adopts advanced ground segmentation methods and feature segmentation strategies, including ground features, edge features, planar features, and spherical features, to improve matching accuracy. In addition, this article improves the previous method for extracting edge and planar features, extracting clearer and more robust line and surface features to address the degradation of geometric features. Finally, by introducing a robust decoupling global registration method for loop closure detection in the backend of the system, the sparsity problem of distant point clouds and the degradation problem caused by the reduction of inner layers in point cloud registration were effectively solved. In the evaluation of the KITTI dataset, our algorithm reduced absolute trajectory error values by 60%, 29%, and 71% compared to LeGO-LOAM in multi loop and feature constrained scenarios (such as sequences 00, 01, and 02), respectively. The evaluation of the M2DGR and Botanic Garden datasets also indicates that the positioning accuracy of our algorithm is superior to other advanced LiDAR SLAM algorithms.

GLIM: 3D range-inertial localization and mapping with GPU-accelerated scan matching factors

This article presents GLIM, a 3D range-inertial localization and mapping framework with GPU-accelerated scan matching factors. The odometry estimation module of GLIM employs a combination of fixed-lag smoothing and keyframe-based point cloud matching that makes it possible to deal with a few seconds of completely degenerated range data while efficiently reducing trajectory estimation drift. It also incorporates multi-camera visual feature constraints in a tightly coupled way to further improve the stability and accuracy. The global trajectory optimization module directly minimizes the registration errors between submaps over the entire map. This approach enables us to accurately constrain the relative pose between submaps with a small overlap. Although both the odometry estimation and global trajectory optimization algorithms require much more computation than existing methods, we show that they can be run in real-time due to the careful design of the registration error evaluation algorithm and the entire system to fully leverage GPU parallel processing.

2D3D-DescNet: Jointly Learning 2D and 3D Local Feature Descriptors for Cross-Dimensional Matching

The cross-dimensional matching of 2D images and 3D point clouds is an effective method by which to establish the spatial relationship between 2D and 3D space, which has potential applications in remote sensing and artificial intelligence (AI). In this paper, we propose a novel multi-task network, 2D3D-DescNet, to learn 2D and 3D local feature descriptors jointly and perform cross-dimensional matching of 2D image patches and 3D point cloud volumes. The 2D3D-DescNet contains two branches with which to learn 2D and 3D feature descriptors, respectively, and utilizes a shared decoder to generate the feature maps of 2D image patches and 3D point cloud volumes. Specifically, the generative adversarial network (GAN) strategy is embedded to distinguish the source of the generated feature maps, thereby facilitating the use of the learned 2D and 3D local feature descriptors for cross-dimensional retrieval. Meanwhile, a metric network is embedded to compute the similarity between the learned 2D and 3D local feature descriptors. Finally, we construct a 2D-3D consistent loss function to optimize the 2D3D-DescNet. In this paper, the cross-dimensional matching of 2D images and 3D point clouds is explored with the small object of the 3Dmatch dataset. Experimental results demonstrate that the 2D and 3D local feature descriptors jointly learned by 2D3D-DescNet are similar. In addition, in terms of 2D and 3D cross-dimensional retrieval and matching between 2D image patches and 3D point cloud volumes, the proposed 2D3D-DescNet significantly outperforms the current state-of-the-art approaches based on jointly learning 2D and 3D feature descriptors; the cross-dimensional retrieval at TOP1 on the 3DMatch dataset is improved by over 12%.

Bijective upsampling and learned embedding for point clouds correspondences

In this paper, we present a novel pipeline to compute and refine a data-driven solution for estimating the correspondence between 3D point clouds. Our method is compatible with the functional map framework, so it relies on a functional representation of the correspondence, but, differently from other similar approaches, this method is specifically designed to exploit this functional scenario for point cloud matching. Our new method merges a data-driven approach to compute functional basis and descriptors on the shape’s surface and a new refinement method designed for the learned basis. This refinement algorithm arises from a different way of converting functional operators into point-to-point correspondence, which we prove to promote bijectivity between maps, exploiting a theoretical result. Iterating this procedure and performing basis upsampling in the same way as other similar methods, ours increases the accuracy of the correspondence, leading to more bijective correspondences. Different from other approaches, our method allows us to train a functional basis, considering the refinement stage. Combining our new pipeline with an improved feature extractor, our solution outperforms previous methods in various evaluations and settings. We test our method over different datasets, comprising near-isometric and non-isometric pairs.

Measuring dynamic deformation of a rotating blade by point cloud matching algorithm

In this paper, a point cloud matching algorithm is proposed for the scenario of measuring the dynamic deformation field of a rotating blade. Unlike conventional scheme based on Digital Image Correlation (DIC), the essence of this scheme is to reconstruct individual 3D feature points, instead of a group of feature points in a finite-size subset, on the measurement surface and then track their relative offset from reference baseline. The principles of 3D feature-point reconstruction and straddle-frame tracking are similar to those being used in the algorithm of Particle Tracking Velocimetry (PTV); therefore, no additional tags on feature points to facilitate the perspective matching and tracking is needed. Experiment test shows that this scheme works well in the scenario of high feature-point density (0.026 points per pixel), which cannot be achieved by existing feature-point-tagging methods. Both the rigid motion of the rotating blade and its dynamic deformation field can be reliably obtained. Comparing to DIC-based scheme, the benefit of the present proposed scheme includes the improvement of spatial resolution, the insensitivity to feature-point density, as well as the avoidance of outliers due to insufficient texture information.

Multimodal fusion simultaneous localization and mapping method based on multilayer point cloud matching closed-loop detection

Multimodal fusion simultaneous localization and mapping method based on multilayer point cloud matching closed-loop detection

D3Former: Jointly learning repeatable dense detectors and feature-enhanced descriptors via saliency-guided transformer

Establishing accurate and representative matches is a crucial step in addressing the point cloud registration problem. A commonly employed approach involves detecting keypoints with salient geometric features and subsequently mapping these keypoints from one frame of the point cloud to another. However, methods within this category are hampered by the repeatability of the sampled keypoints. In this paper, we introduce a saliency-guided transformer, referred to as D3Former, which entails the joint learning of repeatable Dense Detectors and feature-enhanced Descriptors. The model comprises a Feature Enhancement Descriptor Learning (FEDL) module and a Repetitive Keypoints Detector Learning (RKDL) module. The FEDL module utilizes a region attention mechanism to enhance feature distinctiveness, while the RKDL module focuses on detecting repeatable keypoints to enhance matching capabilities. Extensive experimental results on challenging indoor and outdoor benchmarks demonstrate that our proposed method consistently outperforms state-of-the-art point cloud matching methods. Notably, tests on 3DLoMatch, even with a low overlap ratio, show that our method consistently outperforms recently published approaches such as RoReg and RoITr. For instance, with the number of extracted keypoints reduced to 250, the registration recall scores for RoReg, RoITr, and our method are 64.3%, 73.6%, and 76.5%, respectively.

The Extraction of Roof Feature Lines of Traditional Chinese Village Buildings Based on UAV Dense Matching Point Clouds

Traditional Chinese buildings serve as a carrier for the inheritance of traditional culture and national characteristics. In the context of rural revitalization, achieving the 3D reconstruction of traditional village buildings is a crucial technical approach to promoting rural planning, improving living environments, and establishing digital villages. However, traditional algorithms primarily target urban buildings, exhibiting limited adaptability and less ideal feature extraction performance for traditional residential buildings. As a result, guaranteeing the accuracy and reliability of 3D models for different types of traditional buildings remains challenging. In this paper, taking Jingping Village in Western Hunan as an example, we propose a method that combines multiple algorithms based on the slope segmentation of the roof to extract feature lines. Firstly, the VDVI and CSF algorithms are used to extract the building and roof point clouds based on the MVS point cloud. Secondly, according to roof features, village buildings are classified, and a 3D roof point cloud is projected into 2D regular grid data. Finally, the roof slope is segmented via slope direction, and internal and external feature lines are obtained after refinement through Canny edge detection and Hough straight line detection. The results indicate that the CSF algorithm can effectively extract the roofs of I-shaped, L-shaped, and U-shaped traditional buildings. The accuracy of roof surface segmentation based on slope exceeds 99.6%, which is significantly better than the RANSAC algorithm and the region segmentation algorithm. This method is capable of efficiently extracting the characteristic lines of roofs in low-rise buildings within traditional villages. It provides a reference method for achieving the high-precision modeling of traditional village architecture at a low cost and with high efficiency.

Modelling and protection of grotto temples based on iterative closest point and Poisson

As a non-renewable cultural heritage, grotto temples face many value threats and the crises of rapid extinction. Due to the unique environment, diverse volume and complex geometric structure of grotto temples, cultural relic protection faces huge challenges. Digitisation has become the most important and effective preventive protection method. This study proposes a comprehensive modeling method, which uses three-dimensional (3D) laser scanning technology to obtain the point cloud data of grotto ruins, close range photogrammetry to obtain multi-angle and high-resolution two-dimensional (2D) image data, and through the fusion modelling of the laser point cloud and image dense matching point cloud, a complete geometric model of grotto is obtained. The registration method of the improved algorithm is based on Euclidean distance residual. The performance is better, with residuals mainly distributed in the range of 0.02 to 0.03 mm. In terms of texture mapping, the accuracy of model mapping is adjusted by adjusting the constraints of control points. The gray values of 50 pixels around the control points show significant gradient changes in the X direction, while the changes in the Y direction are relatively small. These methods and technologies provide practical and effective means for the protection and preservation of cultural relics in grotto temples.

Multisensor Fusion for Vehicle-to-Vehicle Cooperative Localization With Object Detection and Point Cloud Matching

Multisensor Fusion for Vehicle-to-Vehicle Cooperative Localization With Object Detection and Point Cloud Matching

Global principal planes aided LiDAR-based mobile mapping method in artificial environments

3-D mapping of buildings is crucial for urban renewal, but traditional LiDAR-based mapping methods are often less effective for buildings with narrow spaces and limited geometric features. Current methods attempt to overcome this by integrating additional sensors, such as cameras, which increases cost and complexity. This paper proposes a novel LiDAR-based mobile mapping framework using global principal planes (GPPs) to address this challenge without additional sensors. GPPs are defined as unlimited planes characterized by principal normal vectors (PNVs). GPPs can provide stronger constraints than traditional small planes extracted from one or certain LiDAR frames because they are little affected by the accumulative error from point cloud matching. A PNV estimation method is also proposed based on an inertial measurement unit and polar histogram, and PNVs are axes of the natural cartesian XYZ coordinate system. Point clouds are transformed into the PNVs coordinate system to extract robust edge and plane feature points and GPPs. The proposed framework is tested in various environments. It achieves about 3 cm accuracy in corridors and similar accuracy in stairwells. Compared to five state-of-the-art mapping methods (Cartographer, etc.), its accuracy improves by over 76%, increasing at least an order of magnitude. In the outdoor KITTI dataset, it shows a reduction in absolute pose errors by 4% to 20%. Extensive experiments demonstrate its accuracy, robustness, and generalizability. Ablation experiments further validate the efficacy of different components in the framework.

RANSAC Algorithm and Distributed Framework for Point Cloud Processing of Ancient Buildings

This paper introduces a comprehensive framework for the point cloud processing of traditional buildings. The framework includes segmentation using the RANSAC algorithm and distributed storage based on a fuzzy weighting approach. The methodology employs a height threshold parameter for segmentation to extract building structural elements effectively. Furthermore, an interactive access control model distributes tasks across nodes to achieve load balancing during point cloud matching and analysis. Experimental results demonstrate segmentation accuracy exceeding 99% and alignment time reduction to 967 ms for point cloud models. The distributed computation efficiency reaches 0.8, outperforming conventional methods. The proposed techniques enable accurate dimensional capture, efficient data storage, and information extraction from traditional buildings to support digital preservation.

Stereoscopic Image Enabled Animation Art Design Exploration

Abstract With the emergence of movie art, many related art forms were born. Animation art has gained a large number of audiences due to its exaggerated expression. In order to improve the problems of character image overlapping and scene distortion that may occur in the original two-dimensional space, this paper constructs a semi-global stereo image matching algorithm based on the use of point cloud matching in the animation screen for the determination of the screen position and the morphology of the animated character. To compensate for the radiation difference in the stereo image matching algorithm, object-square geometric constraints are proposed to improve the matching cost calculation process. After calculating the matching cost, the original animation model is optimized or rebuilt by creating a depth space image. Taking Disney animation characters as an example, the number of joints required for the reconstruction of the “Snow White” character is 2.9 times more than that of the “Mickey” character, but the reconstruction time of the skeleton is only 12 times longer than that of the “Mickey” character. However, the skeleton reconstruction time is only 12.1 seconds longer than that of the “Mickey” character, which indicates that the algorithm proposed in this paper does not produce a large difference in reconstruction time due to the difference in characters. For the construction of animated indoor scenes, in the eight experimental scenes, the maximum number of grids can be divided into 42365412, and the processing time required is 276.413 seconds. It shows that the algorithm in this paper is capable of subregionally refining the indoor space of the animation. The proposed semi-global stereo image matching-based pan can quickly reconstruct animated character models and animated scenes, as can be seen.