Point Cloud Features Research Articles

Efficient Single Correspondence Voting for Point Cloud Registration.

3D point cloud registration is a crucial task in a variety of fields, including remote sensing mapping, computer vision, virtual reality, and autonomous driving. However, this task is still challenging due to the challenges of noise, non-uniformity, partial overlap, and repeated local features in large scene point clouds. In this paper, we propose an efficient single correspondence voting method for large scene point cloud registration. Specifically, we first propose an efficient hypothetical transformation prediction method called SCVC, which determines the 5 degrees of freedom of the transformation through one correspondence, and then uses Hough voting to determine the last degree of freedom. This algorithm can significantly improve the accuracy of registration in both indoor and outdoor scenes. On the other hand, we propose a more robust transformation verification function called VDIR, which can obtain the optimal registration result of two raw point clouds. Finally, we conduct a series of experiments that demonstrate that our method achieves state-of-the-art performance on four real-world datasets: 3DMatch, 3DLoMatch, KITTI, and WHU-TLS. Our code is available at https://github.com/xingxuejun1989/SCVC.

Multi-Modal Fusion Based on Depth Adaptive Mechanism for 3D Object Detection

Lidars and cameras are critical sensors for 3D object detection in autonomous driving. Despite the increasing popularity of sensor fusion in this field, accurate and robust fusion methods are still under exploration due to non-homogenous representations. In this paper, we find that the complementary roles of point clouds and images vary with depth. An important reason is that the point cloud appearance changes significantly with increasing distance from the Lidar, while the image's edge, color, and texture information are not sensitive to depth. To address this, we propose a fusion module based on the Depth Attention Mechanism (DAM), which mainly consists of two operations: gated feature generation and point cloud division. The former adaptively learns the importance of bimodal features without additional annotations, while the latter divides point clouds to achieve differential fusion of multi-modal features at different depths. This fusion module can enhance the representation ability of original features for different point sets and provide more comprehensive features by using the dual splicing strategy of concatenation and index connection. Additionally, considering point density as a feature and its negative correlation with depth, we build an Adaptive Threshold Generation Network (ATGN) to generate the depth threshold by extracting density information, which can divide point clouds more reasonably. Experiments on the KITTI dataset demonstrate the effectiveness and competitiveness of our proposed models.

A Robot Navigation System in Complex Terrain Based on Statistical Features of Point Clouds

A Robot Navigation System in Complex Terrain Based on Statistical Features of Point Clouds

Curvature estimation for point cloud 2-manifolds based on the heat kernel

<p>The geometry processing of a point cloud 2-manifold (or point cloud surface) heavily depends on the discretization of differential geometry properties such as Gaussian curvature, mean curvature, principal curvature, and principal directions. Most of the existing algorithms indirectly compute these differential geometry properties by seeking a local approximation surface or fitting point clouds with certain polynomial functions and then applying the curvature formulas in classical differential geometry. This paper initially proposed a new discretized Laplace-Beltrami operator by applying an inherent distance parameter, which acts as the foundation for precisely estimating the mean curvature. Subsequently, the estimated mean curvature was taken as a strong constraint condition for estimating the Gaussian curvatures, principal curvatures, and principal directions by determining an optimal ellipse. The proposed methods are mainly based on the heat kernel function and do not require local surface reconstruction, thus belonging to truly mesh-free methods. We demonstrated the correctness of the estimated curvatures in both analytic and non-analytic models. Various experiments indicated that the proposed methods have high accuracy. As an exemplary application, we utilized the mean curvature for detecting features of point clouds.</p>

From Noise Addition to Denoising: A Self-Variation Capture Network for Point Cloud Optimization.

Point clouds obtained from 3D scanners are often noisy and cannot be directly used for subsequent high-level tasks. In this article, we propose a novel point cloud optimization method capable of denoising and homogenizing point clouds. Our idea is based on the assumption that the noise is generally much smaller than the effective signal. We perform noise perturbation on the noisy point cloud to get a new noisy point cloud, called self-variation point cloud. The noisy point cloud and self-variation point cloud have different noise distribution, but the same point cloud distribution. We compute the potential commonality between two noisy point clouds to obtain a clean point cloud. To implement our idea, we propose a Self-Variation Capture Network (SVCNet). We perturb the point cloud features in the latent space to obtain self-variation feature vectors, and capture the commonality between two noisy feature vectors through the feature aggregation and averaging. In addition, an edge constraint module is introduced to suppress low-pass effects during denoising. Our denoising method does not take into account the noise characteristics, and can filter the drift noise located on the underlying surface, resulting in a uniform distribution of the generated point cloud. The experimental results show that our algorithm outperforms the current state-of-the-art algorithms, especially in generating more uniform point clouds. In addition, extended experiments demonstrate the potential of our algorithm for point clouds upsampling.

Multi-scale Part-based Feature Representation for 3D Domain Generalization and Adaptation.

Deep networks on 3D point clouds have achieved remarkable success in 3D classification, but they are vulnerable to geometric variations resulting from inconsistent data acquisition procedures. This leads to challenging 3D domain generalization and adaptation tasks, aiming to tackle the challenge that the performance of a model trained on a source domain will degrade on an out-of-distribution target domain. In this paper, we introduce a novel Multi-Scale Part-based feature Representation, dubbed MSPR, as a generalizable representation for point cloud domain generalization and adaptation. Rather than relying on the global point cloud feature representation, we align the part-level features of shapes at different scales to a set of learnable part-template features, which can encode local geometric structures shared between the source and the target domains. Specifically, we construct a part-template feature space shared between source and target domains. Shapes from different domains are organized into part-level features at various scales and then aligned to the part-template features. To leverage the generalization ability of small-scale parts and the discrimination ability of large-scale parts, we further design a cross-scale feature fusion module to exchange information between aligned part-based features at different scales. The fused part-based representations are finally aggregated by a part-based feature aggregation module. To improve the robustness of the aligned part-based representations and global shape representation to geometry variations, we further propose a Contrastive Learning framework on Shape Representation (CLSR), applied to both 3D domain generalization and adaptation tasks. We conduct experiments on 3D domain generalization and adaptation benchmarks for point cloud classification. Experimental results demonstrate the effectiveness of our proposed approach, outperforming the previous state-of-the-art methods for both domain generalization and adaptation tasks. Ablation studies confirm the effectiveness of the proposed components in our model.

Feature Distance Representer: Exploring the Connection between Point Cloud Feature Levels from the Perspective of Distance

Feature Distance Representer: Exploring the Connection between Point Cloud Feature Levels from the Perspective of Distance

LiDAR-SLAM loop closure detection based on multi-scale point cloud feature transformer

Loop closure detection is an important part of simultaneous localization and mapping (SLAM), used to detect and correct map and trajectory drift issues caused by accumulated errors. For the LiDAR-SLAM system, the sparsity and disorder of point clouds make position recognition based on 3D point clouds more challenging. In recent years, many deep learning-based closed-loop detection algorithms have been proposed. However, due to the sparsity of point clouds, current deep learning algorithms often convert point clouds into histograms or depth maps and then process them using deep learning algorithms, undoubtedly causing information loss. In this paper, we propose a closed-loop detection method based on multi-scale point cloud features transformer, which introduces multi-scale point cloud feature extraction and transformer global context modeling. We use voxel sparse convolution to obtain features of original point clouds at different resolutions and establish contextual relationships between features at different resolutions using the transformer network to achieve multi-scale feature fusion, and then obtain global descriptors. The obtained global descriptors can be used not only for closed-loop detection but also for front-end registration to address the challenges of point cloud processing in the SLAM system, especially in enhancing global modeling capabilities and reducing information loss. Our method directly processes point cloud data and integrates multi-scale point cloud feature information, which can better adapt to the characteristics of LiDAR-SLAM systems, improving the accuracy and robustness of localization and map construction, thus having broad application prospects in the field of measurement. We evaluated our method on multiple sequences of the KITTI and KITTI-360 datasets, each containing more than 5000 frames of point clouds, and extensively evaluated on a self-collected dataset of over 3.6 km. The experimental results show that our method achieves an accuracy of over 80% on multiple datasets and demonstrates superior performance in different environments.

SGF3D: Similarity-guided fusion network for 3D object detection

The representation of pseudo point cloud can significantly improve the precision of 3D object detection. However, existing pseudo point cloud-based methods typically fuse the processed features through coarse concatenation, which ignores the consistency between the point cloud and pseudo point cloud features. The inconsistency of features in different modal data can lead to detection bias. In this paper, we propose a novel pseudo point cloud-based network called SGF3D, which utilizes a cross-modal attention module cross-modal attention fusion (CMAF) to fuse point cloud and pseudo point cloud features. It can better learn the cross-modal similarity of output features, enabling the detection box to fit better with the target. We also designed a region of interest (RoI) head similarity attention head (SAH) to utilize the overlooked similarity to optimize training without increasing the complexity of the network. By using CMAF and SAH, the proposed method can obtain more accurate bounding boxes. Extensive experiments on KITTI dataset demonstrate that the proposed method can achieve competitive results. Training code and well trained weights are available at https://github.com/ChunZheng2022/SGF3D.

Optimal Pose Estimation and Covariance Analysis with Simultaneous Localization and Mapping Applications

This work provides a theoretical analysis for optimally solving the pose estimation problem using total-least-squares for vector observations from landmark features, which is central to applications involving simultaneous localization and mapping. First, the optimization process is formulated with observation vectors extracted from point-cloud features. Then, error-covariance expressions are derived. The attitude and position estimates obtained via the derived optimization process are proven to reach the bounds defined by the Cramér–Rao lower bound under the small-angle approximation of attitude errors. A fully populated observation noise-covariance matrix is assumed as the weight in the cost function to cover the most general case of the sensor uncertainty. This includes more generic correlations in the errors than previous cases involving an isotropic noise assumption. The proposed solution is verified using Monte Carlo simulations, a Gazebo simulation in a robotics operating system, and an experiment with an actual light detection and ranging sensor to validate the error-covariance analysis.

An obstacle detection method for dual USVs based on SGNN-RMEN registration of dual-view point clouds

This paper proposes an unsupervised obstacle detection method for dual unmanned surface vessels (USVs) using the SGNN-RMEN framework. The method utilizes dual-view point clouds captured by two USVs to improve the detection of small obstacles under the condition of a visible shore. Firstly, a siamese graph neural network (SGNN) is designed to extract global contour features. A specialized task of global contour consistency classification is employed to ensure the interpretability of these features. Secondly, a rotation matrices estimation network (RMEN) is utilized to estimate the optimal rotation transformation for aligning point clouds based on the global contour features of the dual-view point clouds. The overlap degree between the dual-view point clouds before and after registration is used as feedback to train the model, enabling unsupervised learning. Finally, a " gridding - filtering - clustering” method is applied to annotate the size and position of obstacles based on the registration of the dual-view point clouds. Comparative experiments on two datasets demonstrate the effectiveness of the proposed method in accurately extracting contour features, achieving precise dual-view point cloud registration, and improving the detection rate of small obstacles in nearshore environments. The method also exhibits strong adaptability in unfamiliar marine environments.

Self-Attention Mechanism-Based Head Pose Estimation Network with Fusion of Point Cloud and Image Features.

Head pose estimation serves various applications, such as gaze estimation, fatigue-driven detection, and virtual reality. Nonetheless, achieving precise and efficient predictions remains challenging owing to the reliance on singular data sources. Therefore, this study introduces a technique involving multimodal feature fusion to elevate head pose estimation accuracy. The proposed method amalgamates data derived from diverse sources, including RGB and depth images, to construct a comprehensive three-dimensional representation of the head, commonly referred to as a point cloud. The noteworthy innovations of this method encompass a residual multilayer perceptron structure within PointNet, designed to tackle gradient-related challenges, along with spatial self-attention mechanisms aimed at noise reduction. The enhanced PointNet and ResNet networks are utilized to extract features from both point clouds and images. These extracted features undergo fusion. Furthermore, the incorporation of a scoring module strengthens robustness, particularly in scenarios involving facial occlusion. This is achieved by preserving features from the highest-scoring point cloud. Additionally, a prediction module is employed, combining classification and regression methodologies to accurately estimate head poses. The proposed method improves the accuracy and robustness of head pose estimation, especially in cases involving facial obstructions. These advancements are substantiated by experiments conducted using the BIWI dataset, demonstrating the superiority of this method over existing techniques.

Adaptive learning point cloud and image diversity feature fusion network for 3D object detection

3D object detection is a critical task in the fields of virtual reality and autonomous driving. Given that each sensor has its own strengths and limitations, multi-sensor-based 3D object detection has gained popularity. However, most existing methods extract high-level image semantic features and fuse them with point cloud features, focusing solely on consistent information from both sensors while ignoring their complementary information. In this paper, we present a novel two-stage multi-sensor deep neural network, called the adaptive learning point cloud and image diversity feature fusion network (APIDFF-Net), for 3D object detection. Our approach employs the fine-grained image information to complement the point cloud information by combining low-level image features with high-level point cloud features. Specifically, we design a shallow image feature extraction module to learn fine-grained information from images, instead of relying on deep layer features with coarse-grained information. Furthermore, we design a diversity feature fusion (DFF) module that transforms low-level image features into point-wise image features and explores their complementary features through an attention mechanism, ensuring an effective combination of fine-grained image features and point cloud features. Experiments on the KITTI benchmark show that the proposed method outperforms state-of-the-art methods.

TOWARDS WHEAT YIELD ESTIMATION IN PLANT BREEDING FROM INHOMOGENEOUS LIDAR POINT CLOUDS USING STOCHASTIC FEATURES

Abstract. The world relies heavily on wheat, corn, and rice for nutrition, with global challenges such as population growth and climate change threatening food security. To tackle this, plant breeding, supported by digital technologies, focuses on improving food quality and quantity. Currently, crop yield estimation uses indirect observations through hyperspectral data and spectral indices, such as NDVI, which suffer from low sensitivity in breeding scenarios. Terrestrial laser scanners (TLS) present an alternative, allowing observations of the quantity and morphology of wheat ears from point clouds, which are directly linked to grain yield. However, exploiting these observations under field conditions presents challenges, mainly due to reduced resolution and non-homogenous properties of point clouds. In response, we propose an approach for in-field wheat yield estimation using machine learning and stochastic features of TLS point clouds that are specifically handcrafted to be less sensitive to the abovementioned phenomena. This approach avoids the need for explicit 3D reconstruction of individual plants and plant organs. Our initial results show limited success in yield estimation when posed as a regression problem. However, when framed as a classification problem focusing on detecting top- and bottom-performing plant phenotypes, we achieved a promising accuracy of 84.4% and AUC of 0.93. While encouraging, these are only the first results under relaxed conditions and further work is needed to enhance practical applicability.

RESEARCH ON POWER TRANSMISSION CHANNEL CHANGE DETECTION BASED ON MULTI-TEMPORAL POINT CLOUD DATA

Abstract. Airborne LiDAR can directly obtain 3D information of ground objects. By comparing the multi-temporal LiDAR data, ground objects change information of power transmission channel can be detected, providing data support for transmission line operation and maintenance. In this paper, an improved ICP algorithm based on multi-temporal LiDAR point cloud data power transmission channel ground object change detection method is proposed. Firstly, based on the classification of point cloud data, a two-level matching method of multi-temporal point cloud data considering the characteristics of power transmission channel was proposed to achieve accurate registration of point cloud data. Then, change detection and analysis of different types of ground feature point cloud data were carried out through elevation difference. Finally, cluster analysis was carried out on the changed ground feature points to generate multi-temporal relative ratio analysis report. Experimental results show that the proposed method can effectively detect power transmission channel changes.

Defect segmentation with local embedding in industrial 3D point clouds based on transformer

Three-dimensional (3D) defect detection provides an effective method for improving industrial production efficiency. However, the 3D dataset is scarce, which is valuable for the industrial production field. This study proposes a new approach for detecting defect point clouds, which can provide an end-to-end 3D defect detection model. A self-attention mechanism is used to enrich the semantic relationships between local neighborhood features and global features based on the connection between them. Through adding multi-channel features, the rich structural features of the target point cloud are obtained, and the defect areas are accurately segmented to finally complete the 3D point cloud defect detection task. Furthermore, the multi-feature fusion in the model makes the segmented defect regions closer to the ground truth. Our method outperforms four state-of-the-art point cloud segmentation methods in terms of both segmentation region accuracy and defect detection point cloud accuracy. In the field of 3D defect detection, it provides an effective method to detect 3D information of industrial products.

High-quality 3D Object Detection Based on Instance-aware Sampling

Most existing 3D object detection that is researched on point clouds uses random down-sampling algorithms, which preserve the primary shape and feature information of the point clouds and extract robust features of sampling point clouds to decrease the consumption of computing resources and improve processing inference efficiency. On the other hand, this method of sampling may disregard significant areas within the point clouds, thereby diminishing the precision of detecting comparatively minute entities such as pedestrians and cyclists. In this research, an instance perception-based object detection method is presented that adaptively chooses additional foreground points in response to the properties of smaller objects in the initial point clouds. In addition, it offers end-to-end training and addresses the issue that the typical hand-designed multi-scale grouping receptive fields do not accurately reflect the size of the item. To effectively enhance the recall rate of foreground points, we present an instance-aware algorithm, which relies on farthest point sampling. Specifically, we employ dynamic gating networks to attain instance perception. This approach enables sample candidates to encompass a larger number of foreground objects. Then we use an aggregation layer to efficiently capture robust contextual features with rich spatial information. Through experiments performed on the KITTI dataset, we substantiate the effectiveness of our IAS when compared with current advanced 3D object detection methods. We have achieved a notable enhancement, especially in identifying diminutive objects.

3D point cloud object detection algorithm based on Transformer

In response to the difficulty in deploying anchor box based methods in 3D object detection due to the increase in spatial dimensions, this paper studies a point cloud object detection algorithm based on set prediction. This article proposes a Transformer based 3D point cloud object detection algorithm, and combines the characteristics of point clouds in autonomous driving scenarios to propose an improved spatial modulation attention and heat map initialization strategy for training acceleration and query initialization, achieving good detection performance in shallow networks. This article compares it with other algorithms on the KITTI dataset, and the results show that our algorithm has reached an advanced level in performance. We also conducted ablation experiments on the main components of the algorithm to verify the contribution of each module to the detection effect.

Research on the path extraction method of shoe upper grinding based on 3D vision

Shoe upper grinding is a crucial step in the shoe manufacturing process, impacting both the bonding quality between the sole and the upper and the overall appearance of the shoe. Presently, many shoe companies rely on manual or semi-automated grinding methods for shoe uppers, resulting in low production efficiency and subpar grinding quality. This study proposes a 3D vision-based shoe upper grinding path extraction method for guiding a robotic arm to polish the shoe upper and address this issue. Initially, the RGB-D camera was used to obtain the point cloud data of the shoe surface, which was then preprocessed using point cloud filtering, smoothing and registration methods. Second, by combining RGB images with 3D point clouds, we propose a point-cloud region of interest (ROI) extraction method based on RGB information to obtain the feature point cloud of the shoe upper grinding line. Finally, we slice the feature point cloud of the shoe upper grinding line, extract the grinding points using an intersection algorithm, and fit them with a non-uniform rational B-spline curve (NURBS) to obtain the shoe’s upper grinding path. The research demonstrates that this method can effectively extract the shoe upper grinding trajectory curve, significantly reduce the time required to obtain the shoe upper grinding trajectory, and achieve good accuracy, with promising application prospects.

Multi-scale split matching three-dimensional reconstruction method of surface microcracks on the silicon nitride bearing roller

Aiming at the characteristics of parametric discrete and non-significant depth of silicon nitride bearing roller surface microcracks. A multi-scale split matching method is proposed. To realize the full reconstruction of the characteristics of the three-dimensional morphology of the surface microcracks. Combined with the parametric discretization characteristics of the surface microcracks, a box filter is defined to do second-order Gaussian differentiation on the image. Construct a multiscale Hessian matrix in the scale space of the surface microcrack image. Determine the covariance-discrete feature points, and obtain the global covariance-discrete sparse point cloud of silicon nitride bearing roller surface microcracks by non-maximum value suppression. For the non-significant depth features of silicon nitride bearing roller surface microcracks. Dissect the surface microcrack distortion images. Divide each image into multiple regional pixel blocks and construct an aggregation function to determine the exact depth value of each pixel block at the cost of matching each pixel block with its surrounding pixel blocks. Dense point cloud reconstruction of non-significant depth features of silicon nitride bearing roller surface microcracks is realized. The average distribution of dense clusters of reprojection errors of the dense point cloud features is less than 0.4pix, the median residual of the field of view of the surface microcrack image is 0.57pix, and the total number of effectively extracted feature points is about 92.9 %. The detailed lossless recovery reconstruction of the parametric discrete, non-significant depth features of silicon nitride bearing roller surface microcracks is realized. A new idea is provided for the establishment of molecular dynamics geometric modeling of the surface microcrack extension process on the silicon nitride bearings roller.