Point Cloud Features Research Articles

Multi-feature enhancement based on sparse networks for single-stage 3D object detection

In the field of autonomous driving, the accuracy and real-time requirements for 3D object detection technology continue to improve, which is directly related to the commercialization process and market popularity of autonomous vehicles. Despite the efficiency of pillar-based coding for onboard systems, it falls short in terms of accuracy and the reduction of incorrect positives. In this paper, we will examine how to solve the problem of high incorrect rate and low accuracy of existing methods. Firstly, a MAP coding module is introduced to optimize previous point cloud feature coding modules, allowing for the efficient extraction of fine-grained features from point cloud data. Then, we introduce an innovative sparse dual attention (SDA) to efficiently filter out irrelevant details in feature extraction, thereby improving the pertinence and efficiency of information extraction. Finally, to address the potential loss of information from single local feature extraction, a local and global fusion module (CTGC) is introduced. Our method has proactively demonstrated its efficiency and accuracy through rigorous experimentation across diverse datasets. Analysis of the results leads to the conclusion that our solutions provide accurate and robust detection results. Code will be available at https://github.com/lcy199905/MyOpenPCDet.git.

EIDU-Net: edge-preserved inception DenseGCN U-Net for LiDAR point cloud segmentation

With the development of laser scanners and machine learning, point cloud semantic segmentation plays a significant role in autonomous driving, scene reconstruction, human-computer interaction, and other fields. In recent years, point cloud semantic segmentation based on deep learning has become one of the key research directions in point cloud processing. Due to the limited ability to exploit geometric details and contextual information in point clouds, most methods that adopt encoder-decoder architecture lose local structural information easily, especially detailed features, and extract features insufficiently. To address this issue, the edge-preserving inception DenseGCN U-Net (entitled as EIDU-Net) is proposed. EIDU-Net makes full use of the complementation between geometric details in the original point cloud and high-level features. The edge-preserved graph pooling (EGP) layer, the key module of the EIDU-Net, is designed to retain additional edge feature information from the original point cloud during pooling operations. Accordingly, the edge-preserved graph unpooling (EGU) layer can restore the feature graph more efficiently based on the additionally retained edge features. Extensive experiments demonstrate that our proposed EIDU-Net has remarkable improvements on semantic segmentation tasks under whatever S3DIS or Terracotta Warrior fragments. Our code is publicly available at https://github.com/caoxin918/EIDU-Net.

Weakly supervised 3D point cloud semantic segmentation for architectural heritage using teacher-guided consistency and contrast learning

Point cloud semantic segmentation is significant for managing and protecting architectural heritage. Currently, fully supervised methods require a large amount of annotated data, while weakly supervised methods are difficult to transfer directly to architectural heritage. This paper proposes an end-to-end teacher-guided consistency and contrastive learning weakly supervised (TCCWS) framework for architectural heritage point cloud semantic segmentation, which can fully utilize limited labeled points to train network. Specifically, a teacher-student framework is designed to generate pseudo labels and a pseudo label dividing module is proposed to distinguish reliable and ambiguous point sets. Based on it, a consistency and contrastive learning strategy is designed to fully utilize supervision signals to learn the features of point clouds. The framework is tested on the ArCH dataset and self-collected point cloud, which demonstrates that the proposed method can achieve effective semantic segmentation of architectural heritage using only 0.1 % of annotated points.

High precision DSRC and LiDAR data integration positioning method for autonomous vehicles based on CNN

In order to improve the safe driving and automatic positioning capability of autonomous vehicles, a high-precision DSRC and LiDAR data integration positioning technology for autonomous vehicles based on CNN is proposed. Import the data of Dedicated Short Range Communications and Light Detection and Ranging for automatic driving vehicle positioning, carry out kinematic analysis of autonomous driving vehicles under multi-sensor fusion, and transform the data of DSRC and LiDAR sensors into tightly coupled coordinate systems; The CNN depth learning method is used to compensate the position and attitude tracking estimation error under the overall time stamp synchronization of the sensor through adaptive information tracking; The first and second order feedforward compensation is made for the positioning parameters of the autonomous driving vehicle using the PID model, and the point cloud feature matching model is fused to complete the estimation of the positioning attitude parameters of the autonomous driving vehicle. In order to eliminate the noise interference under the DSRC communication mechanism, the Kalman filter function is used to automatically optimize the constraint parameters in the point cloud feature detection model, and the positioning error parameters are dynamically filtered and adjusted; Kinematics analysis is carried out for the driving state of the vehicle, and the positioning error in the vehicle movement is controlled through the difference technology to achieve high-precision DSRC and LiDAR data integration positioning. The simulation results show that this method can integrate and locate the high-precision DSRC and LiDAR data of the autonomous vehicle, and the attitude estimation and positioning accuracy of the vehicle is good, while the error of the attitude parameter estimation of the autonomous vehicle is low.

GNSS/IMU/LO integration with a new LO error model and lateral constraint for navigation in urban areas

The quest for reliable vehicle navigation in urban environments has led the integration of Light Detection and Ranging (LiDAR) Odometry (LO) with Global Navigation Satellite Systems (GNSS) and Inertial Measurement Units (IMU). However, the performance of the integrated system is limited by a lack of accurate LO error modeling. In this paper, we propose a weighted GNSS/IMU/LO integration-based navigation system with a novel LO error model. The Squared Exponential Gaussian Progress Regression (SE-GPR) based LO error model is developed by considering the vehicle velocity and number of point cloud features. Based on error prediction for GNSS positioning and LO, a weighting strategy is designed for integration in an Extended Kalman Filter (EKF). Furthermore, error accumulation of the navigation state, especially in GNSS-challenging scenarios, is restrained by the LiDAR-Aided Lateral Constraint (LALC) and Non-Holonomic Constraint (NHC). An experiment was conducted in a deep urban area to test the proposed algorithm. The results show that the proposed algorithm delivers horizontal and three-dimensional (3D) positioning Root Mean Square Errors (RMSEs) of 3.669 m and 5.216 m, respectively. The corresponding accuracy improvements are 35.9% and 50.0% compared to the basic EKF based GNSS/IMU/LO integration.

Point cloud-based feature extraction and surface reconstruction for parts paring in high-precision assembly

Abstract Assembly plays a crucial role in industrial manufacturing in industrial manufacturing, but the efficiency of conventional manual methods for parts pairing is limited. Previous research has demonstrated the feasibility of deep learning for point cloud feature extraction and 3D reconstruction. An innovative method utilizing deep learning for high-precision feature extraction and surface reconstruction is introduced to optimize parts pairing in this paper. Geometric dimensions and surface topography data are obtained by defining key assembly features and utilizing the Random Sample Consensus method. Deep learning is then used to directly regress the Surface Distance Function from point samples, facilitating comprehensive part surface modeling and supporting assembly simulation in the digital twin. To validate this approach, a case study demonstrates successful matching of 30 shaft parts and 30 hole parts after optimization, with an increase in average uniformity by 0.024. This highlights the proposed method’s superior effectiveness and accuracy in feature extraction and surface reconstruction.

Global Attention-Guided Dual-Domain Point Cloud Feature Learning for Classification and Segmentation

Global Attention-Guided Dual-Domain Point Cloud Feature Learning for Classification and Segmentation

Non-contact measurement of conveyor belt speed based on fast point cloud registration of feature block

Abstract Non-contact, real-time measurement of conveyor belt speed is critical for energy-saving speed regulation and efficient development of coal mine conveyor systems. Existing speed measurement technologies for conveyor systems are often limited by the slippage and wear in contact measurement and complex environmental disturbance. This study introduces three-dimensional point cloud technology into coal flow information detection and innovatively presents a non-contact measurement method of conveyor belt speed based on fast point cloud registration of feature blocks. In the proposed method, a three-dimensional camera is used to capture point cloud data of the dynamically running conveyor belt, and the raw point cloud is preprocessed and segmented into blocks. Then, the C-MANV (curvature and mean angle of normal vectors) features of block point clouds are constructed based on the point cloud neighborhood curvature and the mean angle of neighborhood normal vectors. Finally, an improved block point cloud registration method based on C-MANV features is adopted to achieve the accurate measurement of the dynamic running speed of conveyor belt. Experimental results demonstrate that the proposed method achieves an average relative error of less than 1.8% in high-speed conveyor belt operation with an average processing time of less than 35 ms, which fulfils the accuracy and real-time requirements for conveyor belt speed detection in coal mines. This study provides an effective technical solution for the speed monitoring of coal mine conveyor systems.

TAMC: Textual Alignment and Masked Consistency for Open-Vocabulary 3D Scene Understanding.

Three-dimensional (3D) Scene Understanding achieves environmental perception by extracting and analyzing point cloud data with wide applications including virtual reality, robotics, etc. Previous methods align the 2D image feature from a pre-trained CLIP model and the 3D point cloud feature for the open vocabulary scene understanding ability. We believe that existing methods have the following two deficiencies: (1) the 3D feature extraction process ignores the challenges of real scenarios, i.e., point cloud data are very sparse and even incomplete; (2) the training stage lacks direct text supervision, leading to inconsistency with the inference stage. To address the first issue, we employ a Masked Consistency training policy. Specifically, during the alignment of 3D and 2D features, we mask some 3D features to force the model to understand the entire scene using only partial 3D features. For the second issue, we generate pseudo-text labels and align them with the 3D features during the training process. In particular, we first generate a description for each 2D image belonging to the same 3D scene and then use a summarization model to fuse these descriptions into a single description of the scene. Subsequently, we align 2D-3D features and 3D-text features simultaneously during training. Massive experiments demonstrate the effectiveness of our method, outperforming state-of-the-art approaches.

3D Object Detection via Residual SqueezeDet

Three-dimensional object detection is a critical task in computer vision with applications in autonomous driving, robotics, and augmented reality. This paper introduces Residual SqueezeDet, a novel network architecture that enhances the performance of 3D object detection on the KITTI dataset. Building upon the efficient SqueezeDet framework, we propose the Residual Fire module, which incorporates skip connections inspired by ResNet architectures into the original Fire module. This innovation improves gradient flow, enhances feature propagation, and allows for more effective training of deeper networks. Our method leverages point cloud and image-based features, employing a Residual SqueezeDet to effectively capture local and global context. Extensive experiments on the KITTI dataset demonstrate that Residual SqueezeDet significantly outperforms the original SqueezeDet, with particularly notable improvements in challenging scenarios. The proposed model maintains computational efficiency while achieving state-of-the-art performance, making it well-suited for real-time applications in autonomous driving. Our work contributes to the field by providing a more accurate and robust solution for 3D object detection, paving the way for improved perception systems in dynamic environments.

A method for accurate extraction of three-dimensional point cloud feature data in loading test of container ships

The accuracy of container slot dimensions directly affects the efficiency and safety of container loading and unloading. However, traditional methods of container loading tests need to be improved for their long cycles, high costs, and low production efficiency. With the application of three-dimensional laser scanning devices in ship manufacturing, the use of simulated container loading test methods based on point cloud becomes a breakthrough in key technology for container ship construction. This paper presents a multi-objective segmentation method of cargo hold feature point cloud for container ship simulated loading tests. Data preprocessing algorithms are applied to remove noise and pseudo-image, and then a Random Sample Consensus algorithm is used to remove bulkhead and cargo bottom point clouds. Based on the density-based spatial clustering of applications with noise algorithm and the region growing algorithm, an algorithm for multiple segmentation of guide rail and bottom cone point clouds is devised, and a four-plane segmentation of guide rail point clouds is realized. Finally, a method for three-dimensional point cloud data flatness assessment and a Hausdorff distance based method for calculating the guide rail lateral and longitudinal spacing are proposed. These methods are validated through the loading test of a 16,000 TEU container ship.

Approach angle estimation method for ships based on deep learning

Accurately estimating the ship's approach angle is crucial during berthing, or passing through navigational structures. It forms the foundation for ship navigation decisions. This study introduces the LiDAR point cloud-based ship pose perception network (PP-Net), a novel deep learning-based method for acquiring ship approach angles. Addressing the challenge of manually annotating approach angles, the ship pose perception method based on point cloud feature distribution (FD) was proposed to obtain initial angle estimates. The max-pooling function was utilized to address the disorder of point clouds, and a multi-resolution feature extractor was employed to enhance the network's robustness in extracting features from inconsistently dense point clouds. By using iterative farthest point sampling (IFPS), datasets with varying numbers of point clouds were standardized to a fixed number of points, thereby normalizing input dimensions and improving computational efficiency. Additionally, a novel loss function was proposed to improve the model's predictive accuracy. Experimental validation conducted in the scenario of ships entering ship lifts demonstrates the effectiveness of the proposed method. The outcomes can support safe navigation for ships in restricted waterways.

A lightweight color and geometry feature extraction and fusion module for end-to-end 6D pose estimation

Although advancements in red–green–blue-depth (RGB-D)-based six degree-of-freedom (6D) pose estimation methods, severe occlusion remains challenging. Addressing this issue, we propose a novel feature fusion module that can efficiently leverage the color and geometry information in RGB-D images. Unlike prior fusion methods, our method employs a two-stage fusion process. Initially, we extract color features from RGB images and integrate them into a point cloud. Subsequently, an anisotropic separable set abstraction network-like network is utilized to process the fused point cloud, extracting both local and global features, which are then combined to generate the final fusion features. Furthermore, we introduce a lightweight color feature extraction network to reduce model complexity. Extensive experiments conducted on the LineMOD, Occlusion LineMOD, and YCB-Video datasets conclusively demonstrate that our method significantly enhances prediction accuracy, reduces training time, and exhibits robustness to occlusion. Further experiments show that our model is significantly smaller than the latest popular 6D pose estimation models, which indicates that our model is easier to deploy on mobile platforms.

Enhancing point cloud registration with transformer: cultural heritage protection of the Terracotta Warriors

Point cloud registration technology, by precisely aligning repair components with the original artifacts, can accurately record the geometric shape of cultural heritage objects and generate three-dimensional models, thereby providing reliable data support for the digital preservation, virtual exhibition, and restoration of cultural relics. However, traditional point cloud registration methods face challenges when dealing with cultural heritage data, including complex morphological and structural variations, sparsity and irregularity, and cross-dataset generalization. To address these challenges, this paper introduces an innovative method called Enhancing Point Cloud Registration with Transformer (EPCRT). Firstly, we utilize local geometric perception for positional encoding and combine it with a dynamic adjustment mechanism based on local density information and geometric angle encoding, enhancing the flexibility and adaptability of positional encoding to better characterize the complex local morphology and structural variations of artifacts. Additionally, we introduce a convolutional-Transformer hybrid module to facilitate interactive learning of artifact point cloud features, effectively achieving local–global feature fusion and enhancing detail capture capabilities, thus effectively handling the sparsity and irregularity of artifact point cloud data. We conduct extensive evaluations on the 3DMatch, ModelNet, KITTI, and MVP-RG datasets, and validate our method on the Terracotta Warriors cultural heritage dataset. The results demonstrate that our method has significant performance advantages in handling the complexity of morphological and structural variations, sparsity and irregularity of relic data, and cross-dataset generalization.

Digital Dental Biometrics for Human Identification Based on Automated 3D Point Cloud Feature Extraction and Registration.

Intraoral scans (IOS) provide precise 3D data of dental crowns and gingival structures. This paper explores an application of IOS in human identification. We propose a dental biometrics framework for human identification using 3D dental point clouds based on machine learning-related algorithms, encompassing three stages: data preprocessing, feature extraction, and registration-based identification. In the data preprocessing stage, we use the curvature principle to extract distinguishable tooth crown contours from the original point clouds as the holistic feature identification samples. Based on these samples, we construct four types of local feature identification samples to evaluate identification performance with severe teeth loss. In the feature extraction stage, we conduct voxel downsampling, then extract the geometric and structural features of the point cloud. In the registration-based identification stage, we construct a coarse-to-fine registration scheme in order to realize the identification task. Experimental results on a dataset of 160 individuals demonstrate that our method achieves a Rank-1 recognition rate of 100% using complete tooth crown contours samples. Utilizing the remaining four types of local feature samples yields a Rank-1 recognition rate exceeding 96.05%. The proposed framework proves effective for human identification, maintaining high identification performance even in extreme cases of partial tooth loss.

A novel method for the classification of 3D point clouds based on the improved PointNet++

Abstract In deep learning, point clouds are used as the primary input format for 3D data, which can provide detailed geometric information about objects in the original 3D space. PointNet++ is a deep learning network that uses point cloud data as an input format, which avoids the losses associated with the previous conversion of point cloud into 3D voxelization and a collection of 2D images. Although PointNet++ can directly process point cloud data in various ways, due to the disordered, irregular, and unevenly distributed nature of point cloud data, the effect of extracting point cloud features could be better. The large amount of point cloud data also leads to the training model falling into the local optimal solution, which affects the training results. In recent years, some effective methods and strategies have emerged to address these problems. In this thesis, three methods are proposed based on the PointNet++ network: feature similarity-based attention pooling, small kernel convolution, and diverse branch block method to improve the performance of the PointNet++ network. Experiments show that the improvement methods proposed in this paper effectively improve the feature extraction accuracy, which improves the accuracy of the PointNet++ network for classification on the ModelNet40_Normal_Resampled dataset, with an overall improvement of 1% compared with PointNet++.

FELC-SLAM: feature extraction and loop closure optimized lidar SLAM system

Abstract Simultaneous Localization and Mapping (SLAM) is one of the key technologies in robot navigation and autonomous driving, playing an important role in robot navigation. Due to the sparsity of LiDAR data and the singularity of point cloud features, accuracy loss of LiDAR SLAM can occur during point cloud matching and localization. In response to these issues, this paper proposes a LiDAR Measurement SLAM algorithm that integrates multi type geometric feature extraction and optimized point cloud registration algorithms. This article first adopts advanced ground segmentation methods and feature segmentation strategies, including ground features, edge features, planar features, and spherical features, to improve matching accuracy. In addition, this article improves the previous method for extracting edge and planar features, extracting clearer and more robust line and surface features to address the degradation of geometric features. Finally, by introducing a robust decoupling global registration method for loop closure detection in the backend of the system, the sparsity problem of distant point clouds and the degradation problem caused by the reduction of inner layers in point cloud registration were effectively solved. In the evaluation of the KITTI dataset, our algorithm reduced absolute trajectory error values by 60%, 29%, and 71% compared to LeGO-LOAM in multi loop and feature constrained scenarios (such as sequences 00, 01, and 02), respectively. The evaluation of the M2DGR and Botanic Garden datasets also indicates that the positioning accuracy of our algorithm is superior to other advanced LiDAR SLAM algorithms.

Graph Transformer for 3D point clouds classification and semantic segmentation

Recently, graph-based and Transformer-based deep learning have demonstrated excellent performances on various point cloud tasks. Most of the existing graph-based methods rely on static graph, which take a fixed input to establish graph relations. Moreover, many graph-based methods apply maximizing and averaging to aggregate neighboring features, so that only a single neighboring point affects the feature of centroid or different neighboring points own the same influence on the centroid’s feature, which ignoring the correlation and difference between points. Most Transformer-based approaches extract point cloud features based on global attention and lack the feature learning on local neighbors. To solve the above issues of graph-based and Transformer-based models, we propose a new feature extraction block named Graph Transformer and construct a 3D point cloud learning network called GTNet to learn features of point clouds on local and global patterns. Graph Transformer integrates the advantages of graph-based and Transformer-based methods, and consists of Local Transformer that use intra-domain cross-attention and Global Transformer that use global self-attention. Finally, we use GTNet for shape classification, part segmentation and semantic segmentation tasks in this paper. The experimental results show that our model achieves good learning and prediction ability on most tasks. The source code and pre-trained model of GTNet will be released on https://github.com/NWUzhouwei/GTNet.

A transfer learning-based network model integrating kernel convolution with graph attention mechanism for point cloud segmentation of livestock

Non-contact body size measurement has become a hot research topic in intelligent livestock farming. In regard to body size measurement of large livestock, such as cattle, collecting a substantial number of point clouds is frequently involved. The direct calculation of all point clouds for body size measurement can be impacted as point clouds of different body parts may interfere with each other, which poses huge challenges for the positioning of key points and induces inaccurate positioning, resulting in measurement errors. The accuracy of body size measurement can be improved by segmenting point clouds of different body parts from each other, key measurement points can be precisely located, thus enhancing the accuracy of body size measurement. In this paper, we propose a network model initially trained for pig point cloud segmentation based on the Kernel Convolution integrated with Graph Attention Mechanism (KCGATNet for short), which, through transfer learning techniques, can also be used to achieve successful segmentation of various cattle point clouds using only 7 training samples. The model utilizes two core modules, Kernel Convolution (KC) and Point-based Graph Attention Mechanism (P-GAT), to extract local neighborhood features of point clouds. When using pig body point clouds as training data, it achieved precise segmentation of the head, ears, limbs, torso, and tail of pigs through a downsampling-upsampling architecture. On the test set of pig point clouds, Overall Accuracy (OA) reached 98.1% and mean Intersection over Union (mIoU) was up to 90.5%. Furthermore, when this model served as a pre-trained model and underwent transfer learning using 7 sets of annotated data of Simmental cattle, it achieved a mIoU of 90.1% on a test set of 93 Simmental cattle, 89.6% on a test set of 439 dairy buffalo, 90.2% on a test set of 103 Hereford cattle, and 90.0% on a test set of 119 Black Angus cattle. The experimental outcomes fully demonstrate the robustness of the proposed livestock point cloud segmentation model, KCGATNet. With transfer learning of a small sample size, it can reliably perform point cloud segmentation on a wide range of different breeds of quadrupedal livestock, saving a significant amount of time spent on manual annotation and improving the efficiency of livestock point cloud segmentation models.

TSKPD: twin structure key point detection in point cloud

The point cloud keypoint detection algorithm like USIP that uses downsampling first and then fine-tuning the sampling points cannot effectively detect the defect part of the single view defect point cloud, resulting in the inability to output the keypoints of the defect part. Therefore, this paper proposes the twin structure key point detection algorithm named TSKPD based on the idea of contrastive learning, which uses two single-view defect point clouds to synthesize relatively more complete key points for learning, so as to promote the network model to learn the features of the complete point cloud. The robustness of key point detection of point cloud is effectively improved, and the detection of single view defect point cloud is realized. The test results on ModelNet40 and ShapeNet datasets show that the coverage rate of TSKPD on the missing part of the single view defect point cloud is 12.62 higher than the existing optimal algorithm.