Point Cloud Features Research Articles

Is it possible to generate accurate 3D point clouds with UAS-LIDAR and UAS-RGB photogrammetry without GCPs? A case study on a beach and rocky cliff

ContextRecently, Unoccupied Aerial Systems (UAS) with photographic or Light Detection and Ranging (LIDAR) sensors have incorporated on-board survey-grade Global Navigation Satellite Systems that allow the direct georeferencing of the resulting datasets without Ground Control Points either in Real-Time (RTK) or Post-Processing Kinematic (PPK) modes. These approaches can be useful in hard-to-reach or hazardous areas. However, the resulting 3D models have not been widely tested, as previous studies tend to evaluate only a few points and conclude that systematic errors can be found.ObjectivesWe test the absolute positional accuracy of point clouds produced using UAS with direct-georeferencing systems.MethodsWe test the accuracy and characteristics of point clouds produced using a UAS-LIDAR (with PPK) and a UAS-RGB (Structure-from-Motion or SfM photogrammetry with RTK and PPK) in a challenging environment: a coastline with a composite beach and cliff. The resulting models of each processing were tested using as a benchmark a point cloud surveyed simultaneously by a Terrestrial Laser Scanner.ResultsThe UAS-LIDAR produced the most accurate point cloud, with homogeneous cover and no noise. The systematic bias previously observed in the UAS-RGB RTK approaches are minimized using oblique images. The accuracy observed across the different surveyed landforms varied significantly.ConclusionsThe UAS-LIDAR and UAS-RGB with PPK produced unbiased point clouds, being the latter the most cost-effective method. For the other direct georeferencing systems/approaches, the acquisition of GCP or the co-registration of the resulting point cloud is still necessary.

GLDBF: Global and local dual-branch fusion network for no-reference point cloud quality assessment

No-reference Point Cloud Quality Assessment (NR-PCQA) is a challenge in the field of media quality assessment, such as inability to accurately capture quality-related features due to the unique scattered structure of points and less considering global features and local features jointly in the existing no-reference PCQA metrics. To address these challenges, we propose a Global and Local Dual-Branch Fusion (GLDBF) network for no-reference point cloud quality assessment. Firstly, sparse convolution is used to extract the global quality feature of distorted Point Clouds (PCs). Secondly, graph weighted PointNet++ is proposed to extract the multi-level local features of point cloud, and the offset attention mechanism is further used to enhance local effective features. Transformer-based fusion module is also proposed to fuse multi-level local features. Finally, we joint the global and local dual branch fusion modules via multilayer perceptron to predict the quality score of distorted PCs. Experimental results show that the proposed algorithm can achieves state-of-the-art performance compared with existing methods in assessing the quality of distorted PCs.

DeployFusion: A Deployable Monocular 3D Object Detection with Multi-Sensor Information Fusion in BEV for Edge Devices.

To address the challenges of suboptimal remote detection and significant computational burden in existing multi-sensor information fusion 3D object detection methods, a novel approach based on Bird's-Eye View (BEV) is proposed. This method utilizes an enhanced lightweight EdgeNeXt feature extraction network, incorporating residual branches to address network degradation caused by the excessive depth of STDA encoding blocks. Meantime, deformable convolution is used to expand the receptive field and reduce computational complexity. The feature fusion module constructs a two-stage fusion network to optimize the fusion and alignment of multi-sensor features. This network aligns image features to supplement environmental information with point cloud features, thereby obtaining the final BEV features. Additionally, a Transformer decoder that emphasizes global spatial cues is employed to process the BEV feature sequence, enabling precise detection of distant small objects. Experimental results demonstrate that this method surpasses the baseline network, with improvements of 4.5% in the NuScenes detection score and 5.5% in average precision for detection objects. Finally, the model is converted and accelerated using TensorRT tools for deployment on mobile devices, achieving an inference time of 138 ms per frame on the Jetson Orin NX embedded platform, thus enabling real-time 3D object detection.

Deep Plug-and-Play Non-Iterative Cluster for 3D Global Feature Extraction

Efficient and accurate point cloud feature extraction is crucial for critical tasks such as 3D recognition and semantic segmentation. However, existing global feature extraction methods for 3D data often require designing different models for different input types (point clouds, voxels, and maps). This article proposes an efficient plug-and-play non-iterative clustering method (NICM) to establish a unified point cloud global feature extraction paradigm suitable for any input type to solve the above problems. The core idea of the NICM is to construct the connection between a single point and other global points based only on the cosine similarity between center points to achieve global feature extraction, which has linear complexity characteristics and can be combined with any existing feature extraction model. Additionally, to better integrate the features extracted by NICM and the original model, this article designs an adaptive feature fusion module is designed based on the gate unit, which retains similar features and effectively fuses dissimilar features based on their importance to downstream tasks. We have applied our method to downstream tasks such as point cloud recognition, part segmentation and scene segmentation. Sufficient experiments have proven that our method can provide comprehensive and robust features for the original model, and effectively improve the performance of downstream tasks.

MSSA: Multi-Representation Semantics-Augmented Set Abstraction for 3D Object Detection

Accurate recognition and localization of 3D objects is a fundamental research problem in 3D computer vision. Benefiting from transformation-free point cloud processing and flexible receptive fields, point-based methods have become accurate in 3D point cloud modeling, but still fall behind voxel-based competitors in 3D detection. We observe that the set abstraction module, commonly utilized by point-based methods for downsampling points, tends to retain excessive irrelevant background information, thus hindering the effective learning of features for object detection tasks. To address this issue, we propose MSSA, a Multi-representation Semantics-augmented Set Abstraction for 3D object detection. Specifically, we first design a backbone network to encode different representation features of point clouds, which extracts point-wise features through PointNet to preserve fine-grained geometric structure features, and adopts VoxelNet to extract voxel features and BEV features to enhance the semantic features of key points. Second, to efficiently fuse different representation features of keypoints, we propose a Point feature-guided Voxel feature and BEV feature fusion (PVB-Fusion) module to adaptively fuse multi-representation features and remove noise. At last, a novel Multi-representation Semantic-guided Farthest Point Sampling (MS-FPS) algorithm is designed to help set abstraction modules progressively downsample point clouds, thereby improving instance recall and detection performance with more important foreground points. We evaluate MSSA on the widely used KITTI dataset and the more challenging nuScenes dataset. Experimental results show that compared to PointRCNN, our method improves the AP of “moderate” level for three classes of objects by 7.02%, 6.76%, and 5.44%, respectively. Compared to the advanced point-voxel-based method PV-RCNN, our method improves the AP of “moderate” level by 1.23%, 2.84%, and 0.55% for the three classes, respectively.

Multi-feature enhancement based on sparse networks for single-stage 3D object detection

In the field of autonomous driving, the accuracy and real-time requirements for 3D object detection technology continue to improve, which is directly related to the commercialization process and market popularity of autonomous vehicles. Despite the efficiency of pillar-based coding for onboard systems, it falls short in terms of accuracy and the reduction of incorrect positives. In this paper, we will examine how to solve the problem of high incorrect rate and low accuracy of existing methods. Firstly, a MAP coding module is introduced to optimize previous point cloud feature coding modules, allowing for the efficient extraction of fine-grained features from point cloud data. Then, we introduce an innovative sparse dual attention (SDA) to efficiently filter out irrelevant details in feature extraction, thereby improving the pertinence and efficiency of information extraction. Finally, to address the potential loss of information from single local feature extraction, a local and global fusion module (CTGC) is introduced. Our method has proactively demonstrated its efficiency and accuracy through rigorous experimentation across diverse datasets. Analysis of the results leads to the conclusion that our solutions provide accurate and robust detection results. Code will be available at https://github.com/lcy199905/MyOpenPCDet.git.

EIDU-Net: edge-preserved inception DenseGCN U-Net for LiDAR point cloud segmentation

With the development of laser scanners and machine learning, point cloud semantic segmentation plays a significant role in autonomous driving, scene reconstruction, human-computer interaction, and other fields. In recent years, point cloud semantic segmentation based on deep learning has become one of the key research directions in point cloud processing. Due to the limited ability to exploit geometric details and contextual information in point clouds, most methods that adopt encoder-decoder architecture lose local structural information easily, especially detailed features, and extract features insufficiently. To address this issue, the edge-preserving inception DenseGCN U-Net (entitled as EIDU-Net) is proposed. EIDU-Net makes full use of the complementation between geometric details in the original point cloud and high-level features. The edge-preserved graph pooling (EGP) layer, the key module of the EIDU-Net, is designed to retain additional edge feature information from the original point cloud during pooling operations. Accordingly, the edge-preserved graph unpooling (EGU) layer can restore the feature graph more efficiently based on the additionally retained edge features. Extensive experiments demonstrate that our proposed EIDU-Net has remarkable improvements on semantic segmentation tasks under whatever S3DIS or Terracotta Warrior fragments. Our code is publicly available at https://github.com/caoxin918/EIDU-Net.

Weakly supervised 3D point cloud semantic segmentation for architectural heritage using teacher-guided consistency and contrast learning

Point cloud semantic segmentation is significant for managing and protecting architectural heritage. Currently, fully supervised methods require a large amount of annotated data, while weakly supervised methods are difficult to transfer directly to architectural heritage. This paper proposes an end-to-end teacher-guided consistency and contrastive learning weakly supervised (TCCWS) framework for architectural heritage point cloud semantic segmentation, which can fully utilize limited labeled points to train network. Specifically, a teacher-student framework is designed to generate pseudo labels and a pseudo label dividing module is proposed to distinguish reliable and ambiguous point sets. Based on it, a consistency and contrastive learning strategy is designed to fully utilize supervision signals to learn the features of point clouds. The framework is tested on the ArCH dataset and self-collected point cloud, which demonstrates that the proposed method can achieve effective semantic segmentation of architectural heritage using only 0.1 % of annotated points.

High precision DSRC and LiDAR data integration positioning method for autonomous vehicles based on CNN

In order to improve the safe driving and automatic positioning capability of autonomous vehicles, a high-precision DSRC and LiDAR data integration positioning technology for autonomous vehicles based on CNN is proposed. Import the data of Dedicated Short Range Communications and Light Detection and Ranging for automatic driving vehicle positioning, carry out kinematic analysis of autonomous driving vehicles under multi-sensor fusion, and transform the data of DSRC and LiDAR sensors into tightly coupled coordinate systems; The CNN depth learning method is used to compensate the position and attitude tracking estimation error under the overall time stamp synchronization of the sensor through adaptive information tracking; The first and second order feedforward compensation is made for the positioning parameters of the autonomous driving vehicle using the PID model, and the point cloud feature matching model is fused to complete the estimation of the positioning attitude parameters of the autonomous driving vehicle. In order to eliminate the noise interference under the DSRC communication mechanism, the Kalman filter function is used to automatically optimize the constraint parameters in the point cloud feature detection model, and the positioning error parameters are dynamically filtered and adjusted; Kinematics analysis is carried out for the driving state of the vehicle, and the positioning error in the vehicle movement is controlled through the difference technology to achieve high-precision DSRC and LiDAR data integration positioning. The simulation results show that this method can integrate and locate the high-precision DSRC and LiDAR data of the autonomous vehicle, and the attitude estimation and positioning accuracy of the vehicle is good, while the error of the attitude parameter estimation of the autonomous vehicle is low.

GNSS/IMU/LO integration with a new LO error model and lateral constraint for navigation in urban areas

The quest for reliable vehicle navigation in urban environments has led the integration of Light Detection and Ranging (LiDAR) Odometry (LO) with Global Navigation Satellite Systems (GNSS) and Inertial Measurement Units (IMU). However, the performance of the integrated system is limited by a lack of accurate LO error modeling. In this paper, we propose a weighted GNSS/IMU/LO integration-based navigation system with a novel LO error model. The Squared Exponential Gaussian Progress Regression (SE-GPR) based LO error model is developed by considering the vehicle velocity and number of point cloud features. Based on error prediction for GNSS positioning and LO, a weighting strategy is designed for integration in an Extended Kalman Filter (EKF). Furthermore, error accumulation of the navigation state, especially in GNSS-challenging scenarios, is restrained by the LiDAR-Aided Lateral Constraint (LALC) and Non-Holonomic Constraint (NHC). An experiment was conducted in a deep urban area to test the proposed algorithm. The results show that the proposed algorithm delivers horizontal and three-dimensional (3D) positioning Root Mean Square Errors (RMSEs) of 3.669 m and 5.216 m, respectively. The corresponding accuracy improvements are 35.9% and 50.0% compared to the basic EKF based GNSS/IMU/LO integration.

Point cloud-based feature extraction and surface reconstruction for parts paring in high-precision assembly

Abstract Assembly plays a crucial role in industrial manufacturing in industrial manufacturing, but the efficiency of conventional manual methods for parts pairing is limited. Previous research has demonstrated the feasibility of deep learning for point cloud feature extraction and 3D reconstruction. An innovative method utilizing deep learning for high-precision feature extraction and surface reconstruction is introduced to optimize parts pairing in this paper. Geometric dimensions and surface topography data are obtained by defining key assembly features and utilizing the Random Sample Consensus method. Deep learning is then used to directly regress the Surface Distance Function from point samples, facilitating comprehensive part surface modeling and supporting assembly simulation in the digital twin. To validate this approach, a case study demonstrates successful matching of 30 shaft parts and 30 hole parts after optimization, with an increase in average uniformity by 0.024. This highlights the proposed method’s superior effectiveness and accuracy in feature extraction and surface reconstruction.

Global Attention-Guided Dual-Domain Point Cloud Feature Learning for Classification and Segmentation

Global Attention-Guided Dual-Domain Point Cloud Feature Learning for Classification and Segmentation

Non-contact measurement of conveyor belt speed based on fast point cloud registration of feature block

Abstract Non-contact, real-time measurement of conveyor belt speed is critical for energy-saving speed regulation and efficient development of coal mine conveyor systems. Existing speed measurement technologies for conveyor systems are often limited by the slippage and wear in contact measurement and complex environmental disturbance. This study introduces three-dimensional point cloud technology into coal flow information detection and innovatively presents a non-contact measurement method of conveyor belt speed based on fast point cloud registration of feature blocks. In the proposed method, a three-dimensional camera is used to capture point cloud data of the dynamically running conveyor belt, and the raw point cloud is preprocessed and segmented into blocks. Then, the C-MANV (curvature and mean angle of normal vectors) features of block point clouds are constructed based on the point cloud neighborhood curvature and the mean angle of neighborhood normal vectors. Finally, an improved block point cloud registration method based on C-MANV features is adopted to achieve the accurate measurement of the dynamic running speed of conveyor belt. Experimental results demonstrate that the proposed method achieves an average relative error of less than 1.8% in high-speed conveyor belt operation with an average processing time of less than 35 ms, which fulfils the accuracy and real-time requirements for conveyor belt speed detection in coal mines. This study provides an effective technical solution for the speed monitoring of coal mine conveyor systems.

TAMC: Textual Alignment and Masked Consistency for Open-Vocabulary 3D Scene Understanding.

Three-dimensional (3D) Scene Understanding achieves environmental perception by extracting and analyzing point cloud data with wide applications including virtual reality, robotics, etc. Previous methods align the 2D image feature from a pre-trained CLIP model and the 3D point cloud feature for the open vocabulary scene understanding ability. We believe that existing methods have the following two deficiencies: (1) the 3D feature extraction process ignores the challenges of real scenarios, i.e., point cloud data are very sparse and even incomplete; (2) the training stage lacks direct text supervision, leading to inconsistency with the inference stage. To address the first issue, we employ a Masked Consistency training policy. Specifically, during the alignment of 3D and 2D features, we mask some 3D features to force the model to understand the entire scene using only partial 3D features. For the second issue, we generate pseudo-text labels and align them with the 3D features during the training process. In particular, we first generate a description for each 2D image belonging to the same 3D scene and then use a summarization model to fuse these descriptions into a single description of the scene. Subsequently, we align 2D-3D features and 3D-text features simultaneously during training. Massive experiments demonstrate the effectiveness of our method, outperforming state-of-the-art approaches.

3D Object Detection via Residual SqueezeDet

Three-dimensional object detection is a critical task in computer vision with applications in autonomous driving, robotics, and augmented reality. This paper introduces Residual SqueezeDet, a novel network architecture that enhances the performance of 3D object detection on the KITTI dataset. Building upon the efficient SqueezeDet framework, we propose the Residual Fire module, which incorporates skip connections inspired by ResNet architectures into the original Fire module. This innovation improves gradient flow, enhances feature propagation, and allows for more effective training of deeper networks. Our method leverages point cloud and image-based features, employing a Residual SqueezeDet to effectively capture local and global context. Extensive experiments on the KITTI dataset demonstrate that Residual SqueezeDet significantly outperforms the original SqueezeDet, with particularly notable improvements in challenging scenarios. The proposed model maintains computational efficiency while achieving state-of-the-art performance, making it well-suited for real-time applications in autonomous driving. Our work contributes to the field by providing a more accurate and robust solution for 3D object detection, paving the way for improved perception systems in dynamic environments.

A method for accurate extraction of three-dimensional point cloud feature data in loading test of container ships

The accuracy of container slot dimensions directly affects the efficiency and safety of container loading and unloading. However, traditional methods of container loading tests need to be improved for their long cycles, high costs, and low production efficiency. With the application of three-dimensional laser scanning devices in ship manufacturing, the use of simulated container loading test methods based on point cloud becomes a breakthrough in key technology for container ship construction. This paper presents a multi-objective segmentation method of cargo hold feature point cloud for container ship simulated loading tests. Data preprocessing algorithms are applied to remove noise and pseudo-image, and then a Random Sample Consensus algorithm is used to remove bulkhead and cargo bottom point clouds. Based on the density-based spatial clustering of applications with noise algorithm and the region growing algorithm, an algorithm for multiple segmentation of guide rail and bottom cone point clouds is devised, and a four-plane segmentation of guide rail point clouds is realized. Finally, a method for three-dimensional point cloud data flatness assessment and a Hausdorff distance based method for calculating the guide rail lateral and longitudinal spacing are proposed. These methods are validated through the loading test of a 16,000 TEU container ship.

Approach angle estimation method for ships based on deep learning

Accurately estimating the ship's approach angle is crucial during berthing, or passing through navigational structures. It forms the foundation for ship navigation decisions. This study introduces the LiDAR point cloud-based ship pose perception network (PP-Net), a novel deep learning-based method for acquiring ship approach angles. Addressing the challenge of manually annotating approach angles, the ship pose perception method based on point cloud feature distribution (FD) was proposed to obtain initial angle estimates. The max-pooling function was utilized to address the disorder of point clouds, and a multi-resolution feature extractor was employed to enhance the network's robustness in extracting features from inconsistently dense point clouds. By using iterative farthest point sampling (IFPS), datasets with varying numbers of point clouds were standardized to a fixed number of points, thereby normalizing input dimensions and improving computational efficiency. Additionally, a novel loss function was proposed to improve the model's predictive accuracy. Experimental validation conducted in the scenario of ships entering ship lifts demonstrates the effectiveness of the proposed method. The outcomes can support safe navigation for ships in restricted waterways.

A lightweight color and geometry feature extraction and fusion module for end-to-end 6D pose estimation

Although advancements in red–green–blue-depth (RGB-D)-based six degree-of-freedom (6D) pose estimation methods, severe occlusion remains challenging. Addressing this issue, we propose a novel feature fusion module that can efficiently leverage the color and geometry information in RGB-D images. Unlike prior fusion methods, our method employs a two-stage fusion process. Initially, we extract color features from RGB images and integrate them into a point cloud. Subsequently, an anisotropic separable set abstraction network-like network is utilized to process the fused point cloud, extracting both local and global features, which are then combined to generate the final fusion features. Furthermore, we introduce a lightweight color feature extraction network to reduce model complexity. Extensive experiments conducted on the LineMOD, Occlusion LineMOD, and YCB-Video datasets conclusively demonstrate that our method significantly enhances prediction accuracy, reduces training time, and exhibits robustness to occlusion. Further experiments show that our model is significantly smaller than the latest popular 6D pose estimation models, which indicates that our model is easier to deploy on mobile platforms.

Enhancing point cloud registration with transformer: cultural heritage protection of the Terracotta Warriors

Point cloud registration technology, by precisely aligning repair components with the original artifacts, can accurately record the geometric shape of cultural heritage objects and generate three-dimensional models, thereby providing reliable data support for the digital preservation, virtual exhibition, and restoration of cultural relics. However, traditional point cloud registration methods face challenges when dealing with cultural heritage data, including complex morphological and structural variations, sparsity and irregularity, and cross-dataset generalization. To address these challenges, this paper introduces an innovative method called Enhancing Point Cloud Registration with Transformer (EPCRT). Firstly, we utilize local geometric perception for positional encoding and combine it with a dynamic adjustment mechanism based on local density information and geometric angle encoding, enhancing the flexibility and adaptability of positional encoding to better characterize the complex local morphology and structural variations of artifacts. Additionally, we introduce a convolutional-Transformer hybrid module to facilitate interactive learning of artifact point cloud features, effectively achieving local–global feature fusion and enhancing detail capture capabilities, thus effectively handling the sparsity and irregularity of artifact point cloud data. We conduct extensive evaluations on the 3DMatch, ModelNet, KITTI, and MVP-RG datasets, and validate our method on the Terracotta Warriors cultural heritage dataset. The results demonstrate that our method has significant performance advantages in handling the complexity of morphological and structural variations, sparsity and irregularity of relic data, and cross-dataset generalization.

Digital Dental Biometrics for Human Identification Based on Automated 3D Point Cloud Feature Extraction and Registration.

Intraoral scans (IOS) provide precise 3D data of dental crowns and gingival structures. This paper explores an application of IOS in human identification. We propose a dental biometrics framework for human identification using 3D dental point clouds based on machine learning-related algorithms, encompassing three stages: data preprocessing, feature extraction, and registration-based identification. In the data preprocessing stage, we use the curvature principle to extract distinguishable tooth crown contours from the original point clouds as the holistic feature identification samples. Based on these samples, we construct four types of local feature identification samples to evaluate identification performance with severe teeth loss. In the feature extraction stage, we conduct voxel downsampling, then extract the geometric and structural features of the point cloud. In the registration-based identification stage, we construct a coarse-to-fine registration scheme in order to realize the identification task. Experimental results on a dataset of 160 individuals demonstrate that our method achieves a Rank-1 recognition rate of 100% using complete tooth crown contours samples. Utilizing the remaining four types of local feature samples yields a Rank-1 recognition rate exceeding 96.05%. The proposed framework proves effective for human identification, maintaining high identification performance even in extreme cases of partial tooth loss.