Large-scale Point Clouds Research Articles

An Efficient and Stable Registration Framework for Large Point Clouds at Two Different Moments

Point cloud registration plays a great role in many application scenarios; however, the registration of large-scale point clouds for actual different moments suffers from the problems of low efficiency, low accuracy, and a lack of stability. In this paper, we propose a registration framework for large-scale point clouds at different moments, which firstly downsamples large-scale point clouds using a random sampling method, then performs a random expansion strategy to make up for the loss of information caused by the random sampling, then completes the first registration by a deep learning network based on the extraction of keypoints and feature descriptors in combination with RANSAC, and finally completes the registration using the point-to-point ICP method. We conducted validation experiments and application experiments on large-scale point clouds of key train components, and the experimental results are much higher in accuracy or efficiency than other methods, which proves the effectiveness of our framework, which can be applied to actual large-scale point clouds.

Underwater point cloud transmission framework: hybrid encoder implementation based on CNN and transformer

Abstract In underwater multi-robot systems, 3D point cloud data generated by sonar and depth sensors facilitates the execution of more complex collaborative tasks among robots, which partly rely on efficient data transmission. In this work, we propose a hybrid encoder framework based on convolutional neural network and Transformer for underwater point cloud transmission, aimed at dealing with large-scale point cloud data. Our encoder allows setting lower compression rate on regions or objects of interest for semantic point clouds, preserving crucial information in the point cloud. For underwater acoustic communication, we employ orthogonal frequency division multiplexing combined with deep joint source-channel coding for transmission to enhance the system’s error-resilience. Compared to state-of-the-art methods in the simulation experiments, our end-to-end framework achieves considerable compression performance while eliminating certain cliff and leveling effects, also demonstrating robustness even with changing channels.

3D Voxel-based Path Planning for AVs in Dynamic Complex Environments

Abstract. With the rapid advancement of science and technology, autonomous driving technology has been widely applied in various fields such as the automotive industry, intelligent warehousing, and emergency rescue. Traditional 2D path planning has limitations in dealing with the complex and dynamic environments of the real world, making it difficult to meet the high precision and complex path planning requirements of autonomous vehicles. With the development of data acquisition technology, large-scale point cloud data has become easily accessible, which can better extract the three-dimensional complex of the environment. Nevertheless, the unstructured nature and vast amount of point cloud data make path planning on it extremely challenging. Voxel models can effectively compress point cloud data and provide neighboring topological structures. In view of this, we propose a voxel-based framework for the path planning of autonomous vehicles in dynamic and complex 3D environments, which involves converting point cloud data into voxels and extracting navigation spaces. Subsequently, deep reinforcement learning techniques are utilized to achieve obstacle avoidance and navigation on the voxel map. It is expected that this framework will contribute to the development of 3D path planning and enhance the utilization of point cloud data in the navigation field.

High-Visibility Edge-Highlighting Visualization of 3D Scanned Point Clouds Based on Dual 3D Edge Extraction

Recent advances in 3D scanning have enabled the digital recording of complex objects as large-scale point clouds, which require clear visualization to convey their 3D shapes effectively. Edge-highlighting visualization is used to improve the comprehensibility of complex 3D structures by enhancing the 3D edges and high-curvature regions of the scanned objects. However, traditional methods often struggle with real-world objects due to inadequate representation of soft edges (i.e., rounded edges) and excessive line clutter, impairing resolution and depth perception. To address these challenges, we propose a novel visualization method for 3D scanned point clouds based on dual 3D edge extraction and opacity–color gradation. Dual 3D edge extraction separately identifies sharp and soft edges, integrating both into the visualization. Opacity–color gradation enhances the clarity of fine structures within soft edges through variations in color and opacity, while also creating a halo effect that improves both resolution and depth perception of the visualized edges. Computation times required for dual 3D edge extraction are comparable to conventional binary statistical edge-extraction methods. Visualizations with opacity–color gradation are executable at interactive rendering speeds. The effectiveness of the proposed method is demonstrated using 3D scanned point cloud data from high-value cultural heritage objects.

Rock discontinuity extraction from 3D point clouds using pointwise clustering algorithm

Recognizing discontinuities within rock masses is a critical aspect of rock engineering. The development of remote sensing technologies has significantly enhanced the quality and quantity of the point clouds collected from rock outcrops. In response, we propose a workflow that balances accuracy and efficiency to extract discontinuities from massive point clouds. The proposed method employs voxel filtering to downsample point clouds, constructs a point cloud topology using K-d trees, utilizes principal component analysis to calculate the point cloud normals, and employs the pointwise clustering (PWC) algorithm to extract discontinuities from rock outcrop point clouds. This method provides information on the location and orientation (dip direction and dip angle) of the discontinuities, and the modified whale optimization algorithm (MWOA) is utilized to identify major discontinuity sets and their average orientations. Performance evaluations based on three real cases demonstrate that the proposed method significantly reduces computational time costs without sacrificing accuracy. In particular, the method yields more reasonable extraction results for discontinuities with certain undulations. The presented approach offers a novel tool for efficiently extracting discontinuities from large-scale point clouds.

GRNet:Geometry Restoration for G-PCC Compressed Point Clouds Using Auxiliary Density Signaling.

The lossy Geometry-based Point Cloud Compression (G-PCC) inevitably impairs the geometry information of point clouds, which deteriorates the quality of experience (QoE) in reconstruction and/or misleads decisions in tasks such as classification. To tackle it, this work proposes GRNet for the geometry restoration of G-PCC compressed large-scale point clouds. By analyzing the content characteristics of original and G-PCC compressed point clouds, we attribute the G-PCC distortion to two key factors: point vanishing and point displacement. Visible impairments on a point cloud are usually dominated by an individual factor or superimposed by both factors, which are determined by the density of the original point cloud. To this end, we employ two different models for coordinate reconstruction, termed Coordinate Expansion and Coordinate Refinement, to attack the point vanishing and displacement, respectively. In addition, 4-byte auxiliary density information is signaled in the bitstream to assist the selection of Coordinate Expansion, Coordinate Refinement, or their combination. Before being fed into the coordinate reconstruction module, the G-PCC compressed point cloud is first processed by a Feature Analysis Module for multiscale information fusion, in which kNN-based Transformer is leveraged at each scale to adaptively characterize neighborhood geometric dynamics for effective restoration. Following the common test conditions recommended in the MPEG standardization committee, GRNet significantly improves the G-PCC anchor and remarkably outperforms state-of-the-art methods on a great variety of point clouds (e.g., solid, dense, and sparse samples) both quantitatively and qualitatively. Meanwhile, GRNet runs fairly fast and uses a smaller-size model when compared with existing learning-based approaches, making it attractive to industry practitioners.

VPA-Net: A visual perception assistance network for 3d lidar semantic segmentation

The semantic segmentation of 3D point clouds holds paramount importance in visual perception tasks of automatic driving, including obstacle avoidance, decision control, path planning, and map construction, etc. Multi-sensor fusion is a rational and pivotal technique for implementing LiDAR semantic segmentation. However, effectively fusing and utilizing multi-source data remains a challenging task. In this work, we introduce a novel visual perception-assisted point cloud segmentation network, termed VPA-Net. This network architecture employs a dual-branch design to effectively combine spatial information from point clouds and visual cues from images, thereby bolstering the performance of 3D LiDAR semantic segmentation. More specifically, the dual-branch network structure processes and fuses multi-modal data from both point clouds and RGB images. Subsequently, the intermediate features from the two branches are merged via the proposed attention-based feature fusion module. Furthermore, to address the challenge of precise boundary prediction in large-scale point cloud scene segmentation, we introduce a refinement module based on 3D sparse convolution to enhance the spatial information of the LiDAR point clouds. The effectiveness of our method is validated on SemanticKITTI and a more challenging 3D semantic segmentation dataset. Experimental results underscore significant improvement on SemanticKITTI, with our approach surpassing the state-of-the-art method, achieving a 7.1% higher mIoU than SalsaNext.

LESA-Net: Semantic segmentation of multi-type road point clouds in complex agroforestry environment

Point-cloud semantic segmentation is a visual task essential for agricultural robots to comprehend natural agroforestry environments. However, owing to the extremely large amount of point-cloud data in agroforestry environments, learning effective features for semantic segmentation from large-scale point clouds is challenging. Therefore, to address this issue and achieve accurate semantic segmentation of different types of road-surface point clouds in large-scale agroforestry environments, this study proposes a point-cloud semantic segmentation network framework based on double-distance self-attention. First, a point-cloud local feature enhancement module is proposed. This module primarily extends the receptive field and enhances the generalizability of multidimensional features by incorporating reflection intensity information and a spatial feature-encoding block that is enhanced with contextual semantic information. Second, we introduce a dual-distance attention pooling (DDAPS) block based on the self-attention mechanism. This block initially learns the feature representation of the local neighborhood of each point through the self-attention mechanism. Then, it uses the DDAPS block to aggregate more discriminative local neighborhood point features. Finally, extensive experimental results on large-scale point-cloud datasets, SemanticKITTI and RELLIS-3D, demonstrate that our algorithm outperforms similar algorithms in large-scale agroforestry environments.

Fast supervoxel segmentation of connectivity median simulation based on Manhattan distance

Supervoxels provide a more natural and compact 3D point cloud representation, which is a fundamental problem in point cloud processing and has attracted extensive attention from many scholars. At present, although the supervoxel segmentation has achieved significant results, the processing efficiency still restricts its further processing in the face of large-scale point cloud processing. Therefore, this paper proposes a new supervoxel segmentation method for point clouds to achieve fast supervoxel over-segmentation of large-scale point clouds while maintaining good boundaries and accuracy. Firstly, the supervoxel segmentation is formulated as a subset selection problem. With the idea of greedy strategy, the interval quick sorting and region growing methods is designed to realize the rapid merging of subsets in the iterative process. Secondly, the Manhattan distance metric and the seed point median simulation method are proposed to enhance the precision of iterative optimization and improve the efficiency of point cloud supervoxel segmentation. Experimental results show that, compared with state-of-the-art supervoxel segmentation method, the proposed method achieves the best results under UE, GCE, BR and other indicators, while reducing the running time of the algorithm by 10.6% to 28.3%.

RAAFNet: Reverse Attention Adaptive Fusion Network for Large-Scale Point Cloud Semantic Segmentation

Point cloud semantic segmentation is essential for comprehending and analyzing scenes. However, performing semantic segmentation on large-scale point clouds presents challenges, including demanding high memory requirements, a lack of structured data, and the absence of topological information. This paper presents a novel method based on the Reverse Attention Adaptive Fusion network (RAAFNet) for segmenting large-scale point clouds. RAAFNet consists of a reverse attention encoder–decoder module, an adaptive fusion module, and a local feature aggregation module. The reverse attention encoder–decoder module is applied to extract point cloud features at different scales. The adaptive fusion module enhances fine-grained representation within multi-resolution feature maps. Furthermore, a local aggregation classifier is introduced, which aggregates the features of neighboring points to the center point in order to leverage contextual information and enhance the classifier’s perceptual capability. Finally, the predicted labels are generated. Notably, our method excels at extracting point cloud features across different dimensions and produces highly accurate segmentation results. Experimental results on the Semantic3D dataset achieved an overall accuracy of 89.9% and a mIoU of 74.4%.

Step feature line extraction from large-scale point clouds of open-pit mine based on structural characteristics

High-precision step feature lines play a crucial role in open-pit mine design, production scheduling, mining volume calculations, road network planning, and slope maintenance. Compared with the feature lines of the geometric model, step feature lines are more irregular, complex, higher in density, and richer in detail. In this study, a novel technique for extracting step feature line from large-scale point clouds of open-pit mine by leveraging structural attributes, that is, SFLE_OPM (Step Feature Line Extraction for Open-Pit Mine), is proposed. First, we adopt the k-dimensional tree (KD-tree) resampling method to reduce the point-cloud density while retaining point-cloud features and utilize bilateral filtering for denoising. Second, we use Point Cloud Properties Network (PCPNET) to estimate the normal, calculate the slope and aspect, and then filter them. We then apply morphological operations to the step surface and obtain more continuous and smoother slope lines. In addition, we construct an Open-Pit Mine Step Feature Line (OPMSFL) dataset and benchmarked SFLE_OPM, achieving an accuracy score of 89.31% and true positive rate score of 80.18%. The results demonstrate that our method yields a higher extraction accuracy and precision than most of the existing methods. Our dataset is available at https://github.com/OPMDataSets/OPMSFL.

FVDB : A Deep-Learning Framework for Sparse, Large Scale, and High Performance Spatial Intelligence

We present f VDB, a novel GPU-optimized framework for deep learning on large-scale 3D data. f VDB provides a complete set of differentiable primitives to build deep learning architectures for common tasks in 3D learning such as convolution, pooling, attention, ray-tracing, meshing, etc. f VDB simultaneously provides a much larger feature set (primitives and operators) than established frameworks with no loss in efficiency: our operators match or exceed the performance of other frameworks with narrower scope. Furthermore, f VDB can process datasets with much larger footprint and spatial resolution than prior works, while providing a competitive memory footprint on small inputs. To achieve this combination of versatility and performance, f VDB relies on a single novel VDB index grid acceleration structure paired with several key innovations including GPU accelerated sparse grid construction, convolution using tensorcores, fast ray tracing kernels using a Hierarchical Digital Differential Analyzer algorithm (HDDA), and jagged tensors. Our framework is fully integrated with PyTorch enabling interoperability with existing pipelines, and we demonstrate its effectiveness on a number of representative tasks such as large-scale point-cloud segmentation, high resolution 3D generative modeling, unbounded scale Neural Radiance Fields, and large-scale point cloud reconstruction.

Semantic segmentation for large-scale point clouds based on hybrid attention and dynamic fusion

This paper investigates the semantic segmentation problem for large-scale point clouds. Recent segmentation methods usually employ an encoder–decoder architecture. However, these methods may not effectively extract neighboring information in the encoder. Additionally, they typically use nearest neighbor interpolation and skip connections in the decoder, overlooking the semantic gap between encoder and decoder features. To resolve these issues, we propose HADF-Net, which consists of a Hybrid Attention Encoder (HAE), an Edge Dynamic Fusion module (EDF), and a Dynamic Cross-attention Decoder (DCD). HAE leverages the distinctive properties of geometric and semantic relations to aggregate local features at different stages. EDF aims to alleviate information loss during decoder upsampling by dynamically integrating the neighboring information. DCD employs an enhanced fusion mechanism with spatial-wise cross-attention to bridge the semantic gap between encoder and decoder features. Experimental results on 4 datasets demonstrate that our HADF-Net achieves superior performance.

Automated geometric reconstruction and cable force inference for cable-net structures using 3D point clouds

Laser scanning provides an efficient solution to digital twin construction in civil engineering. The complexity and redundancy of large-scale point clouds substantially prolong the labor-intensive model reconstruction process. This paper presents an automated and high-precision geometric reconstruction approach for cable-net structures with a complete workflow from raw points to the extraction of cable shapes and forces. The strategy involves the extraction of coordinates of key cable regions through the characterization of cable geometry and local registration methods, followed by the computation of cable forces and shapes using segmented catenary theory and nonlinear optimization. The approach is validated with a single-curvature cable-net structure, with the accuracy of cable shapes within ±5 mm and errors in cable forces <5%. The method contributes to health monitoring and rapid reconstruction of cable-net structures. The precise geometry and forces will facilitate the creation of the mechanical models reflecting the physical authenticity of structures.

Efficient Calculation of Multi-Scale Features for MMS Point Clouds

Abstract. Point clouds acquired by Mobile Mapping System (MMS) are useful for creating 3D maps that can be used for autonomous driving and infrastructure development. However, many applications require semantic labels to each point of the point clouds, and the manual labeling process is very time consuming and expensive. Therefore, there is a strong need to develop a method to automatically assigning semantic labels. For automatic labeling tasks, classification methods using multiscale features are effective because multiscale features include features of various scales of roadside objects. Multiscale features are calculated using points inside spheres of multiscale radii centered at each point in a point cloud. When calculating multiscale features that are useful for classifying MMS point clouds, it is necessary to calculate features using relatively large radii. However, when calculating multiscale features using wide range of neighbor points, existing methods, such as kd-tree, require unacceptably long computation time for neighbor search. In this paper, we propose a method to calculate multiscale features in practical time for semantic labeling of large-scale point clouds. In our method, an MMS point cloud is first divided into small spherical regions. Then, radius search using multiscale radii is performed, and multiscale features are calculated using those neighbor points. Our experimental results showed that our method achieved significantly faster computational performance than conventional methods, and multiscale features could be calculated from large-scale point clouds in practical time.

Application of Machine Learning to Detect Building Points in Photogrammetry-based Point Clouds

Different point cloud technologies such as Terrestrial Laser Scanners (TLS), Airborne Laser Scanners (ALS), Mobile Mapping Systems (MMS), and Unmanned Aerial Vehicles (UAV) have become increasingly more common in land surveying and geoinformatics over recent years. Thanks to these modern tools, experts can survey large areas cost-effectively with either high resolution or high accuracy. However, processing the point cloud, which consists of millions of points, can be a massive challenge. Manual processing of these large datasets can often be very time-consuming and hardware-demanding, and most of the time, only a limited part of the point cloud is used to derive the final products. The solution can be to automate the process as much as possible. Several advanced mathematical methods, especially Machine Learning (ML) algorithms, allow efficient automated processing of point clouds. This paper presents a processing chain to detect and separate building points from large-scale photogrammetry-based point clouds. The processing is based on the combination of Random Sample Consensus (RANSAC) and Machine Learning (ML) algorithms like Density-Based Spatial Clustering of Applications with Noise (DBSCAN) and Multi-Layer Perceptron (MLP). Presented methods were trained and tested on established and open available Heissigheim 3D (H3D) dataset to separate roof and vegetation points with over 90% accuracy in order to enhance the separation of building points on large-scale point clouds.

TCFAP-Net: Transformer-based Cross-feature Fusion and Adaptive Perception Network for large-scale point cloud semantic segmentation

Point cloud semantic segmentation is an ingredient in understanding real-world scenes. Most existing approaches perform poorly on scene boundaries and struggle with recognizing objects of different scales. In this paper, we propose a novel framework that incorporates Transformer into the U-Net architecture for inferring pointwise semantics. Specifically, the Transformer-based cross-feature fusion module is designed first to employ geometric and semantic information to learn feature offsets to overcome the border ambiguity of segmentation results, and then it utilizes the Transformer to learn cross-feature enhanced and fused encoder features. Additionally, to facilitate the overall network’s structure-to-detail perception capabilities, the adaptive perception module is designed, which employs cross-attention to adaptively allocate weights to encoder features at varying resolutions, establishing long-range contextual dependencies. Ablation studies validate the individual contributions of our module design choices. Compared with the existing competitive methods, our approach achieves state-of-the-art performance and exhibits superior results on benchmarks. Code is available at https://github.com/xiluo-cug/TCFAP-Net.

Building Extraction from LiDAR Point Clouds Based on Revised RandLA-Net

Abstract. 3D building models is crucial for applications in smart cities. Automatic reconstruction of 3D buildings has been investigated based on various data sources. Point clouds from airborne LiDAR scanners can be used to extract buildings data due to its high accuracy and point density. In this paper, we present a methodology to segment buildings and corresponding rooftop structure from point clouds. First, RandLA-Net, which is an efficient and lightweight neural network for semantic segmentation of large-scale point clouds, is revised and adopted for building segmentation. By implementing local feature aggregation of each point, RandLA-Net can effectively preserve geometric details in point clouds. Besides 3D coordinates of point clouds, we incorporated point attributes including pulse intensity and return numbers into the network as additional features. Feature normalizations are applied to the input features. To achieve a better result of the local feature aggregation, hyperparameters of the network are fine-tuned according to the density of points and building size. Based on the classified building point clouds, DBSCAN clustering algorithm is implemented for segmenting individual buildings. Elevation histogram analysis is conducted to determine optimal threshold values for delineating candidate rooftop point clouds of individual buildings. For the buildings with multiple rooftops, multiple elevation threshold values are necessary to extract corresponding rooftops or walls. Then DBSCAN is employed again for segmentation of individual rooftops and denoising of point clouds of each building. Finally, Alpha-shape analysis is applied based on adaptive threshold values to build the envelope of each rooftop. Experiments show that our implementation of building segmentation using RandLA-net achieves higher mean IoU (Intersection over Union) and better classification performance in building segmentation. ISPRS benchmark data was used in our experiment and our methodology produce results with accuracy of 90.79%.

Global Localization in Large-Scale Point Clouds via Roll-Pitch-Yaw Invariant Place Recognition and Low-Overlap Global Registration

Global Localization in Large-Scale Point Clouds via Roll-Pitch-Yaw Invariant Place Recognition and Low-Overlap Global Registration

Semantic segmentation of large-scale segmental lining point clouds using 3D deep learning

It is highly challenging to comprehensively measure the deformations of large-scale tunnel structures because of the numerous individual segments of tunnel linings. Laser scanning and 3D deep learning (DL) can provide abundant geometric data and the ability to interpret such data automatically. To fill the gap in applying 3D DL to the segment-wise segmentation of large-scale tunnel point clouds, a new 3D dataset and an optimized DL network are presented in this paper. The point cloud dataset was manually annotated by labeling every individual segment and element. Subsequently, a 3D DL network capable of handling large-scale point clouds (approximately 106 points) was developed to perform the data experiments. The influences of network parameters, namely, the number of neighbor points, layer number, down-sampling ratio, and channel number, were investigated in the data experiments. The results indicate that the number of neighboring points and channel number have a notable influence on the segmentation performance, with the best mean interaction over union reaching 85.5%. This study provides preliminary insights into the 3D DL network design for large-scale tunnel point clouds.