Real-world Point Clouds Research Articles

RotInv-PCT: Rotation-Invariant Point Cloud Transformer via feature separation and aggregation.

The widespread use of point clouds has spurred the rapid development of neural networks for point cloud processing. A crucial property of these networks is maintaining consistent output results under random rotations of the input point cloud, namely, rotation invariance. The dominant approach achieves rotation invariance is to construct local coordinate systems for computing invariant local point cloud coordinates. However, this method neglects the relative pose relationships between local point cloud structures, leading to a decline in network performance. To address this limitation, we propose a novel Rotation-Invariant Point Cloud Transformer (RotInv-PCT). This method extracts the local abstract shape features of the point cloud using Local Reference Frames (LRFs) and explicitly computes the spatial relative pose features between local point clouds, both of which are proven to be rotation-invariant. Furthermore, to capture the long-range pose dependencies between points, we introduce an innovative Feature Aggregation Transformer (FAT) model, which seamlessly fuses the pose features with the shape features to obtain a globally rotation-invariant representation. Moreover, to manage large-scale point clouds, we utilize hierarchical random downsampling to gradually decrease the scale of point clouds, followed by feature aggregation through FAT. To demonstrate the effectiveness of RotInv-PCT, we conducted comparative experiments across various tasks and datasets, including point cloud classification on ScanObjectNN and ModelNet40, part segmentation on ShapeNet, and semantic segmentation on S3DIS and KITTI. Thanks to our provable rotation-invariant features and FAT, our method generally outperforms state-of-the-art networks. In particular, we highlight that RotInv-PCT achieved a 2% improvement in real-world point cloud classification tasks compared to the strongest baseline. Furthermore, in the semantic segmentation task, we improved the performance on the S3DIS dataset by 10% and, for the first time, realized rotation-invariant point cloud semantic segmentation on the KITTI dataset.

Verification of 3D Electrical Equipment Model Based on Cross-source Point Cloud Registration Using Deep Neural Netwo

With the popularization of digital twin techniques in power substations, assessment and verification of electrical equipment 3D models in digital twins according to as-built LiDAR point clouds become essential for the quality assurance of the designed substation models. However, computing the shape and texture differences between a 3D model and its corresponding point cloud is challenging due to the difficulty in aligning cross-source equipment point clouds with local geometric shape variations. In this paper, we propose a 3D model verification method based on overlap-aware cross-source point cloud registration. The key of the method is an overlap attention-based point cloud registration network with grouped KPConv, attention mechanism, and overlap-weighted circle loss. It improves the registration accuracy against local geometric shape variations between 3D models and LiDAR point clouds. In addition, due to the lack of real-world point cloud samples of electrical equipment, a novel point cloud augmentation method is employed for generating synthetic point clouds for improving the sim-to-real generalization capability of the network. Based on the pose alignment of the 3D model and the corresponding point cloud, a facet-level computing method is proposed for model differentiation and colorization. Experimental results using real-world point clouds of power substation equipment validate the performance of the proposed method.

Analysis Study of Mobile LiDAR Performance Degradation in Rainfall Based on Real—World Point Cloud Data

Analysis Study of Mobile LiDAR Performance Degradation in Rainfall Based on Real—World Point Cloud Data

ModelNet-O: A large-scale synthetic dataset for occlusion-aware point cloud classification

ModelNet-O: A large-scale synthetic dataset for occlusion-aware point cloud classification

Instance-Incremental Scene Graph Generation From Real-World Point Clouds via Normalizing Flows

Instance-Incremental Scene Graph Generation From Real-World Point Clouds via Normalizing Flows

DEVELOPING COMPLETE URBAN DIGITAL TWINS IN BUSY ENVIRONMENTS: A FRAMEWORK FOR FACILITATING 3D MODEL GENERATION FROM MULTI-SOURCE POINT CLOUD DATA

Abstract. The proliferation of affordable LiDAR technology and photogrammetry sensors has revolutionized 3D data acquisition in built environments, enabling comprehensive data capture from citywide scales to interior structures. This data can be transformed into digital twins, providing valuable resources for city planners, architects, engineers, and decision-makers. However, current studies often overlook the limitations of real-world point cloud datasets derived from LiDAR systems, which are voluminous, noisy, incomplete, and lacking information, which hinders monitoring, interpretation, and automated analysis. To address these challenges, methods are required to prepare point cloud data, ensuring accurate and reliable 3D representations. This research proposes a detailed framework for point cloud data preparation in busy urban environments. It includes precise algorithms, software, and parameter guidelines, allowing for the creation of comprehensive point cloud datasets. The framework has been successfully implemented on datasets acquired in Toronto, converting point cloud data from various platforms and parameters into an integrated dataset. Results demonstrate the framework's effectiveness in producing accurate and complete point cloud datasets for applications such as classification, information extraction, 3D model generation, and smart cities' monitoring and management.

Repurposing existing skeletal spatial structure (SkS) system designs using the Field Information Modeling (FIM) framework for generative decision-support in future construction projects

Skeletal spatial structure (SkS) systems are modular systems which have shown promise to support mass customization, and sustainability in construction. SkS have been used extensively in the reconstruction efforts since World War II, particularly to build geometrically flexible and free-form structures. By employing advanced digital engineering and construction practices, the existing SkS designs may be repurposed to generate new optimal designs that satisfy current construction demands of contemporary societies. To this end, this study investigated the application of point cloud processing using the Field Information Modeling (FIM) framework for the digital documentation and generative redesign of existing SkS systems. Three new algorithms were proposed to (i) expand FIM to include generative decision-support; (ii) generate as-built building information modeling (BIM) for SkS; and (iii) modularize SkS designs with repeating patterns for optimal production and supply chain management. These algorithms incorporated a host of new AI-inspired methods, including support vector machine (SVM) for decision support; Bayesian optimization for neighborhood definition; Bayesian Gaussian mixture clustering for modularization; and Monte Carlo stochastic multi-criteria decision making (MCDM) for selection of the top Pareto front solutions obtained by the non-dominant sorting Genetic Algorithm (NSGA II). The algorithms were tested and validated on four real-world point cloud datasets to solve two generative modeling problems, namely, engineering design optimization and facility location optimization. It was observed that the proposed Bayesian neighborhood definition outperformed particle swarm and uniform sampling by 34% and 27%, respectively. The proposed SVM-based linear feature detection outperformed k-means and spectral clustering by 56% and 9%, respectively. Finally, the NSGA II algorithm combined with the stochastic MCDM produced diverse “top four” solutions based on project-specific criteria. The results indicate promise for future utilization of the framework to produce training datasets for generative adversarial networks that generate new designs based only on stakeholder requirements.

TransPCGC: Point Cloud Geometry Compression Based on Transformers

Due to the often substantial size of the real-world point cloud data, efficient transmission and storage have become critical concerns. Point cloud compression plays a decisive role in addressing these challenges. Recognizing the importance of capturing global information within point cloud data for effective compression, many existing point cloud compression methods overlook this crucial aspect. To tackle this oversight, we propose an innovative end-to-end point cloud compression method designed to extract both global and local information. Our method includes a novel Transformer module to extract rich features from the point cloud. Utilization of a pooling operation that requires no learnable parameters as a token mixer for computing long-distance dependencies ensures global feature extraction while significantly reducing both computations and parameters. Furthermore, we employ convolutional layers for feature extraction. These layers not only preserve the spatial structure of the point cloud, but also offer the advantage of parameter independence from the input point cloud size, resulting in a substantial reduction in parameters. Our experimental results demonstrate the effectiveness of the proposed TransPCGC network. It achieves average Bjontegaard Delta Rate (BD-Rate) gains of 85.79% and 80.24% compared to Geometry-based Point Cloud Compression (G-PCC). Additionally, in comparison to the Learned-PCGC network, our approach attains an average BD-Rate gain of 18.26% and 13.83%. Moreover, it is accompanied by a 16% reduction in encoding and decoding time, along with a 50% reduction in model size.

Variational Relational Point Completion Network for Robust 3D Classification.

Real-scanned point clouds are often incomplete due to viewpoint, occlusion, and noise, which hampers 3D geometric modeling and perception. Existing point cloud completion methods tend to generate global shape skeletons and hence lack fine local details. Furthermore, they mostly learn a deterministic partial-to-complete mapping, but overlook structural relations in man-made objects. To tackle these challenges, this paper proposes a variational framework, Variational Relational point Completion network (VRCNet) with two appealing properties: 1) Probabilistic Modeling. In particular, we propose a dual-path architecture to enable principled probabilistic modeling across partial and complete clouds. One path consumes complete point clouds for reconstruction by learning a point VAE. The other path generates complete shapes for partial point clouds, whose embedded distribution is guided by distribution obtained from the reconstruction path during training. 2) Relational Enhancement. Specifically, we carefully design point self-attention kernel and point selective kernel module to exploit relational point features, which refines local shape details conditioned on the coarse completion. In addition, we contribute multi-view partial point cloud datasets (MVP and MVP-40 dataset) containing over 200,000 high-quality scans, which render partial 3D shapes from 26 uniformly distributed camera poses for each 3D CAD model. Extensive experiments demonstrate that VRCNet outperforms state-of-the-art methods on all standard point cloud completion benchmarks. Notably, VRCNet shows great generalizability and robustness on real-world point cloud scans. Moreover, we can achieve robust 3D classification for partial point clouds with the help of VRCNet, which can highly increase classification accuracy. Our project is available at https://paul007pl.github.io/projects/VRCNet.

Classification of structural building damage grades from multi-temporal photogrammetric point clouds using a machine learning model trained on virtual laser scanning data

Classification of structural building damage grades from multi-temporal photogrammetric point clouds using a machine learning model trained on virtual laser scanning data

Adaptive random sample consensus method for ground filtering of airborne LiDAR

Ground filtering is an essential step in the comprehensive processing of airborne LiDAR point clouds. However, the performances of existing ground filtering algorithms are usually affected by manual thresholds, and many algorithms have high complexity and are not suitable for applications with high timeliness requirements. In this paper, a fast ground filtering algorithm for airborne point clouds based on Random Sample Consensus (RANSAC) and adaptive threshold acquisition is proposed. Statistical filtering algorithm is used to filter out outliers and abnormal points based on the Z-scale sequence of the point clouds, the threshold is adaptively obtained according to the filtered Z-scale sequence, and then the adaptive threshold and RANSAC are combined to achieve rapid ground filtering. The proposed algorithm is used to perform threshold calculation and ground filtering on point clouds dataset and real-world point clouds, the results show that the adaptively obtained threshold is located in the optimal threshold interval, and the algorithm in this paper can extract ground points quickly and with small errors. The proposed method provides a reference algorithm for the airborne point clouds processing field that requires high timeliness, such as reconnaissance and Strike Integrated UAV, ground target identification and tracking, etc.

SAT3D: Slot Attention Transformer for 3D Point Cloud Semantic Segmentation

Semantic segmentation of 3D point cloud is a key task in numerous intelligent transportation system applications, e.g. self-driving vehicles, traffic monitoring. Due to the sparsity and varying density of points in the outdoor point clouds, it becomes particularly challenging to extract object-centric features from data. This leads to poor semantic segmentation, especially for the rare object classes. To address that, we introduce the first-ever Slot Attention Transformer based technique to effectively model object-centric features in point cloud data. Our method uses cylindrical splits of space for voxelization and computes channel-wise positional embeddings before repetitively encoding the point cloud with slot attentions. Our second major contribution is a Large-Scale Outdoor Point Cloud dataset (SWAN), collected in a dense urban environment, driving 150km distance. It provides 16 billion points in more than 200K frames. The dataset also provides annotations for 10K frames for 24 classes. We also contribute a data augmentation scheme to handle rare object classes in real-world point clouds. Besides benchmarking popular existing methods on SWAN for the first time, we thoroughly evaluate our technique on the existing large-scale datasets, Semantic KITTI and nuScenes. Our results demonstrate a consistent performance gain for our technique, and verify the need of the more challenging SWAN dataset.

AdaSplats: Adaptive Splatting of Point Clouds for Accurate 3D Modeling and Real-Time High-Fidelity LiDAR Simulation

LiDAR sensors provide rich 3D information about their surroundings and are becoming increasingly important for autonomous vehicles tasks such as localization, semantic segmentation, object detection, and tracking. Simulation accelerates the testing, validation, and deployment of autonomous vehicles while also reducing cost and eliminating the risks of testing in real-world scenarios. We address the problem of high-fidelity LiDAR simulation and present a pipeline that leverages real-world point clouds acquired by mobile mapping systems. Point-based geometry representations, more specifically splats (2D oriented disks with normals), have proven their ability to accurately model the underlying surface in large point clouds, mainly with uniform density. We introduce an adaptive splat generation method that accurately models the underlying 3D geometry to handle real-world point clouds with variable densities, especially for thin structures. Moreover, we introduce a fast LiDAR sensor simulator, working in the splatted model, that leverages the GPU parallel architecture with an acceleration structure while focusing on efficiently handling large point clouds. We test our LiDAR simulation in real-world conditions, showing qualitative and quantitative results compared to basic splatting and meshing techniques, demonstrating the interest of our modeling technique.

Reconstructing compact building models from point clouds using deep implicit fields

While three-dimensional (3D) building models play an increasingly pivotal role in many real-world applications, obtaining a compact representation of buildings remains an open problem. In this paper, we present a novel framework for reconstructing compact, watertight, polygonal building models from point clouds. Our framework comprises three components: (a) a cell complex is generated via adaptive space partitioning that provides a polyhedral embedding as the candidate set; (b) an implicit field is learned by a deep neural network that facilitates building occupancy estimation; (c) a Markov random field is formulated to extract the outer surface of a building via combinatorial optimization. We evaluate and compare our method with state-of-the-art methods in generic reconstruction, model-based reconstruction, geometry simplification, and primitive assembly. Experiments on both synthetic and real-world point clouds have demonstrated that, with our neural-guided strategy, high-quality building models can be obtained with significant advantages in fidelity, compactness, and computational efficiency. Our method also shows robustness to noise and insufficient measurements, and it can directly generalize from synthetic scans to real-world measurements. The source code of this work is freely available at https://github.com/chenzhaiyu/points2poly.

Robust real-world point cloud registration by inlier detection

Real-world point cloud registration is challenging because of large outliers in correspondence search. The mixture variations, such as partial overlap, noise and cross sources, are the root cause of these large outliers. Existing methods face challenges in effectively removing the large outliers. We propose a novel coarse-to-fine framework to remove the outliers by detecting the accurate inlier correspondences. Specifically, our coarse module predicts the top-K accurate correspondences. The coarse module is trained by jointly leveraging global and local structured information. Then, our refinement module checks the correspondences further using our proposed novel higher-order filter, which enables the structure conformity of correspondences to improve the quality of inlier correspondences. The final transformation matrix is calculated by using the refined inlier correspondences. Furthermore, a new cross-source point cloud dataset is proposed to further demonstrate the robustness in real-world point clouds. Experimental results demonstrate that our algorithm achieves the state-of-the-art accuracy on both indoor and outdoor, same-source and newly proposed cross-source real-world point clouds. • Works on both same-source and cross-source point cloud registration. • A coarse-to-fine framework combines both merits of deep learning and optimization. • Our algorithm can explicitly remove the outliers. • A real-world cross-source point cloud registration dataset. • Comprehensive experiments on both same-source and cross-source datasets.

Geometric models from laser scanning data for superstructure components of steel girder bridges

To advance the scan-to-model process for steel girder bridges, this paper presents an automated approach for creating complete geometric models for the steel superstructure elements based on the segmentation results from Yan and Hajjar (2021). The key innovation is two-fold: 1) cross-frames are automatically partitioned into cross-frame members that need to be modeled separately; 2) a 3D occlusion labeling algorithm is developed to identify occluded spaces, which are subsequently used in the modeling process to mitigate the effects of occlusions. The proposed approach is validated using real-world point clouds collected from a highway bridge, and the validation indicates that the errors in computing element widths and thicknesses are less than ±5% and ± 8% respectively, provided a reasonable data completeness. As the level of data completeness decreases, the errors for element widths can still be kept less than ±5%, but the element thicknesses can be overestimated by up to 52%.

Unsupervised Point Cloud Registration by Learning Unified Gaussian Mixture Models

Sampling noise and density variation widely exist in the point cloud acquisition process, leading to few accurate point-to-point correspondences. Since they rely on point-to-point correspondence search, existing state-of-the-art point cloud registration methods face difficulty in overcoming the sampling noise and density variation accurately or efficiently. Moreover, the recent state-of-the-art learning-based methods requires ground-truth transformation as supervised information which lead to large labor costs in real scenes. In this paper, our motivation is that two point-clouds are considered two samples from a unified Gaussian Mixture Model (UGMM). Then, we leverage the advantage of the statistic model to overcome the noise and density variants, and uses the alignment score in the UGMM to supervise the network training. To achieve this motivation, we propose a new unsupervised learning-based probabilistic registration algorithm to reconstruct the unified GMM and solve the registration problem simultaneously. The proposed method formulates the registration problem into a clustering problem, which estimates the posterior probability that classifies the points of two input point clouds to components of the unified GMM. A new feature interaction module is designed to learn the posterior probability using both the self and cross point cloud information. Then, two differential modules are proposed to calculate the GMM parameters and transformation matrices. Experimental results on synthetic and real-world point cloud datasets demonstrate that our unsupervised method achieves better registration accuracy and efficiency than the state-of-the-art supervised and semi-supervised methods in handling noisy and density variant point clouds.

Decouple the object: Component-level semantic recognizer for point clouds classification

3D point clouds classification is fundamental but always challenging in point clouds analysis, of which the key is efficiently extracting their distinguishing features. At present, several studies based on deep learning perform well in 3D point clouds classification task but are still weak in the capability of resemblance categories recognition. This paper presents a component semantic recognizer to decrease misclassification between the resemblance categories, which can better identify the different structural blocks of point clouds. Specifically, a point energy measurement unit is first built to pre-divide point clouds into non-overlapping blocks. Then, considering the fact that the closer the distance between 3D points, the stronger the feature interaction should be, an affinity bias is proposed for preserving the spatial properties when points features interact within the structural block. Besides validating the method on both synthetic and real-world point clouds benchmarks, the performance of the components is also tested and visualized. Comparing with other state-of-the-art methods, the proposed approach shows superiority.

RIConv++: Effective Rotation Invariant Convolutions for 3D Point Clouds Deep Learning

3D point clouds deep learning is a promising field of research that allows a neural network to learn features of point clouds directly, making it a robust tool for solving 3D scene understanding tasks. While recent works show that point cloud convolutions can be invariant to translation and point permutation, investigations of the rotation invariance property for point cloud convolution has been so far scarce. Some existing methods perform point cloud convolutions with rotation-invariant features, existing methods generally do not perform as well as translation-invariant only counterpart. In this work, we argue that a key reason is that compared to point coordinates, rotation-invariant features consumed by point cloud convolution are not as distinctive. To address this problem, we propose a simple yet effective convolution operator that enhances feature distinction by designing powerful rotation invariant features from the local regions. We consider the relationship between the point of interest and its neighbors as well as the internal relationship of the neighbors to largely improve the feature descriptiveness. Our network architecture can capture both local and global context by simply tuning the neighborhood size in each convolution layer. We conduct several experiments on synthetic and real-world point cloud classifications, part segmentation, and shape retrieval to evaluate our method, which achieves the state-of-the-art accuracy under challenging rotations.

Rotation invariant point cloud analysis: Where local geometry meets global topology

Point cloud analysis is a fundamental task in 3D computer vision. Most previous works have conducted experiments on synthetic datasets with well-aligned data; while real-world point clouds are often not pre-aligned. How to achieve rotation invariance remains an open problem in point cloud analysis. To meet this challenge, we propose an approach toward achieving rotation-invariant (RI) representations by combining local geometry with global topology. In our local-global-representation (LGR)-Net, we have designed a two-branch network where one stream encodes local geometric RI features and the other encodes global topology-preserving RI features. Motivated by the observation that local geometry and global topology have different yet complementary RI responses in varying regions, two-branch RI features are fused by an innovative multi-layer perceptron (MLP) based attention module. To the best of our knowledge, this work is the first principled approach toward adaptively combining global and local information under the context of RI point cloud analysis. Extensive experiments have demonstrated that our LGR-Net achieves the state-of-the-art performance on various rotation-augmented versions of ModelNet40, ShapeNet, ScanObjectNN, and S3DIS.