Synthetic Point Clouds Research Articles

Optimal Point-to-Point geodesic path generation on point clouds

Computing the shortest path between two points or a set of points on a point cloud representing a manifold is a crucial problem in Computational Science, particularly in Computer Graphics and CAD/CAM. Most methods calculate geodesic paths on a polygonal model of the manifold, while only a few compute them directly on the point cloud. When the point cloud has noise, constructing a triangular mesh is challenging and results in inaccurate geodesic path calculations. This work calculates the geodesic path by minimizing the discrete geodesic curvature of the connecting curve, using an iterative Newton minimization and a projection procedure based on directed point-projection and Gabriel neighborhoods. The proposed algorithm has quadratic convergence and produces accurate results in synthetic and free-form point clouds, outperforming triangle mesh-based methods in noisy point clouds. The paper concludes with a demonstration of geodesic path computation in diverse free-form point-clouds and a case study in template-based digital surface reconstruction. The source code of the proposed method can be found in https://github.com/agalex1974/LibGeodesicOPPGC.

Using Synthetic Tree Data in Deep Learning-Based Tree Segmentation Using LiDAR Point Clouds

Deep learning, neural networks and other data-driven processing techniques are increasingly used in the analysis of LiDAR point cloud data in forest environments due to the benefits offered in accuracy and adaptability to new environments. One of the downsides of these techniques in practical applications is the requirement for manually annotated data necessary for training neural networks, which can be time consuming and costly to attain. We develop an approach to training neural networks for forest tree stem segmentation from point clouds that uses synthetic data from a custom tree simulator, which can generate large quantities of training examples without manual human effort. Our tree simulator captures the geometric characteristics of tree stems and foliage, from which automatically-labelled synthetic point clouds can be generated for training a semantic segmentation algorithm based on the PointNet++ architecture. Using evaluations on real aerial and terrestrial LiDAR point clouds from a range of different forest sites, we demonstrate our synthetic data-trained models can out-perform, or provide comparable performance with models trained on real data from other sites or when available real training data is limited (increases in IoU from 1–7%). Our simulation code is open-source and made available to the research community.

MVPCC-Net: Multi-View Based Point Cloud Completion Network for MLS data

In this paper, we introduce a novel multi view-based method for completing high-resolution 3D point clouds of partial object shapes obtained by mobile laser scanning (MLS) platforms. Our approach estimates both the geometry and color cues of the missing or incomplete object segments, by projecting the 3D input point cloud by multiple virtual cameras, and performing 2D inpainting in the image domains of the different views. In contrast to existing state-of-the-art methods, our method can generate point clouds consisting of a variable number of points, depending on the detailedness of the input measurement, which property highly facilitates the efficient processing of MLS data with inhomogeneous point density. For training and quantitative evaluation of the proposed method, we provide a new point cloud dataset that consists of both synthetic point clouds of four different street objects with accurate ground truth, and real MLS measurements of partially or fully scanned vehicles. The quantitative and qualitative experiments on the provided dataset demonstrate that our method surpasses state-of-the-art approaches in reconstructing the local fine geometric structures as well as in estimating the overall shape and color pattern of the objects.

Point cloud-based dimensional quality assessment of precast concrete components using deep learning

The dimensional quality of precast concrete (PC) subcomponents (concrete and rebars) should be inspected in advance to ensure assembly quality. Currently, PC components are mainly inspected in a manual manner using tools such as tape measure, which is error-prone and inefficient. This study developed an innovative approach for automatic dimensional quality assessment of PC components using point cloud-based deep learning techniques. The approach consists of 1) a dataset-generating method to automatically create the synthetic dataset of PC components’ point clouds, 2) an enhanced focal loss-based precast concrete component recognition net (PCCR-Net) employing hierarchical feature learning to segment the synthetic point clouds dataset into rebars and concrete (i.e. the synthetic dataset generated is used to train the PCCR-Net), and 3) a quantitative measurement protocol that can estimate the dimensional quality of the segmented concrete and rebars. Experiments were conducted to test the capability of the approach, and the results show that the proposed approach was able to yield satisfactory performance. First, the dataset-generating method can solve the shortage of point cloud datasets in engineering practice. Second, the PCCR-Net segmentation network can simultaneously realize the high-precision identification of various typical PC components, including PC columns, beams, slabs, and walls. Third, the dimension average deviations between experimental results and manual measurements demonstrate that the assessment approach can accurately estimate the dimensions of PC components.

Semantic segmentation of bridge point clouds with a synthetic data augmentation strategy and graph-structured deep metric learning

Deep learning techniques are capable of providing versatile solutions to automate classification of bridge point clouds into corresponding constituent components, but training sample scarcity and erroneous boundary segmentation have hindered the extent of their application. To respond to these challenges, this study presents two synthetic data augmentation strategies for alleviating the data scarcity problem and improves the weighted superpoint graph (WSPG) model by using a graph-structured deep metric learning method for generating high quality superpoints to address the erroneous boundary segmentation problem. Evaluation experiments were conducted to validate their effectiveness. Experimental results showed that the synthetic data augmentation strategies can significantly alleviate the training sample scarcity problem. The synthetic superpoint augmentation strategy outperformed the synthetic point cloud augmentation strategy with an increase of around 6% for overall accuracy (OA), 3% for mean class accuracy (mAcc) and 18% for mean Intersection over Union (mIoU). Compared with the original WSPG model, the improved WSPG model with graph-structured deep metric learning method increased the mIoU of the overall dataset and the pier caps by 4% and 21% respectively, while also reducing the boundary segmentation errors.

Automatic Generation of Synthetic Heritage Point Clouds: Analysis and Segmentation Based on Shape Grammar for Historical Vaults

Automatic Generation of Synthetic Heritage Point Clouds: Analysis and Segmentation Based on Shape Grammar for Historical Vaults

EB-LG Module for 3D Point Cloud Classification and Segmentation

Thanks to the development of deep learning technology and computer science, 3D point cloud analysis is becoming a research hotspot. Based on convolution operator, local feature learning is fundamental but far from perfect regarding point cloud analysis task. Actually, when people see a wing-like part (local features), our brain immediately associates it with a plane model (global features), and then we further trust that it's a wing based on the feedback from our brain. Such hidden features between local features and global features are commonly-existed, but ignored by existing methods. To this end, we propose Error feature Back-projection based Local-Global (EB-LG) feature learning module for better representations of point clouds. Specifically, EB-LG module adequately captures the hidden features firstly; and then borrowing from the successful idea of error-feedback mechanism, the learned hidden features will be back-projected to original local features, so that the enhanced local features are obtained. Serving as a plug-and-play, EB-LG module is lightweight and can be easily integrated into existing state-of-the-art networks to boost their performance. Extensive evaluations on both synthetic and real-world 3D point cloud benchmarks demonstrate the effectiveness and the generalization ability of our method.

PU-MFA: Point Cloud Up-Sampling via Multi-Scale Features Attention

Recently, research using point clouds has been increasing with the development of 3D scanner technology. According to this trend, the demand for high-quality point clouds is increasing, but there is still a problem with the high cost of obtaining high-quality point clouds. Therefore, with the recent remarkable development of deep learning, point cloud up-sampling research, which uses deep learning to generate high-quality point clouds from low-quality point clouds, is one of the fields attracting considerable attention. This paper proposes a new point cloud up-sampling method called Point cloud Up-sampling via Multi-scale Features Attention (PU-MFA). Inspired by prior studies that reported good performance at generating high-quality dense point set using the multi-scale features or attention mechanisms, PU-MFA merges the two through a U-Net structure. In addition, PU-MFA adaptively uses multi-scale features to refine the global features effectively. The PU-MFA was compared with other state-of-the-art methods in various evaluation metrics through various experiments using the PU-GAN dataset, which is a synthetic point cloud dataset, and the KITTI dataset, which is the real-scanned point cloud dataset. In various experimental results, PU-MFA showed superior performance of generating high-quality dense point set in quantitative and qualitative evaluation compared to other state-of-the-art methods, proving the effectiveness of the proposed method. The attention map of PU-MFA was also visualized to show the effect of multi-scale features.

Reconstructing compact building models from point clouds using deep implicit fields

While three-dimensional (3D) building models play an increasingly pivotal role in many real-world applications, obtaining a compact representation of buildings remains an open problem. In this paper, we present a novel framework for reconstructing compact, watertight, polygonal building models from point clouds. Our framework comprises three components: (a) a cell complex is generated via adaptive space partitioning that provides a polyhedral embedding as the candidate set; (b) an implicit field is learned by a deep neural network that facilitates building occupancy estimation; (c) a Markov random field is formulated to extract the outer surface of a building via combinatorial optimization. We evaluate and compare our method with state-of-the-art methods in generic reconstruction, model-based reconstruction, geometry simplification, and primitive assembly. Experiments on both synthetic and real-world point clouds have demonstrated that, with our neural-guided strategy, high-quality building models can be obtained with significant advantages in fidelity, compactness, and computational efficiency. Our method also shows robustness to noise and insufficient measurements, and it can directly generalize from synthetic scans to real-world measurements. The source code of this work is freely available at https://github.com/chenzhaiyu/points2poly.

Enhanced GPIS Learning Based on Local and Global Focus Areas

Implicit surface learning is one of the most widely used methods for 3D surface reconstruction from raw point cloud data. Current approaches employ deep neural networks or Gaussian process models with the trade-offs across computational performance, object fidelity, and generalization capabilities. We propose a novel method based on Gaussian process regression to build implicit surfaces for 3D surface reconstruction (GPIS), which leads to better accuracy in comparison to the standard GPIS formulation. Our approach encodes local and global shape information from the data to maintain the correct topology of the underlying shape. The proposed pipeline works on dense, sparse, and noisy raw point clouds and can be parallelized to improve computational efficiency. We evaluate our approach on synthetic and real point cloud datasets including data from robot visual and tactile sensors. Results show that our approach leads to high accuracy compared to baselines.

A Realism Metric for Generated LiDAR Point Clouds

A considerable amount of research is concerned with the generation of realistic sensor data. LiDAR point clouds are generated by complex simulations or learned generative models. The generated data is usually exploited to enable or improve downstream perception algorithms. Two major questions arise from these procedures: First, how to evaluate the realism of the generated data? Second, does more realistic data also lead to better perception performance? This paper addresses both questions and presents a novel metric to quantify the realism of LiDAR point clouds. Relevant features are learned from real-world and synthetic point clouds by training on a proxy classification task. In a series of experiments, we demonstrate the application of our metric to determine the realism of generated LiDAR data and compare the realism estimation of our metric to the performance of a segmentation model. We confirm that our metric provides an indication for the downstream segmentation performance.

SEMI-AUTOMATED TREE SPECIES CLASSIFICATION BASED ON ROUGHNESS PARAMETERS USING AIRBORNE LIDAR DATA

Automated tree species classification using high density airborne LiDAR data supports precise forest inventory. This work shows a method based on evaluating roughness descriptors from aerial LiDAR data to automatically classify tree species. The proposed method includes treetops detection, neighbouring distance analysis for selecting the interest points, 3D fit surface creation, evaluation of roughness parameters, and K-means clustering. Among the evaluated roughness parameters, Skewness (Rsk) and Kurtosis (Rku) show robust classification. A synthetic point cloud was generated to test the methodology in a mixed forest formed by three tree species, Pinus sp., Quercus sp., and Eucalyptus sp. The Overall Accuracy (OA) of the classification method was 80 % for Quercus sp., 100 % for Pinus sp. and 80.6 % for Eucalyptus sp. In addition, the methodology was tested in three study areas and the results demonstrate that roughness parameters can be used to individual tree species classification in a mixed temperate forest with an OA of 82% in study area 1, 93 % in study area 2 and 92 % in study area 3. Keywords: aerial LiDAR, point cloud processing, tree species classification, spatial analysis, roughness parameters

Zero-Shot Learning on 3D Point Cloud Objects and Beyond

Zero-shot learning, the task of learning to recognize new classes not seen during training, has received considerable attention in the case of 2D image classification. However, despite the increasing ubiquity of 3D sensors, the corresponding 3D point cloud classification problem has not been meaningfully explored and introduces new challenges. In this paper, we identify some of the challenges and apply 2D Zero-Shot Learning (ZSL) methods in the 3D domain to analyze the performance of existing models. Then, we propose a novel approach to address the issues specific to 3D ZSL. We first present an inductive ZSL process and then extend it to the transductive ZSL and Generalized ZSL (GZSL) settings for 3D point cloud classification. To this end, a novel loss function is developed that simultaneously aligns seen semantics with point cloud features and takes advantage of unlabeled test data to address some known issues (e.g., the problems of domain adaptation, hubness, and data bias). While designed for the particularities of 3D point cloud classification, the method is shown to also be applicable to the more common use-case of 2D image classification. An extensive set of experiments is carried out, establishing state-of-the-art for ZSL and GZSL on synthetic (ModelNet40, ModelNet10, McGill) and real (ScanObjectNN) 3D point cloud datasets.

Unsupervised Point Cloud Registration by Learning Unified Gaussian Mixture Models

Sampling noise and density variation widely exist in the point cloud acquisition process, leading to few accurate point-to-point correspondences. Since they rely on point-to-point correspondence search, existing state-of-the-art point cloud registration methods face difficulty in overcoming the sampling noise and density variation accurately or efficiently. Moreover, the recent state-of-the-art learning-based methods requires ground-truth transformation as supervised information which lead to large labor costs in real scenes. In this paper, our motivation is that two point-clouds are considered two samples from a unified Gaussian Mixture Model (UGMM). Then, we leverage the advantage of the statistic model to overcome the noise and density variants, and uses the alignment score in the UGMM to supervise the network training. To achieve this motivation, we propose a new unsupervised learning-based probabilistic registration algorithm to reconstruct the unified GMM and solve the registration problem simultaneously. The proposed method formulates the registration problem into a clustering problem, which estimates the posterior probability that classifies the points of two input point clouds to components of the unified GMM. A new feature interaction module is designed to learn the posterior probability using both the self and cross point cloud information. Then, two differential modules are proposed to calculate the GMM parameters and transformation matrices. Experimental results on synthetic and real-world point cloud datasets demonstrate that our unsupervised method achieves better registration accuracy and efficiency than the state-of-the-art supervised and semi-supervised methods in handling noisy and density variant point clouds.

Transfer Learning from Synthetic to Real LiDAR Point Cloud for Semantic Segmentation

Knowledge transfer from synthetic to real data has been widely studied to mitigate data annotation constraints in various computer vision tasks such as semantic segmentation. However, the study focused on 2D images and its counterpart in 3D point clouds segmentation lags far behind due to the lack of large-scale synthetic datasets and effective transfer methods. We address this issue by collecting SynLiDAR, a large-scale synthetic LiDAR dataset that contains point-wise annotated point clouds with accurate geometric shapes and comprehensive semantic classes. SynLiDAR was collected from multiple virtual environments with rich scenes and layouts which consists of over 19 billion points of 32 semantic classes. In addition, we design PCT, a novel point cloud translator that effectively mitigates the gap between synthetic and real point clouds. Specifically, we decompose the synthetic-to-real gap into an appearance component and a sparsity component and handle them separately which improves the point cloud translation greatly. We conducted extensive experiments over three transfer learning setups including data augmentation, semi-supervised domain adaptation and unsupervised domain adaptation. Extensive experiments show that SynLiDAR provides a high-quality data source for studying 3D transfer and the proposed PCT achieves superior point cloud translation consistently across the three setups. The dataset is available at https://github.com/xiaoaoran/SynLiDAR.

FROM BIM TO POINTCLOUD: AUTOMATIC GENERATION OF LABELED INDOOR POINTCLOUD

Abstract. With the development of deep learning technology, a large number of indoor spatial applications, such as robotics and indoor navigation, have raised higher data requirements for indoor semantic model. However, creating deep learning classifiers requires a large number of labeled datasets, and the collection of such datasets requires a lot of manually labeling proces, which is labor-intensive and time-consuming. In this paper, we propose a method to automatically create 3D point clouds datasets with indoor semantic labels based on parametric BIM model. First, a automatic BIM generation method is proposed through simulating the structure of interior space Secondly, we use a viewpoint-guided labeled point cloud generation method to generate synthetic 3D point clouds with different labels, color information. Especially, noise are also simulated with a gaussian model. As shown in the experiments, the point cloud data with labels can be quickly obtained from existing BIM models, which will largely reduce the complexity of data labeling and improve efficiency. These simulated data can be used in the deep learning training process and improve the semantic segmentation accuracy.

RESAMPLING METHODS FOR A RELIABLE VALIDATION SET IN DEEP LEARNING BASED POINT CLOUD CLASSIFICATION

Abstract. A validation data set plays a pivotal role in tweaking a machine learning model trained in a supervised manner. Many existing algorithms select a part of available data by using random sampling to produce a validation set. However, this approach can be prone to overfitting. One should follow careful data splitting to have reliable training and validation sets that can produce a generalized model with a good performance for the unseen (test) data. Data splitting based on resampling techniques involves repeatedly drawing samples from the available data. Hence, resampling methods can give better generalization power to a model, because they can produce and use many training and/or validation sets. These techniques are computationally expensive, but with increasingly available high-performance computing facilities, one can exploit them. Though a multitude of resampling methods exist, investigation of their influence on the generality of deep learning (DL) algorithms is limited due to its non-linear black-box nature. This paper contributes by: (1) investigating the generalization capability of the four most popular resampling methods: k-fold cross-validation (k-CV), repeated k-CV (Rk-CV), Monte Carlo CV (MC-CV) and bootstrap for creating training and validation data sets used for developing, training and validating DL based point cloud classifiers (e.g., PointNet; Qi et al., 2017a), (2) justifying Mean Square Error (MSE) as a statistically consistent estimator, and (3) exploring the use of MSE as a reliable performance metric for supervised DL. Experiments in this paper are performed on both synthetic and real-world aerial laser scanning (ALS) point clouds.

BIM-driven data augmentation method for semantic segmentation in superpoint-based deep learning network

This paper describes a universal workflow to synthesize point clouds containing both geometry and color information by utilizing the IFC model or its 3D geometry model to automatically generate annotated point clouds for semantic segmentation in deep learning. In our experiments, we selected 44 scenes from the S3DIS dataset to rebuild BIM models and synthesize point clouds, replaced training datasets in different proportions with synthetic point clouds, and fed the mixed datasets to a superpoint-based network. The training results show that increasing the appropriate proportion of synthetic point clouds in the training dataset can significantly improve the semantic segmentation performance. Furthermore, we extend our experiments to two other state-of-the-art networks and another similar dataset, ScanNet, and the experimental results illustrate that the synthetic point clouds can be applied to augmenting the limited and low-quality annotated point clouds for deep learning and boost the automatic remodeling work in the architectural field.

Decouple the object: Component-level semantic recognizer for point clouds classification

3D point clouds classification is fundamental but always challenging in point clouds analysis, of which the key is efficiently extracting their distinguishing features. At present, several studies based on deep learning perform well in 3D point clouds classification task but are still weak in the capability of resemblance categories recognition. This paper presents a component semantic recognizer to decrease misclassification between the resemblance categories, which can better identify the different structural blocks of point clouds. Specifically, a point energy measurement unit is first built to pre-divide point clouds into non-overlapping blocks. Then, considering the fact that the closer the distance between 3D points, the stronger the feature interaction should be, an affinity bias is proposed for preserving the spatial properties when points features interact within the structural block. Besides validating the method on both synthetic and real-world point clouds benchmarks, the performance of the components is also tested and visualized. Comparing with other state-of-the-art methods, the proposed approach shows superiority.

RIConv++: Effective Rotation Invariant Convolutions for 3D Point Clouds Deep Learning

3D point clouds deep learning is a promising field of research that allows a neural network to learn features of point clouds directly, making it a robust tool for solving 3D scene understanding tasks. While recent works show that point cloud convolutions can be invariant to translation and point permutation, investigations of the rotation invariance property for point cloud convolution has been so far scarce. Some existing methods perform point cloud convolutions with rotation-invariant features, existing methods generally do not perform as well as translation-invariant only counterpart. In this work, we argue that a key reason is that compared to point coordinates, rotation-invariant features consumed by point cloud convolution are not as distinctive. To address this problem, we propose a simple yet effective convolution operator that enhances feature distinction by designing powerful rotation invariant features from the local regions. We consider the relationship between the point of interest and its neighbors as well as the internal relationship of the neighbors to largely improve the feature descriptiveness. Our network architecture can capture both local and global context by simply tuning the neighborhood size in each convolution layer. We conduct several experiments on synthetic and real-world point cloud classifications, part segmentation, and shape retrieval to evaluate our method, which achieves the state-of-the-art accuracy under challenging rotations.