Unstructured Point Clouds Research Articles

A framework for automated supraglacial lake detection and depth retrieval in ICESat-2 photon data across the Greenland and Antarctic ice sheets

Abstract. Water depths of supraglacial lakes on the ice sheets are difficult to monitor continuously due the lakes' ephemeral nature and inaccessible locations. Supraglacial lakes have been linked to ice shelf collapse in Antarctica and accelerated flow of grounded ice in Greenland. However, the impact of supraglacial lakes on ice dynamics has not been quantified accurately enough to predict their contribution to future mass loss and sea level rise. This is largely because ice-sheet-wide assessments of meltwater volumes rely on models that are poorly constrained due to a lack of accurate depth measurements. Various recent case studies have demonstrated that accurate supraglacial lake depths can be obtained from NASA's Ice, Cloud and land Elevation Satellite (ICESat-2) ATL03 photon-level data product. ATL03 comprises hundreds of terabytes of unstructured point cloud data, which has made it challenging to use this bathymetric capability at scale. Here, we present two new algorithms – Flat Lake and Underlying Ice Detection (FLUID) and Surface Removal and Robust Fit (SuRRF) – which together provide a fully automated and scalable method for lake detection and along-track depth determination from ATL03 data and establish a framework for its large-scale implementation using distributed high-throughput computing. We report FLUID–SuRRF algorithm performance over two regions known to have significant surface melt – central West Greenland and the Amery Ice Shelf catchment in East Antarctica – during two melt seasons. FLUID–SuRRF reveals a total of 1249 ICESat-2 lake segments up to 25 m deep, with more water during higher-melt years. In the absence of ground-truth data, manual annotation of test data suggests that our method reliably detects melt lakes along ICESat-2's ground tracks whenever the lake bed is visible or partially visible and estimates water depths with a mean absolute error <0.27 m. These results imply that our proposed framework has the potential to generate a comprehensive data product of accurate meltwater depths across both ice sheets.

Automated anthropometric measurements from 3D point clouds of scanned bodies

Anthropometry plays a critical role across numerous sectors, particularly within healthcare and fashion, by facilitating the analysis of the human body structure. The significance of anthropometric data cannot be overstated; it is crucial for assessing nutritional status among children and adults alike, enabling early detection of conditions such as malnutrition, obesity, and being overweight. Furthermore, it is instrumental in creating tailored dietary interventions. This study introduces a novel automated technique for extracting anthropometric measurements from any body part. The proposed method leverages a parametric model to accurately determine the measurement parameters from either an unstructured point cloud or a mesh. We conducted a comprehensive evaluation of our approach by comparing perimetral measurements from over 400 body scans with expert assessments and existing state-of-the-art methods. The results demonstrate that our approach significantly surpasses the current methods for measuring the waist, hip, thigh, chest, and wrist perimeters with exceptional accuracy. These findings indicate the potential of our method to automate anthropometric analysis and offer efficient and accurate measurements for various applications in healthcare and fashion industries.

An Efficient and Versatile Variational Method for High-Dimensional Data Classification

High-dimensional data classification is a fundamental task in machine learning and imaging science. In this paper, we propose an efficient and versatile multi-class semi-supervised classification method for classifying high-dimensional data and unstructured point clouds. To begin with, a warm initialization is generated by using a fuzzy classification method such as the standard support vector machine or random labeling. Then an unconstraint convex variational model is proposed to purify and smooth the initialization, followed by a step which is to project the smoothed partition obtained previously to a binary partition. These steps can be repeated, with the latest result as a new initialization, to keep improving the classification quality. We show that the convex model of the smoothing step has a unique solution and can be solved by a specifically designed primal–dual algorithm whose convergence is guaranteed. We test our method and compare it with the state-of-the-art methods on several benchmark data sets. Thorough experimental results demonstrate that our method is superior in both the classification accuracy and computation speed for high-dimensional data and point clouds.

Quadratic surface preserving parameterization of unorganized point data

Finding parameterizations of spatial point data is a fundamental step for surface reconstruction in Computer Aided Geometric Design. Especially the case of unstructured point clouds is challenging and not widely studied. In this work, we show how to parameterize a point cloud by using barycentric coordinates in the parameter domain, with the aim of reproducing the parameterizations provided by quadratic triangular Bézier surfaces. To this end, we train an artificial neural network that predicts suitable barycentric parameters for a fixed number of data points. In a subsequent step we improve the parameterization using non-linear optimization methods. We then use a number of local parameterizations to obtain a global parameterization using a new overdetermined barycentric parameterization approach. We study the behavior of our method numerically in the zero-residual case (i.e., data sampled from quadratic polynomial surfaces) and in the non-zero residual case and observe an improvement of the accuracy in comparison to standard methods. We also compare different approaches for non-linear surface fitting such as tangent distance minimization, squared distance minimization and the Levenberg Marquardt algorithm.

Leveraging local neighborhood features in 3D unstructured point cloud data for geometry-based automated damage delineation

In a post-earthquake environment, manual inspection is resource-intensive and subjective. This article presents a real-world damage assessment study of the Improved Iterative Bi-Level Thresholding (IIBLT) algorithm. Damaged regions are delineated using IIBLT based on the local point curvature. The methodology is tested using a unique dataset: Mono Temporal Disaster City 3D Dataset (C3DO). C3DO is a point cloud data generated by a 3D reconstruction of a controlled post-earthquake environment, which presents challenges for concrete façade damage assessment algorithms. Results from IIBLT demonstrated a 16% increase in Matthews Correlation Coefficient (MCC) and Intersection over Union (mGIoU), compared to the current state-of-the-art methods. This corresponds to improving accuracy and completeness in defect region identification without the dependence on knowledge of damaged regions. Results from this study highlight the novel contributions of the IIBLT algorithm including the elimination of parameter tuning, and demonstrate the applicability of the novel iterative framework for concrete surface defect delineation at large.

ANTLER: Bayesian Nonlinear Tensor Learning and Modeler for Unstructured, Varying-Size Point Cloud Data

Unstructured point clouds of varying sizes are increasingly acquired in a variety of environments through laser triangulation or Light Detection and Ranging (LiDAR). Predicting a vector response based on unstructured point clouds is a common problem that arises in a wide variety of applications. The current literature relies on several pre-processing steps such as structured subsampling and feature extraction to analyze the point cloud data. Those techniques lead to quantization artifacts and do not consider the relationship between the regression response and the point cloud during pre-processing. Therefore, we propose a general and holistic “Bayesian Nonlinear Tensor Learning and Modeler” (ANTLER) to model the relationship of unstructured, varying-size point cloud data with a vector response. The proposed ANTLER simultaneously optimizes a nonlinear tensor dimensionality reduction and a nonlinear regression model with a 3D point cloud input and a regression response. ANTLER can consider the complex data representation, high-dimensionality, and inconsistent size of the 3D point cloud data. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper is motivated by a real-world case study concerning the prediction of the transmission error and eccentricity based on unstructured point clouds of varying sizes in gear manufacturing. In the current state-of-the-art method, those characteristics can only be obtained via expensive and time-consuming Finite Element Analysis (FEA) or test benches. The proposed ANTLER framework can directly link the measurement point clouds with a vector response and serves as a guiding example for the immense potential of the ANTLER.

PECo: A Point-Edge Collaborative Framework for Global-Aware Urban Building Contouring From Unstructured Point Clouds

PECo: A Point-Edge Collaborative Framework for Global-Aware Urban Building Contouring From Unstructured Point Clouds

PointSGRADE: Sparse learning with graph representation for anomaly detection by using unstructured 3D point cloud data

Surface anomaly detection by using 3D point cloud data has recently received significant attention. To completely measure the common free-form surfaces without loss of details, advanced 3D scanning technologies, such as 3D laser scanners, can be applied and will produce an unstructured point cloud. However, this irregular data structure poses challenges to anomaly detection, in that the existing methods based on regular data, e.g., 2D image, cannot be directly applied. This article proposes a sparse learning framework with a graph representation of the unstructured point cloud for anomaly detection (PointSGRADE). Specifically, the free-form surface is assumed to be smooth. Then, the associated point cloud can be represented as a graph. Subsequently, considering the sparse anomalies, we propose a sparse learning framework and formulate the anomaly detection problem as a penalized optimization problem, which is further solved by a computationally efficient majorization-minimization framework. Case studies demonstrate the accuracy and robustness of the proposed method. This article proposes a novel methodology for sparse anomaly detection on smooth free-form surfaces represented by unstructured point cloud, which is critical for quality inspection in manufacturing and other application areas.

QUICK PROTOCOL FOR INTEGRATING THE ATTRIBUTE INFORMATION OF UNSTRUCTURED POINT CLOUD DATA INTO A SOLAR ENVELOPE SIMULATION

ABSTRACT This study proposes a novel method of solar geometry by considering the potential application of point cloud data combined with the simulation of solar radiation. With the support of geometric and radiometric information stored in the point cloud such as position information (XYZ) color information (RGB), and reflection intensity (I), architects may compensate for missing information on the existing context during the simulation, especially due to the limited capacity of current 3D modelling sites. However, the dataset often comes in the format of unstructured point cloud data retrieved from merged data scans and as a result, the radiometric information is difficult to occupy due to multiple reference points. Through a 3D subtractive procedure, this study not only examines volumetric samples of the three-dimensional matrix that fulfills the criteria of solar envelopes but also finds the optimal values of the merged data scan for input of solar radiation. In this regard, simulation of solar radiation contributes to identifying the most and the least exposed areas to the sun in existing contexts. This provides information related to visible sun hours that can be used to perform ray tracing analysis between the proposed 3D plot and surrounding contexts. Our proposed method ultimately helps architects not only generate solar geometry based on real contextual settings but also to understand comprehensively the microclimate conditions of the design context.

PointDMS: An Improved Deep Learning Neural Network via Multi-Feature Aggregation for Large-Scale Point Cloud Segmentation in Smart Applications of Urban Forestry Management

Background: The development of laser measurement techniques is of great significance in forestry monitoring and park management in smart cities. It provides many conveniences for improving landscape planning efficiency and strengthening digital construction. However, capturing 3D point clouds in large-scale landscape environments is a complex task that generates massive amounts of unstructured data with characteristics such as randomness, rotational invariance, sparsity, and serious barriers. Methods: To improve the processing efficiency of intelligent devices for massive point clouds, we propose a novel deep learning neural network based on a multi-feature aggregation strategy. This network is designed to divide 3D laser point clouds in complex large-scale scenarios. Firstly, we utilize multiple terrestrial laser sensors to collect a large amount of data in open scenes such as parks, streets, and forests in urban environments. These data are integrated into a practical database called DMSdataset, which contains different information variables, densities, and dimensions. Then, an automatic block integrated with a multi-feature extractor is constructed to pre-process the unstructured point cloud data and standardize the datasets. Finally, a novel semantic segmentation framework called PointDMS is designed using 3D convolutional deep networks. PointDMS achieves a better segmentation performance of point clouds with a lightweight parameter structure. Here, “D” stands for deep network, “M” stands for multi-feature, and “S” stands for segmentation. Results: Extensive experiments on self-built datasets show that the proposed PointDMS achieves similar or better performance in point cloud segmentation compared to other methods. The overall identification accuracy of the proposed model is up to 93.5%, which is a 14% increase. Particularly for living wood objects, the average identification accuracy is up to 88.7%, which is, at least, an 8.2% increase. These results effectively prove that PointDMS is beneficial for 3D point cloud processing, division, and mining applications in urban forest environments. It demonstrates good robustness and generalization.

NeAF: Learning Neural Angle Fields for Point Normal Estimation

Normal estimation for unstructured point clouds is an important task in 3D computer vision. Current methods achieve encouraging results by mapping local patches to normal vectors or learning local surface fitting using neural networks. However, these methods are not generalized well to unseen scenarios and are sensitive to parameter settings. To resolve these issues, we propose an implicit function to learn an angle field around the normal of each point in the spherical coordinate system, which is dubbed as Neural Angle Fields (NeAF). Instead of directly predicting the normal of an input point, we predict the angle offset between the ground truth normal and a randomly sampled query normal. This strategy pushes the network to observe more diverse samples, which leads to higher prediction accuracy in a more robust manner. To predict normals from the learned angle fields at inference time, we randomly sample query vectors in a unit spherical space and take the vectors with minimal angle values as the predicted normals. To further leverage the prior learned by NeAF, we propose to refine the predicted normal vectors by minimizing the angle offsets. The experimental results with synthetic data and real scans show significant improvements over the state-of-the-art under widely used benchmarks. Project page: https://lisj575.github.io/NeAF/.

SURFACE OR SKELETON? AUTOMATIC HIERARCHICAL CLUSTERING OF 3D POINT CLOUDS OF BRONZE FROG DRUMS FOR HERITAGE DIGITAL TWINS

Abstract. In the era of digital twins, high-definition 3D point clouds of cultural relics, such as the bronze drums of ancient Southeast Asia and China, are increasingly available as digital heritage. This study applies an automatic hierarchical clustering method to compare and cluster 14 unstructured 3D models of frogs on drums based on the dissimilarity metric of the minimum error from 2,000 iterations of global registration. Furthermore, this study compares two forms of 3D presentation: surface points and 3D shape skeletons. The experimental results on 14 high-definition frogs showed that four groups – three-legged with baby, four-legged with baby, three-legged without baby, and four-legged without baby – were consistently (TPR = 0.857) detected, regardless of the 3D presentation using point clouds or shape skeletons. Both basic surface points and advanced shape skeleton effectively clustered 3D heritage details for heritage digital twins and advanced heritage documentation. The findings also imply that geospatial analytics using either surface 3D point clouds or skeleton can shed light on unsupervised learning and quantitative understanding of unstructured point clouds of numerous cultural heritages.

Residual learning with annularly convolutional neural networks for classification and segmentation of 3D point clouds

Analysis of point clouds through deep convolutional neural networks is an active area of research due to their massive real-world applications including autonomous driving, indoor navigation, robotics, virtual/augmented reality, unmanned aerial vehicles, and drone technology. However, capturing the fine-grained geometric and semantic properties for the underlying recognition task with raw unstructured point cloud is highly challenging due to the lack of explicit neighborhood relationship and sparsity among the points. In this paper, we have introduced a deep, hierarchical 3D point based architecture for object classification and part segmentation that is able to learn robust geometric features which remain invariant to both the geometry and orientation of the local patches. The proposed architecture consists of multiple layers of sampling, concentric annular convolution, pooling, and residual feature propagation blocks. In the skip connections of our deep residual design, we propose to use a combination of linear projection shortcut and nonlinear ReLU group normalization shortcut with batch normalization, to improve both the optimization landscape and the representational power. Our network achieves on par or even better than state-of-the-art results on synthetic and real-world object classification (i.e., ModelNet40 and ScanObjectNN) and part segmentation (i.e., ShapeNet-part) benchmark datasets. The implementation and results have been made available at …https://github.com/Rabbia-Hassan/Deep_Annular_Residual_Feature_Learning_for_3dPointCloudsGitHub-link

As-built BIM reconstruction of piping systems using PipeNet

Building Information Modeling (BIM) is gradually recognized and promoted as the new standard practice in the construction industry as well as the built environment. Mechanical, electrical, and plumbing system, as a system requiring regular maintenance, takes an important place in the building operation and maintenance; its failure can cause significant impacts operationally, economically, and even environmentally. As-built BIM is needed for efficient BIM-enabled facility management of buildings that were built before the existence of BIM or with outdated BIM. Modeling of the as-built BIM is currently practiced in a very manual and tedious way, requiring a considerable amount of time and effort even for a skilled modeler. This paper proposes a solution that reconstructs the as-built BIM of piping systems in buildings from LiDAR scanned point cloud data automatically. Compared to the existing works, it requires no additional data other than the unstructured point cloud with XYZ fields, no data preprocessing, and no prior knowledge of the pipe directions or dimensions. The solution comprises two stages. The first is a novel deep learning network, PipeNet, that is able to detect pipes regardless of the size of the input data and the scale of the target scene, and predict the pipe centerline points together with other pipe parameters. In the second stage, the pipe model is reconstructed through line fitting, refinement, and graph-based connectivity analysis constrained by domain knowledge that maximizes the coherence of the piping system model. The final output is converted to the Industry Foundation Classes format which is neutrally acceptable in the BIM industry. The solution is validated on both synthetic and actual scan data, and the results demonstrate its robustness, fast speed, and high recognition rate and precision. The results are discussed in detail and further improvements such as improving the precision of recognition are suggested for future works.

Surface Reconstruction from Unstructured Point Cloud Data for Building Digital Twin

Surface Reconstruction from Unstructured Point Cloud Data for Building Digital Twin

Real-Time 3D Object Detection and Classification in Autonomous Driving Environment Using 3D LiDAR and Camera Sensors

The rapid development of Autonomous Vehicles (AVs) increases the requirement for the accurate prediction of objects in the vicinity to guarantee safer journeys. For effectively predicting objects, sensors such as Three-Dimensional Light Detection and Ranging (3D LiDAR) and cameras can be used. The 3D LiDAR sensor captures the 3D shape of the object and produces point cloud data that describes the geometrical structure of the object. The LiDAR-only detectors may be subject to false detection or even non-detection over objects located at high distances. The camera sensor captures RGB images with sufficient attributes that describe the distinct identification of the object. The high-resolution images produced by the camera sensor benefit the precise classification of the objects. However, hindrances such as the absence of depth information from the images, unstructured point clouds, and cross modalities affect assertion and boil down the environmental perception. To this end, this paper proposes an object detection mechanism that fuses the data received from the camera sensor and the 3D LiDAR sensor (OD-C3DL). The 3D LiDAR sensor obtains point clouds of the object such as distance, position, and geometric shape. The OD-C3DL employs Convolutional Neural Networks (CNN) for further processing point clouds obtained from the 3D LiDAR sensor and the camera sensor to recognize the objects effectively. The point cloud of the LiDAR is enhanced and fused with the image space on the Regions of Interest (ROI) for easy recognition of the objects. The evaluation results show that the OD-C3DL can provide an average of 89 real-time objects for a frame and reduces the extraction time by a recall rate of 94%. The average processing time is 65ms, which makes the OD-C3DL model incredibly suitable for the AVs perception. Furthermore, OD-C3DL provides mean accuracy for identifying automobiles and pedestrians at a moderate degree of difficulty is higher than that of the previous models at 79.13% and 88.76%.

Automated extraction of geometric primitives with solid lines from unstructured point clouds for creating digital buildings models

Point clouds produced by laser scanners are an invaluable source of data for reconstructing multi-dimensional digital models that reflect the as-is conditions of built facilities. However, previous studies aimed to reconstruct models by overlaying the dataset on top of ground-truth reference models to manually adjust the accuracy of the output. Therefore, this paper describes the extraction of geometric primitives with solid lines—the simplest form of objectified data that computer-aided design systems can handle—from unorganized data points and creation of digital models of built facilities in a form of floor plan. The geometric primitives are extracted from 3D points by hybridizing machine learning algorithms, which are mean-shift clustering, non-convex hull, and random sample and consensus (RANSAC). This paper provides a solution for creating a new form of as-built model with high accuracy and robustness from scratch without the involvement of ground-truth solutions or manual adjustments.

The SPDE Approach to Matérn Fields: Graph Representations

This paper investigates Gaussian Markov random field approximations to nonstationary Gaussian fields using graph representations of stochastic partial differential equations. We establish approximation error guarantees building on the theory of spectral convergence of graph Laplacians. The proposed graph representations provide a generalization of the Matérn model to unstructured point clouds, and facilitate inference and sampling using linear algebra methods for sparse matrices. In addition, they bridge and unify several models in Bayesian inverse problems, spatial statistics and graph-based machine learning. We demonstrate through examples in these three disciplines that the unity revealed by graph representations facilitates the exchange of ideas across them.

GCN-Based Pavement Crack Detection Using Mobile LiDAR Point Clouds

Mobile Laser Scanning (MLS) system can provide high-density and accurate 3D point clouds that enable rapid pavement crack detection for road maintenance tasks. Supervised learning-based algorithms have been proved pretty effective for handling such a large amount of inhomogeneous and unstructured point clouds. However, these algorithms often rely on a lot of annotated data, which is labor-intensive and time-consuming. This paper presents a semi-supervised point-level approach to overcome this challenge. We propose a graph-widen module to construct a reasonable graph structure for point clouds, increasing the detection performance of graph convolutional networks (GCN). The constructed graph characterizes the local features from a small amount of annotated data, avoiding information loss and dramatically reduces the dependence on annotated data. The MLS point clouds acquired by a commercial RIEGL VMX-450 system are used in this study. The experimental results demonstrate that our method outperforms the state-of-the-art point-level methods in terms of recall, F1 score, and efficiency while achieving comparable accuracy.

Intensity Image-Based LiDAR Fiducial Marker System

The fiducial marker system for LiDAR is crucial for the robotic application but it is still rare to date. In this paper, an Intensity Image-based LiDAR Fiducial Marker (IILFM) system is developed. This system only requires an unstructured point cloud with intensity as the input and it has no restriction on marker placement and shape. A marker detection method that locates the predefined 3D fiducials in the point cloud through the intensity image is introduced. Then, an approach that utilizes the detected 3D fiducials to estimate the LiDAR 6-DOF pose that describes the transmission from the world coordinate system to the LiDAR coordinate system is developed. Moreover, all these processes run in real-time (approx 40 Hz on Livox Mid-40 and approx 143 Hz on VLP-16). Qualitative and quantitative experiments are conducted to demonstrate that the proposed system has similar convenience and accuracy as the conventional visual fiducial marker system. The codes and results are available at: https://github.com/York-SDCNLab/IILFM.