Point Cloud Research Articles

Associative graph convolution network for point cloud analysis

Since point cloud is the raw output of most 3D sensors, its effective analysis is in huge demand in the field of autonomous driving and robotic manipulation. However, directly processing point clouds is challenging because point clouds are a kind of disordered and unstructured geometric data. Recently, numerous graph convolution neural networks are proposed for introducing graph structure to point clouds yet far from perfect. Specially, DGCNN tries to learn local geometric of points in semantic space and recomputes the graph using nearest neighbors in the feature space in each layer. However, it discards all the information of the previous graph after each graph update, which neglects the relations between each dynamic update. To this end, we propose an associative graph convolution neural network (AGCN) which mainly consists of associative graph convolution (AGConv) and two kinds of residual connections. AGConv additionally considers the information from the previous graph when computing the edge function on current local neighborhoods in each layer, and it can precisely and continuously capture the local geometric features on point clouds. Residual connections further explore the semantic relations between layers for effective learning on point clouds. Extensive experiments on several benchmark datasets show that our network achieves competitive classification and segmentation results.

Boosting point cloud understanding through graph convolutional network with scale measurement and high-frequency enhancement

Graph-based methods have exhibited exceptional performance in point cloud understanding by capturing local geometric relationships. However, existing approaches often struggle to characterize the overall spatial scale of local graphs. In addition, they fail to capture the differences between nodes effectively, which is crucial for distinguishing different classes. This study introduces SM-HFEGCN, a novel graph convolutional network that addresses these limitations through two key innovations: scale measurement and high-frequency enhancement. First, we introduce a spatial scale feature derived from the diagonal vectors of the neighborhood, which serves as a unique graph-specific property related to the geometry and density of the local point cloud. This feature can characterize the overall spatial scale of the local point cloud. Second, we enhance the high-frequency information to capture node variations and integrate it with smoothed information to represent the differences and similarities between nodes simultaneously. Extensive experiments demonstrate the effectiveness of SM-HFEGCN in point cloud classification and segmentation tasks.

GLDBF: Global and local dual-branch fusion network for no-reference point cloud quality assessment

No-reference Point Cloud Quality Assessment (NR-PCQA) is a challenge in the field of media quality assessment, such as inability to accurately capture quality-related features due to the unique scattered structure of points and less considering global features and local features jointly in the existing no-reference PCQA metrics. To address these challenges, we propose a Global and Local Dual-Branch Fusion (GLDBF) network for no-reference point cloud quality assessment. Firstly, sparse convolution is used to extract the global quality feature of distorted Point Clouds (PCs). Secondly, graph weighted PointNet++ is proposed to extract the multi-level local features of point cloud, and the offset attention mechanism is further used to enhance local effective features. Transformer-based fusion module is also proposed to fuse multi-level local features. Finally, we joint the global and local dual branch fusion modules via multilayer perceptron to predict the quality score of distorted PCs. Experimental results show that the proposed algorithm can achieves state-of-the-art performance compared with existing methods in assessing the quality of distorted PCs.

Research on laser measurement point cloud preprocessing and 3D reconstruction technology for free-form surfaces.

Surface morphology measurement and reconstruction technology based on point cloud data is one of the key technologies for 3D information processing in the digital manufacturing industry and has been widely applied in fields such as reverse engineering, computer vision, and unmanned driving system navigation. A method for 3D modeling of aircraft-engine blade profiles based on laser measurement point cloud data is proposed to address the difficulties in measuring the 3D morphology of aircraft-engine blades and the low modeling accuracy. This method first preprocesses the measured point cloud and then uses Poisson's algorithm to reconstruct the blade surface in three dimensions based on the calculation of the point cloud normal. Through error statistical analysis, the overall reconstruction effect is good. The experimental results further validated the generality and effectiveness of this method.

Gradient-based sparse voxel attacks on point cloud object detection

Point cloud object detection is crucial for a variety of applications, including autonomous driving and robotics. Voxel-based representation for 3D point clouds has drawn significant attention due to their efficiency and effectiveness. Recent studies have revealed the vulnerability of deep learning models to adversarial attacks, while considerably less attention is paid to the robustness of voxel-based point cloud object detectors. Existing adversarial attacks on the point cloud data involve generating fake obstacles, removing objects or producing fake predictions. Despite the demonstrated success, these approaches have three limitations. First, manipulating point data, which was originally designed for point-based representation, is inapplicable to voxel-based representation. Second, existing works that modified points in the hold scene yield redundant perturbations. Third, the evaluation primarily performed on small-scale datasets, such as KITTI, does not scale well. To address these limitations, we propose a gradient-based sparse voxel attack (GSVA) algorithm for voxel-based 3D point cloud object detectors. Two novel frameworks, i.e., re-voxelization-based voxel attack framework and light voxel attack framework, successfully modify voxel-based representation instead of raw points. In addition to KITTI, extensive experiments on large-scale datasets including nuScenes and Waymo Open Dataset demonstrate the favorable attack performance (with mAP decrease by 86.2%∼99.5%) and the slight perturbation costs (with lowest modification rate of 3.5%) of our voxel attack method over the state-of-the-art approaches.

Multimodal feature integration network for lithology identification from point cloud data

Multimodal feature integration network for lithology identification from point cloud data

Construction error control method of large-span spatial structures based on digital twin

In the process of construction of large-span spatial structures, there are many control elements, which need to carry on the dual control of internal force and shape. The traditional control method cannot measure the construction deviation completely and accurately, and there is a lag in information feedback. In this paper, spatio-temporal information is integrated into digital twin and a construction error control method is proposed for large-span spatial structures. By analyzing the fusion mechanism of spatio-temporal information and digital twin, a closed-loop control system for structural errors before, during and after construction is formed. Driven by the control architecture, the method of structure simulation analysis and real-time monitoring during construction is proposed, and the functional relationship between cable force and cable length is established. Based on point cloud data, construction error optimization detection method is proposed for structure formation state. The nonlinear relation between elevation change rate and cable force deviation is formed and the control measures of formation state error are obtained. The experimental model of cable truss structure is taken as an example to apply and verify the theoretical method. The construction error control twin platform is formed in the experiment. Driven by the platform, the twinning analysis and real-time monitoring of the construction process are carried out to control the error of the structure formation. The results show that this method integrates spatio-temporal information into digital twin to realize data integration processing. The optimized error detection method can save 30% time cost and effectively control the construction error within 3%. The integration of spatio-temporal information and digital twin provides theoretical and algorithm support for the intelligent management and control of large-span spatial structure construction.

Architectural simulation from point clouds: Between precision and historical validity

Architectural simulation from point clouds: Between precision and historical validity

Fast Building Instance Proxy Reconstruction for Large Urban Scenes.

Digitalization of large-scale urban scenes (in particular buildings) has been a long-standing open problem, which attributes to the challenges in data acquisition, such as incomplete scene coverage, lack of semantics, low efficiency, and low reliability in path planning. In this paper, we address these challenges in urban building reconstruction from aerial images, and we propose an effective workflow and a few novel algorithms for efficient 3D building instance proxy reconstruction for large urban scenes. Specifically, we propose a novel learning-based approach to instance segmentation of urban buildings from aerial images followed by a voting-based algorithm to fuse the multi-view instance information to a sparse point cloud (reconstructed using a standard Structure from Motion pipeline). Our method enables effective instance segmentation of the building instances from the point cloud. We also introduce a layer-based surface reconstruction method dedicated to the 3D reconstruction of building proxies from extremely sparse point clouds. Extensive experiments on both synthetic and real-world aerial images of large urban scenes have demonstrated the effectiveness of our approach. The generated scene proxy models can already provide a promising 3D surface representation of the buildings in large urban scenes, and when applied to aerial path planning, the instance-enhanced building proxy models can significantly improve data completeness and accuracy, yielding highly detailed 3D building models.

Automatic 3-dimensional quantification of orthodontically induced root resorption in cone-beam computed tomography images based on deep learning

Orthodontically induced root resorption (OIRR) is a common and undesirable consequence of orthodontic treatment. Traditionally, studies employ manual methods to conduct 3-dimensional quantitative analysis of OIRR via cone-beam computed tomography (CBCT), which is often subjective and time-consuming. With advancements in computer technology, deep learning-based approaches have gained traction in medical image processing. This study presents a deep learning-based model for the fully automatic extraction of root volume information and the localization of root resorption from CBCT images. In this cross-sectional, retrospective study, 4534 teeth from 105 patients were used to train and validate an automatic model for OIRR quantification. The protocol encompassed several steps: preprocessing of CBCT images involving automatic tooth segmentation and conversion into point clouds, followed by segmentation of tooth crowns and roots via the Dynamic Graph Convolutional Neural Network. The root volume was subsequently calculated, and OIRR localization was performed. The intraclass correlation coefficient was employed to validate the consistency between the automatic model and manual measurements. The proposed method strongly correlated with manual measurements in terms of root volume and OIRR severity assessment. The intraclass correlation coefficient values for average volume measurements at each tooth position exceeded 0.95 (P<0.001), with the accuracy of different OIRR severity classifications surpassing 0.8. The proposed methodology provides automatic and reliable tools for OIRR assessment, offering potential improvements in orthodontic treatment planning and monitoring.

Virtual ADA Compliance Assessment: Mimicking Digital Inclinometers to Measure Slopes within Point Clouds

Virtual ADA Compliance Assessment: Mimicking Digital Inclinometers to Measure Slopes within Point Clouds

Research and application on deep learning-based point cloud completion for marine structures with point coordinate fusion and coordinate-supervised point cloud generator

Research and application on deep learning-based point cloud completion for marine structures with point coordinate fusion and coordinate-supervised point cloud generator

Open source tool for Micro-CT aided meso-scale modeling and meshing of complex textile composite structures

Volumetric image-based modeling of textile reinforcements and composites is favored over ideal geometric modeling because of its ability to represent complex structures in sufficient detail. Although several approaches were devised, there is still a scarcity of dedicated tools capable of effectively transferring pertinent information from images to high-fidelity models. This work presents the open source project, PolyTex, a Python-based object-oriented application that establishes a streamlined and reproducible workflow for such tasks. Dual kriging serves as the foundational theory for the parametric approach developed to represent, simplify, and approximate the morphology and topology of fiber tows. The code takes two types of input, either an explicit representation of tow geometry using point clouds or implicit representations, such as image masks representing fiber tows separately with grayscale values. Tailored APIs allow for smooth integration between PolyTex’s modeling capabilities and the simulation environments offered by OpenFOAM and Abaqus. Case studies on virtual testing of textile permeability were presented to demonstrate this capability. The modular and object-oriented design makes PolyTex a highly reusable and extensible tool that allows users to create a customized pipeline.

Denaturants and Solutol® HS15 in ophthalmic formulations: Insights into their combined effects

In the field of ophthalmology, significant attention is dedicated to developing micellar carriers to enhance ocular drug delivery. Urea is a well-known denaturant and has been used as an excipient in several ocular formulations due to its efficacy as an ocular hypotensive agent. The presence of urea as an excipient may influence the characteristics of the micellar nanocarrier in the formulation and needs to be investigated. Solutol® HS15, a well-known nonionic surfactant and a commonly used nanocarrier in drug formulations, enhances the solubilization and stability of ocular drugs. However, a detailed study of its interaction with various pharmaceutical excipients is still lacking. With this intention, the present study focuses on comprehending the interaction between Solutol® HS15 micelles with urea and its derivatives. Here, we present a detailed analysis on the influence of urea and its derivatives (methylurea, dimethylurea) on Solutol® HS15 using small-angle neutron scattering (SANS), dynamic light scattering (DLS), and cloud point (CP) measurements. The presence of urea and its derivatives strongly affects the CP, leading to a significant increase in its values when urea derivatives are present. Additionally, the SANS and DLS results also indicate a significant decrease in micellar size, suggesting demicellization. The observed effects of urea derivatives on Solutol® HS15 micelles underscore the importance of considering solvent-surfactant interactions in understanding the behavior of surfactant-based systems. This research will provide greater insight into the interactions between urea and its derivatives and Solutol® HS15, which will assist in understanding the morphology and size distribution of surfactant micelles for drug delivery applications.

Clouding of Triton X-100 solutions in the presence of resorcinol or β-naphthol and their application for mercury cloud point extraction

The clouding behavior of Triton X-100 solutions was investigated in the presence of resorcinol or β-naphthol. Both agents can effectively decrease the cloud point of Triton X-100 solutions. Resorcinol is active over a wider pH range, while β-naphthol is required at concentrations 10–50 times lower than resorcinol. It was also shown that about 60–70 % of β-naphthol transfers to the micellar phase, whereas resorcinol barely enters the micellar phase (about 10–15 %). Thermodynamic parameters of the clouding process were also calculated, revealing no significant differences between thermodynamic parameters for these two substances studied. Based on the results obtained, we assume different mechanisms of phase separation by these agents. Using the systems studied, mercury cloud point extraction was successfully carried out with 8-hydroxyquinoline as a ligand. The optimal parameters for extraction were determined as follows: pH of 8 (β-naphthol) and 6 (resorcinol), ligand concentration of 0.03–0.26 mmol L−1, and surfactant concentration of 1.7 g L−1.

Enhancing Preservation: Addressing Humidity Challenges in Indonesian Heritage Buildings Through Advanced Detection Methods Point Cloud Data

Heritage buildings are valuable assets that represent national cultural identity. Proper building maintenance is a major issue for preservation, as building monitoring aspects and preventive measures are often only taken after physical damage happens. In the context of Indonesian heritage buildings, high levels of humidity which may cause condensation and soil dampness are often overlooked. Early detection methods are urgently required to effectively detect potential risks of condensation. This study aims to investigate condensation risk for heritage building surfaces by calculating thermal properties (i.e., emissivity, albedo) and Blinn-Phong BRDF values through the integration of thermal imaging and 3D scanning techniques. This approach supports architects and conservators in making informed decisions to protect and maintain cultural heritage structures. The study also highlights gaps in current Indonesian regulations regarding moisture presence and condensation risk detection in heritage buildings.

Automatic identification of discontinuities and refined modeling of rock blocks from 3D point cloud data of rock surfaces

Automatic identification of discontinuities and refined modeling of rock blocks from 3D point cloud data of rock surfaces

Defect focused Harris3D & boundary fine-tuning optimized region growing: Lithium battery pole piece defect segmentation

As a vital component of lithium battery, the pole piece is prone to defects during the production process, which adversely affects performance and even causes safety accidents. Therefore, conducting quality inspections for lithium battery pole pieces is crucial. In this research, an optimized region growing algorithm based on 3D point cloud data is proposed to automatically detect pole piece defects. Specifically, a potential defect-focused Harris3D method (DFHarris3D) for keypoint detection and a boundary fine-tuning algorithm (BFT) are introduced. Comparative experimental results prove that our method outperforms others in terms of completeness, accuracy, and robustness. Therefore, this novel non-destructive detection technique demonstrates the potential to accurately segment surface defects in lithium battery pole pieces, which can be integrated into smart factories for fully automated defect detection.

A calibration method of line laser 3D gear measurement system

It is of great importance to ensure the accuracy of measurements taken by the line laser three-dimensional (3D) gear measurement system through the calibration of system in an accurate manner. This paper established a relevant measurement model based on the principle of line laser 3D gear measurement using generalized polar coordinates, designed a dedicated Polyhedral Artefact (PA), and proposed an accurate calibration method to address the challenges posed by the representation of 6 degrees of freedom (DOF) parameters for line laser sensor data in measurement space. Among them, the PA includes features such as cylindrical plane, planar plane, and V-groove etc., which can meet the requirements of high-precision machining while maintaining consistency with gear installation accuracy to further improve the accuracy of sensor pose parameters. The calibration method uses nonlinear optimization equations with point cloud data to simultaneously solve the required parameters for accurate measurement. Calibration and gear measurement experiments reveal that the standard deviation and range deviation of the total tooth profile deviation, obtained from 10 repeated measurements of a single tooth flank, are 0.244 μm and 0.86 μm, respectively. The results of the tooth profile measurements after the calibration of the system show good consistency compared with the contact measurements, which verifies the effectiveness of the method.

2S-SGCN: A two-stage stratified graph convolutional network model for facial landmark detection on 3D data

Facial Landmark Detection (FLD) algorithms play a crucial role in numerous computer vision applications, particularly in tasks such as face recognition, head pose estimation, and facial expression analysis. While FLD on images has long been the focus, the emergence of 3D data has led to a surge of interest in FLD on it due to its potential applications in various fields, including medical research. However, automating FLD in this context presents significant challenges, such as selecting suitable network architectures, refining outputs for precise landmark localization and optimizing computational efficiency. In response, this paper presents a novel approach, the 2-Stage Stratified Graph Convolutional Network (2S-SGCN), which addresses these challenges comprehensively. The first stage aims to detect landmark regions using heatmap regression, which leverages both local and long-range dependencies through a stratified approach. In the second stage, 3D landmarks are precisely determined using a new post-processing technique, namely MSE-over-mesh. 2S-SGCN ensures both efficiency and suitability for resource-constrained devices. Experimental results on 3D scans from the public Facescape and Headspace datasets, as well as on point clouds derived from FLAME meshes collected in the DAD-3DHeads dataset, demonstrate that the proposed method achieves state-of-the-art performance across various conditions. Source code is accessible at https://github.com/gfacchi-dev/CVIU-2S-SGCN.