Three-dimensional Point Cloud Research Articles

Test Input Prioritization for 3D Point Clouds

3D point cloud applications have become increasingly prevalent in diverse domains, showcasing their efficacy in various software systems. However, testing such applications presents unique challenges due to the high-dimensional nature of 3D point cloud data and the vast number of possible test cases. Test input prioritization has emerged as a promising approach to enhance testing efficiency by prioritizing potentially misclassified test cases during the early stages of the testing process. Consequently, this enables the early labeling of critical inputs, leading to a reduction in the overall labeling cost. However, applying existing prioritization methods to 3D point cloud data is constrained by several factors: (1) inadequate consideration of crucial spatial information, and (2) susceptibility to noises inherent in 3D point cloud data. In this article, we propose PCPrior , the first test prioritization approach specifically designed for 3D point cloud test cases. The fundamental concept behind PCPrior is that test inputs closer to the decision boundary of the model are more likely to be predicted incorrectly. To capture the spatial relationship between a point cloud test and the decision boundary, we propose transforming each test (a point cloud) into a low-dimensional feature vector, toward indirectly revealing the underlying proximity between a test and the decision boundary. To achieve this, we carefully design a group of feature generation strategies, and for each test input, we generate four distinct types of features, namely spatial features, mutation features, prediction features, and uncertainty features. Through a concatenation of the four feature types, PCPrior assembles a final feature vector for each test. Subsequently, a ranking model is employed to estimate the probability of misclassification for each test based on its feature vector. Finally, PCPrior ranks all tests based on their misclassification probabilities. We conducted an extensive study based on 165 subjects to evaluate the performance of PCPrior, encompassing both natural and noisy datasets. The results demonstrate that PCPrior outperforms all of the compared test prioritization approaches, with an average improvement of 10.99% to 66.94% on natural datasets and 16.62% to 53% on noisy datasets.

Numerical Simulation Study Considering Discontinuous Longitudinal Joints in Soft Soil under Symmetric Loading

In shield tunneling, the joint is one of the most vulnerable parts of the segmental lining. Opening of the joint reduces the overall stiffness of the ring, leading to structural damage and issues such as water leakage. Currently, the Winkler method is commonly used to calculate structural deformation, simplifying the interaction between segments and soil as radial and tangential Winkler springs. However, when introducing connection springs or reduction factors to simulate the joint stiffness of segments, the challenge lies in determining the reduction coefficient and the stiffness of the springs. Currently, the hyperstatic reflection method cannot simulate the discontinuity effect at the connection of the tunnel segments, while the state space method overlooks the nonlinear interaction between the tunnel and the soil. Therefore, this paper proposes a numerical simulation method considering the interaction between the tunnel and the soil, which is subjected to compression rather than tension, and the discontinuity of the joints between the segments. The model structure and external load are symmetrical, resulting in symmetrical calculation results. This method is based on the soft soil layers and shield tunnel structures of the Shanghai Metro, and the applicability of the model is verified through deformation calculations using three-dimensional laser scanning point clouds of sections from the Shanghai Metro Line 5. When the subgrade reaction coefficient is 5000 kN/m3, the model can effectively simulate the deformation of operational tunnels. By adjusting the bending stiffness of individual connection springs, we investigate the influence of bending stiffness reduction on the bending moment, radial displacement, and rotational displacement of the ring. The results indicate that a decrease in joint bending stiffness significantly affects the mechanical response of the ring, and the extent and degree of this influence are correlated with the joint position and the magnitude of joint bending stiffness.

UAV vision-based crack quantification and visualization of bridges: system design and engineering application

Accurately measuring visible cracks in bridges is crucial for their structural health diagnosis, damage detection, performance evaluation, and maintenance planning. The primary means of visual crack detection still relies heavily on manual visual inspection, an inefficient process that can pose significant safety risks. This article develops a unmanned aerial vehicle (UAV) vision-based surface crack measurement methodology and visualization scheme for the bridges that can detect and measure cracks automatically with improved efficiency. The surface crack measurement methodology is achieved by designing a three-stage crack sensing system including the You Only Look Once-based crack recognition, U-shaped network-based crack segmentation, and deep-vision-based crack width calculation. This workflow is integrated into a comprehensive UAV inspection system, which is intended for operation at the field. The surface crack visualization scheme is accomplished by taking advantage of time-series image fusion, GPS information migration, and three-dimensional (3D) point cloud technique to reconstruct the 3D geometrical model of the tested bridge, which is convenient for unveiling the crack information in the bridge. The proposed methodology was successfully validated by a case study on an arch bridge. The achievement of this article promotes the UAV vision-based bridge’s surface crack inspection technology to a new status that no preparation for pasting calibration marker is needed, and crack identification, segmentation, and width calculation are realized promptly during the UAV flying on-site, as well as damage evaluation for bridges is visually fulfilled based on the reconstructed digital-graphical 3D model. The working environments and influencing factors to the developed system are sufficiently discussed. Certain limitations in the current application are pointed out for future improvements.

Application of 3D laser scanning technology for mapping and accuracy assessment of the point cloud model for the Great Achievement Palace heritage building

Heritage buildings represent history and act as vessels of human civilisation. The significance of these architectural phenomena increases as society increases, which renders their preservation and ethical use increasingly crucial. Hence, preserving heritage buildings is necessary for societal benefit. The accuracy of three-dimensional (3D) point cloud models of heritage buildings is important to their digital conservation. Traditional mapping methods typically require more time and human resources. Furthermore, the outcome is subject to measurement omissions, errors, and other issues, which are contrary to the protection of the object measured. The complex elements of ancient Chinese architecture render it challenging for traditional measuring techniques to accurately capture spatial structural information. The 3D laser scanning technology is a novel technology to obtain 3D data rapidly. In this paper, the Great Achievement Palace of the Confucian Temple in Yuci District, Jinzhong City, Shanxi Province, was used as an example to examine the application of 3D laser scanning technology to acquire point cloud models of heritage buildings. Field data collection for architectural heritage using 3D laser scanning technology requires measuring station setting analysis to ensure effect accuracy while considering the elements of target location and quantity. For the 3D point cloud model of the Great Achievement Palace, error analysis and accuracy assessment were conducted on the quality elements of alignment accuracy between scanning stations, point cloud model reliability, point cloud data noise condition, and whether the point cloud data were stratified. The findings demonstrated that the improved method quickly and accurately acquired 3D point cloud model data and obtained realistic spatial data models of the heritage building.

Implementation of MIMO Radar-Based Point Cloud Images for Environmental Recognition of Unmanned Vehicles and Its Application

High-performance radar systems are becoming increasingly popular for accurately detecting obstacles in front of unmanned vehicles in fog, snow, rain, night and other scenarios. The use of these systems is gradually expanding, such as indicating empty space and environment detection rather than just detecting and tracking the moving targets. In this paper, based on our high-resolution radar system, a three-dimensional point cloud image algorithm is developed and implemented. An axis translation and compensation algorithm is applied to minimize the point spreading caused by the different mounting positions and the alignment error of the Global Navigation Satellite System (GNSS) and radar. After applying the algorithm, a point cloud image for a corner reflector target and a parked vehicle is created to directly compare the improved results. A recently developed radar system is mounted on the vehicle and it collects data through actual road driving. Based on this, a three-dimensional point cloud image including an axis translation and compensation algorithm is created. As a results, not only the curbstones of the road but also street trees and walls are well represented. In addition, this point cloud image is made to overlap and align with an open source web browser (QtWeb)-based navigation map image to implement the imaging algorithm and thus determine the location of the vehicle. This application algorithm can be very useful for positioning unmanned vehicles in urban area where GNSS signals cannot be received due to a large number of buildings. Furthermore, sensor fusion, in which a three-dimensional point cloud radar image appears on the camera image, is also implemented. The position alignment of the sensors is realized through intrinsic and extrinsic parameter optimization. This high-performance radar application algorithm is expected to work well for unmanned ground or aerial vehicle route planning and avoidance maneuvers in emergencies regardless of weather conditions, as it can obtain detailed information on space and obstacles not only in the front but also around them.

Corrosion segmentation method of concrete drainage pipes based on point transformer

Deep learning based Closed Circuit Television (CCTV) image detection has achieved remarkable accuracy in identifying and segmenting concrete pipeline damages. However, the lack of in-depth information and requirement for large amounts of training data are two major drawbacks. Therefore, this study proposes a point cloud data segmentation method that incorporates depth information to address the issue of insufficient training data for depth models, enabling efficient and accurate segmentation. Firstly, 3D simulation technology is employed to construct a simulation dataset. Secondly, an optimized point transformation model is established to identify and segment the point cloud data. The proposed method achieved a test accuracy of 94.36 % by optimizing the network structure and the training strategy. The Mean Intersection over Union (MIoU) increased by 3.62 % and 10.3 %, respectively, compared with the classic PointNet++ and before adding simulation data, reaching 86.31 %. These results demonstrate that the proposed method exhibits high precision defect detection capability on three-dimensional point clouds of drainage pipelines.

Underground Target Extraction by Local Entropy Feature of Voxelized 3D Ground Penetrating Radar

Abstract. To solve the current problem of insufficient exploration of three-dimensional spatial information detected by 3D ground penetrating radar (3D GPR) and the data processing mainly based on the analysis and interpretation of two-dimensional slice images, a method is proposed to extract underground target based on the local entropy feature of discrete point clouds after the voxelization of 3D GPR data. First, the acquired 3D GPR data was voxelized into discrete three-dimensional point clouds. Then the local entropy feature of the voxelized point clouds over the entire region were calculated. The soil background and underground targets were distinguished by classifying them from multiple dimensions through Support Vector Machine (SVM). Finally, the urban road underground environment was taken as the research object, and this method was used for experimental analysis using measured data. The experimental results show that the accuracy of this method in extracting underground targets is as high as 90.1%, and the missing detection rate of missing underground targets is as low as 7.8%. The proposed method is accurate and effective, providing a new approach for 3D GPR to extract underground target.

Autonomous Landing Spot Detection for Unmanned Aerial Vehicles Based on Monocular Vision

Abstract. Enhancing the autonomous landing capability of unmanned aerial vehicles (UAVs) is of great significance for improving their operational efficiency and field survival capabilities. To this end, we propose a real-time autonomous landing spot detection method for UAVs. Firstly, the pose of the UAV at any given time and the initial three-dimensional point cloud of the scene are estimated using simultaneous localization and mapping (SLAM) techniques. Then, since the initial point cloud is sparse and cannot be used for terrain analysis, we generate a voxel-based elevation map of the scene, which can establish interconnectivity among the points. Finally, we propose a shift-box strategy to comprehensively analyze various terrain factors in the elevation map, determine the landing spot for UAVs, and update the landing spot in real time. The UAV flight experiments conducted in the real world have demonstrated the effectiveness and real-time performance of the proposed method.

Detecting and tracking a road-drivable area with three-dimensional point clouds and IoT for autonomous applications

Detecting and tracking a road-drivable area with three-dimensional point clouds and IoT for autonomous applications

An LLM-based vision and language cobot navigation approach for Human-centric Smart Manufacturing

Industry 5.0 prioritizes Human-centric Smart Manufacturing (HSM), aiming to enhance human operators’ well-being and needs. This necessitates collaborative robots with advanced natural interaction capabilities and improved perception, cognition, and action intelligence. The Large Language Model (LLM) exhibits strong reasoning abilities and generalization capabilities, which can significantly advance the development of HSM once integrated into human–robot interaction and collaboration. Accordingly, this paper explores part of LLM’s ability in the context of smart manufacturing, focusing on addressing interruptions in the manufacturing process caused by repetitive tool fetching. To alleviate this issue, a vision and language cobot navigation approach is innovatively adopted in the manufacturing environment that could be further used to assist operators in retrieving tools. Specifically, a real Human–Robot Collaboration (HRC) manufacturing scene is first reconstructed and annotated using Three-Dimensional (3D) point cloud techniques. Then the LLM is utilized for the Automated Guided Vehicle (AGV) to comprehend natural language commands and generate Python code to triger navigation actions. Finally, the Pathfinder algorithm is applied for corresponding path planning. The framework is implemented in the Artificial Intelligence (AI) Habitat simulator, and the case studies demonstrate that the AGV can accurately comprehend complex language instructions, empowering human operators to complete manufacturing tasks efficiently.

Application of UAV 3D Point Cloud Data in Highway Slope Disaster Investigation

The safety and stability of highway slopes have always been critical concerns in highway management and maintenance. However, mountain road slopes, characterized by their small scale, large quantity, and complex geological conditions, pose challenges for effective deformation detection, putting significant pressure on highway management and maintenance. To address the difficulty in identifying hidden hazards of highway slopes in mountainous areas, this study proposes the use of unmanned aerial vehicle (UAV) closed-range photogrammetry technology for slope inspection. The M3C2 (Multiscale Model-to-Model Cloud Comparison) algorithm is employed to accurately calculate differences in multi-phase three-dimensional point cloud data, enabling the detection of overall slope deformation with centimeter-level precision. Field verification conducted on a highway slope in Qingtian County, Zhejiang Province, demonstrated the applicability of UAV 3D point cloud data in investigating highway slope hazards, thereby enhancing inspection efficiency and quality and providing technical support for highway management and maintenance.

Research on prediction of water target motion status of unmanned surface vehicle based on lidar

To improve the perception ability of unmanned boats on water targets and improve the autonomous collision avoidance function of unmanned boats, the problem of target motion state prediction is studied and a method for predicting the motion state of unmanned water targets based on lidar is proposed. Firstly, a three-dimensional point cloud space model is established and the down-sampling and clutter filtering are completed through various filters. Secondly, based on European clustering target segmentation, a target tracking algorithm combined with 3D-IOU metric is proposed to achieve target tracking. Finally, the Kalman filter is introduced for denoising, combined with the iterative closest point algorithm (Iterative Closest Point, ICP) and the centroid calculation algorithm to realize the prediction of the target speed and course. After the test on the lake, the method can meet the real-time requirements, complete the accurate prediction of the target motion state and control the error at a low level. At the same time, this method can also lay the foundation for the development of subsequent autonomous navigation tasks of unmanned vehicles.

Research on slope algorithm based laser welding guidance technology for 3D vision inspection of thin-walled parts

Visual guidance technology advances quickly and so do the technical requirements for weld trajectory, which needs to fulfill performance indices for high precision, anti-interference and quick processing. In this study, the approach of getting 3D point cloud data by directly scanning the workpiece to be welded with a 3D scanner is used because it is more precise and reliable than processing gray-layer data obtained from a depth camera, turning it into a 3D point cloud. The weld boundary is retrieved by slope calculation for the butt weld after preprocessing the three-dimensional point cloud data (cut, down-sampling, and abnormal value removal). The center point is then determined. After that, a welding trajectory is produced by fitting the center point, which is transmitted back to the actuator to guide its welding work and realize the identification and tracking of the weld.

Nonlinear ray tracing in focused fields, part 2. Tracing the flux: tutorial

In this three-part paper series, we develop a method to trace the lines of flux through a three-dimensional wavefield by following a direction that is governed by the derivative of the phase at each point, a process that is best described as flux tracing but which we interchangeably name “nonlinear ray tracing.” In the first part we focused on the high-speed calculation of focused three-dimensional complex wavefields in the paraxial approximation for TEM00 and TEM01 laser modes. The algorithms developed in the first paper are first used to generate the three-dimensional grid of samples of the complex wavefield in the focal region. In this second part, we focus on tracing a flux through this three-dimensional point cloud. For a given “ray” at an arbitrary position in the 3D volume, interpolation of the three-dimensional samples is applied to determine the derivative of the phase (normal to the direction of propagation) at the ray position, which is then used to direct the ray as it “propagates” forward in a straight line over a short distance to a subsequent plane; the process is repeated between consecutive planes. The initial origin of the ray can be chosen arbitrarily at any point, and the ray can then be traced through the volume with appropriate interpolation. Results are demonstrated for focused wavefields in the absence of aberrations, corresponding to the cases highlighted in the first paper. Some of the most interesting results relate to focused Laguerre-Gaussian beams, for which the rays are found to spiral at different rates of curvature. In the third paper we extend the application of this algorithm to the investigation of lens aberration.

Measuring the profile of aircraft engine blades using spectral confocal sensors

The geometric parameters of aircraft engine blades need to be precisely measured to ensure the quality of the blades for the normal operation of aircraft engines. This study aims to address the challenges of existing measurement systems in balancing efficiency, accuracy, and completeness. Additionally, it aims to enhance the accuracy and robustness of the algorithm for extracting blade profile characteristic parameters. In this paper, a spectral confocal sensor is employed to establish a blade profile measurement system. The design includes a probe sampling strategy, and a standard ball is used to calibrate the sensor probe’s light emission direction and the precise rotation center of the turntable. The paper proposes the use of methods such as partition search algorithm, binary search, and curvature segmentation to process point cloud data of blade body and tenon. We have conducted experimental measurements on the blades of an aircraft engine. The acquired three-dimensional point cloud data of multiple sets of blade cross-sections and dovetail sections were processed. After calculation, the maximum measurement errors for chord length, maximum blade thickness, tenon width, bottom height, top angle, and bottom angle are −0.0036 mm, −0.00721 mm, −0.0102 mm, −0.00928 mm, 0.0086°, and −0.0058°, respectively. This process validates the effectiveness of the proposed method and has high measurement accuracy. Compared with the CMM method, this method is more accurate in measuring small pits and large curvature micro surfaces, with higher measurement integrity and higher measurement efficiency.

Building facade structure extraction method based on three-dimensional laser point cloud by considering semantic information

ABSTRACT Building facade structures serve as important data support for three-dimensional (3D) building modelling. A building facade structure extraction method based on 3D Laser Point Cloud Data (LPCD) by considering semantic information is proposed. The proposed method mainly involves three steps. First, an improved 3D LPCD semantic segmentation method is introduced to extract and label point clouds as the building structure and walls. Second, structure line segments are recognized from the initial building facade structure based on the Regulated Block RANSAC (RB-RANSAC) algorithm. Third, geometric information such as distance, vector and positional relationship between structure line segments is considered to optimize the initial building facade structure. The proposed method can effectively extract and fit segment structures by considering the semantic and geometric information, obtaining significantly accurate and concise building facade structures. Case studies on 3D LPCD of local buildings and open-source datasets (Semantic3D) prove the extraction accuracy and efficiency of the proposed method.

Evaluating a Novel Approach to Detect the Vertical Structure of Insect Damage in Trees Using Multispectral and Three-Dimensional Data from Drone Imagery in the Northern Rocky Mountains, USA

Remote sensing is a well-established tool for detecting forest disturbances. The increased availability of uncrewed aerial systems (drones) and advances in computer algorithms have prompted numerous studies of forest insects using drones. To date, most studies have used height information from three-dimensional (3D) point clouds to segment individual trees and two-dimensional multispectral images to identify tree damage. Here, we describe a novel approach to classifying the multispectral reflectances assigned to the 3D point cloud into damaged and healthy classes, retaining the height information for the assessment of the vertical distribution of damage within a tree. Drone images were acquired in a 27-ha study area in the Northern Rocky Mountains that experienced recent damage from insects and then processed to produce a point cloud. Using the multispectral data assigned to the points on the point cloud (based on depth maps from individual multispectral images), a random forest (RF) classification model was developed, which had an overall accuracy (OA) of 98.6%, and when applied across the study area, it classified 77.0% of the points with probabilities greater than 75.0%. Based on the classified points and segmented trees, we developed and evaluated algorithms to separate healthy from damaged trees. For damaged trees, we identified the damage severity of each tree based on the percentages of red and gray points and identified top-kill based on the length of continuous damage from the treetop. Healthy and damaged trees were separated with a high accuracy (OA: 93.5%). The remaining damaged trees were separated into different damage severities with moderate accuracy (OA: 70.1%), consistent with the accuracies reported in similar studies. A subsequent algorithm identified top-kill on damaged trees with a high accuracy (OA: 91.8%). The damage severity algorithm classified most trees in the study area as healthy (78.3%), and most of the damaged trees in the study area exhibited some amount of top-kill (78.9%). Aggregating tree-level damage metrics to 30 m grid cells revealed several hot spots of damage and severe top-kill across the study area, illustrating the potential of this methodology to integrate with data products from space-based remote sensing platforms such as Landsat. Our results demonstrate the utility of drone-collected data for monitoring the vertical structure of tree damage from forest insects and diseases.

Enhancing Building Point Cloud Reconstruction from RGB UAV Data with Machine-Learning-Based Image Translation.

The performance of three-dimensional (3D) point cloud reconstruction is affected by dynamic features such as vegetation. Vegetation can be detected by near-infrared (NIR)-based indices; however, the sensors providing multispectral data are resource intensive. To address this issue, this study proposes a two-stage framework to firstly improve the performance of the 3D point cloud generation of buildings with a two-view SfM algorithm, and secondly, reduce noise caused by vegetation. The proposed framework can also overcome the lack of near-infrared data when identifying vegetation areas for reducing interferences in the SfM process. The first stage includes cross-sensor training, model selection and the evaluation of image-to-image RGB to color infrared (CIR) translation with Generative Adversarial Networks (GANs). The second stage includes feature detection with multiple feature detector operators, feature removal with respect to the NDVI-based vegetation classification, masking, matching, pose estimation and triangulation to generate sparse 3D point clouds. The materials utilized in both stages are a publicly available RGB-NIR dataset, and satellite and UAV imagery. The experimental results indicate that the cross-sensor and category-wise validation achieves an accuracy of 0.9466 and 0.9024, with a kappa coefficient of 0.8932 and 0.9110, respectively. The histogram-based evaluation demonstrates that the predicted NIR band is consistent with the original NIR data of the satellite test dataset. Finally, the test on the UAV RGB and artificially generated NIR with a segmentation-driven two-view SfM proves that the proposed framework can effectively translate RGB to CIR for NDVI calculation. Further, the artificially generated NDVI is able to segment and classify vegetation. As a result, the generated point cloud is less noisy, and the 3D model is enhanced.

A Color- and Geometric-Feature-Based Approach for Denoising Three-Dimensional Cultural Relic Point Clouds.

In the acquisition process of 3D cultural relics, it is common to encounter noise. To facilitate the generation of high-quality 3D models, we propose an approach based on graph signal processing that combines color and geometric features to denoise the point cloud. We divide the 3D point cloud into patches based on self-similarity theory and create an appropriate underlying graph with a Markov property. The features of the vertices in the graph are represented using 3D coordinates, normal vectors, and color. We formulate the point cloud denoising problem as a maximum a posteriori (MAP) estimation problem and use a graph Laplacian regularization (GLR) prior to identifying the most probable noise-free point cloud. In the denoising process, we moderately simplify the 3D point to reduce the running time of the denoising algorithm. The experimental results demonstrate that our proposed approach outperforms five competing methods in both subjective and objective assessments. It requires fewer iterations and exhibits strong robustness, effectively removing noise from the surface of cultural relic point clouds while preserving fine-scale 3D features such as texture and ornamentation. This results in more realistic 3D representations of cultural relics.

A Coal Mine Tunnel Deformation Detection Method Using Point Cloud Data.

In recent years, the deformation detection technology for underground tunnels has played a crucial role in coal mine safety management. Currently, traditional methods such as the cross method and those employing the roof abscission layer monitoring instrument are primarily used for tunnel deformation detection in coal mines. With the advancement of photogrammetric methods, three-dimensional laser scanners have gradually become the primary method for deformation detection of coal mine tunnels. However, due to the high-risk confined spaces and distant distribution of coal mine tunnels, stationary three-dimensional laser scanning technology requires a significant amount of labor and time, posing certain operational risks. Currently, mobile laser scanning has become a popular method for coal mine tunnel deformation detection. This paper proposes a method for detecting point cloud deformation of underground coal mine tunnels based on a handheld three-dimensional laser scanner. This method utilizes SLAM laser radar to obtain complete point cloud information of the entire tunnel, while projecting the three-dimensional point cloud onto different planes to obtain the coordinates of the tunnel centerline. By using the calculated tunnel centerline, the three-dimensional point cloud data collected at different times are matched to the same coordinate system, and then the tunnel deformation parameters are analyzed separately from the global and cross-sectional perspectives. Through on-site collection of tunnel data, this paper verifies the feasibility of the algorithm and compares it with other centerline fitting and point cloud registration algorithms, demonstrating higher accuracy and meeting practical needs.