Surface Point Cloud Research Articles

Geometric defects identification and deviation compensation in laser deposition manufacturing

Geometric defect can result in a failed part and is costly in terms of time and material. Detecting and control defect is essential for apartbyadditivemanufacturing to avoid further deterioration of the part accuracy. In this paper, a defect identification and compensation method for laser deposition manufacturing(LDM) is proposed to enhance the part quality. This work investigated the use of high-speed laser profiler to collect the surface point cloud concerning a part under building, and then these data were utilized to detect the defect region according to the given deviation threshold. Statistic filter, least-square method and projection approach were employed for deviation point cloud denoising, segmentation and contour extraction. An algorithm was presented to distinguish regions which needed compensation and generated corresponding scan path based on in-situ point cloud processing. The experimental results showed that the setup successfuldetected the geometric defects of the LDM part and controlled the machine motion system to deposit the deviation region, and the resulting part was more geometrically identical to the expected 3D representation.

INDEXED 3D SCENE LAYERS (I3S) – AN EFFICIENT ENCODING AND STREAMING OGC COMMUNITY STANDARD FOR MASSIVE GEOSPATIAL CONTENT

Abstract. Indexed 3D Scene Layers (I3S), an OGC Community Standard for streaming and storing massive amounts of geospatial content has been rapidly evolving to capture new use cases and techniques to advance geospatial visualization and analysis. I3S enables efficient transmission of various 3D geospatial data types including discrete 3D objects with attributes, integrated surface meshes and point cloud data covering vast geographic areas as well as highly detailed BIM (Building Information Model) content, to web browsers, mobile apps and desktop.In this paper, we will explore multiple evolutions of I3S, including the latest, OGC I3S Version 1.2, that brings dramatic improvements in performance and scalability. We will demonstrate the advantages, including its support for a paged node access pattern, a more compact geometry layout, advanced material definitions property that supports PBR, as well as its support for supercompression of texture data using the Basis Universal SuperCompressed Texture format in KTX™2.0 containers. We will also demonstrate collaborative & research work done to dramatically improve in Basis compressed texture creation in KTX™2.0 container leveraging both the CPU and GPU – contributions that benefit both the geospatial and 3d graphics communities.The paper will conclude by highlighting and documenting various use cases and application, where formats such as I3S are pushing the envelope in geospatial technology – by enabling seamless and ubiquitous access to vast amounts of geospatial data which traditionally have required specialized hardware and software platforms.

Estimation of mean dominant height using NAIP digital aerial photogrammetry and lidar over mixed deciduous forest in the southeastern USA

• Temperate deciduous tree heights estimated with point clouds from digital aerial photogrammetry. • Photogrammetrically-derived point clouds a possible surrogate for those derived from lidar data. • Lidar-derived DEM necessary to normalize point clouds derived from digital aerial photogrammetry. • A poor photogrammetric solution led to reduced estimation accuracy in some shadowed areas. In the absence of complete lidar coverage, digital surface models (DSMs) and point clouds produced from the United States Department of Agriculture National Agriculture Imagery Program (NAIP) are increasingly being analyzed for quality and application feasibility. This study compared canopy heights derived from NAIP DSMs (10 m) and point clouds to those derived from lidar data collected over Mountain Lake Biological Station and the Great Smoky Mountains Twin Creeks Site by the National Ecological Observatory Network (NEON) Airborne Observation Platform for 62 mixed deciduous tree plots. Mean dominant height (MDH) was estimated using lidar and the NAIP products using the 90th percentile of heights in a given plot as the independent variable for both the lidar- and NAIP-derived point clouds. The dependent variable was field-measured MDH, calculated using the four tallest trees for each 0.04-hectare plot based on the NEON woody vegetation structure dataset. All data (field and remotely sensed) were collected in 2018. Using maximum likelihood spatial error model for all analyses, the NAIP DSM (10 m resolution) resulted in a strong relationship with MDH (coefficient of determination ( R 2 ) = 0.90, standard error ( SE ) = 1.71 m). However, the 90th percentiles of heights derived from the point clouds were better at estimating MDH than was the comparatively coarse resolution DSM (NAIP point clouds: R 2 = 0.94, SE = 1.40 m; lidar: R 2 = 0.95, SE = 1.29 m, respectively) and are strongly correlated to each other ( R 2 = 0.99, SE = 0.68 m). The main limitation of the NAIP datasets was found to be where shadowing occurred due to steep terrain in the Great Smoky Mountain site. These areas resulted in erroneously high vegetation heights. Mean dominant heights estimated using NAIP DSMs and point clouds are thus comparable to those estimated using lidar data in these closed-canopy temperate deciduous forests where shadowing from steep terrain is not present. The utility of both the NAIP-derived 10 m DSM and the point clouds for estimating tree heights paves the way for statewide mapping of heights over the deciduous forests in Tennessee, Virginia, and possibly beyond.

Robust Robotic 3-D Drawing Using Closed-Loop Planning and Online Picked Pens

This article develops a flexible and robust robotic system for autonomously drawing on 3-D surfaces. The system takes 2-D drawing strokes and a 3-D target surface (mesh or point clouds) as input. It maps the 2-D strokes onto the 3-D surface and generates a robot motion to draw the mapped strokes using visual recognition, grasp pose reasoning, and motion planning. The system is flexible compared to conventional robotic drawing systems as we do not fix drawing tools to the end of a robot arm. Instead, a robot recognizes and picks up pens online and holds the pens to draw 3-D strokes. Meanwhile, the system has high robustness thanks to the following crafts: First, a high-quality mapping method is developed to minimize deformation in the strokes. Second, visual detection is used to reestimate the drawing tool’s pose before executing each drawing motion. Third, force control is employed to compensate for noisy visual detection and calibration and ensure a firm touch between the pen tip and the surface. Fourth, error detection and recovery are implemented to deal with slippage and other anomalies. The planning and executions are performed in a closed-loop manner until the strokes are successfully drawn. We evaluate the system and analyze the necessity of the various crafts using different real-world tasks. The results show that the proposed system is flexible and robust to generate robotic motion that picks up the pens and successfully draws 3-D strokes on given surfaces.

A point cloud registration-based calibration algorithm for robot offline programming automatic loading in aero-grinding applications

PurposeDuring the process of the robotic grinding and polishing operations on aero-engine blades, the key problem of calibration error lies in fixture error and uneven margin. To solve this problem, this paper aims to propose a novel method to achieve rapid online calibration of the workpiece coordinate system through laser-based measurement techniques.Design/methodology/approachThe authors propose a calibration strategy based on point cloud registration algorithm. The main principle is presented as follows: aero blade mounted on clamping end-effector is hold by industry robot, the whole device is then scanned by a 3D laser scanner to obtain its surface point cloud, and a fast segmentation method is used to acquire the point cloud of the workpiece. Combining Super4PCS algorithm with trimmed iterative closest point, we can align the key points of the scanned point cloud and the sampled points of the blade model, thus obtaining the translation and rotation matrix for calculating the workpiece coordinate and machining allowance. The proposed calibration strategy is experimentally validated, and the positioning error, as well as the margin distribution, is finally analyzed.FindingsThe experimental results show that the algorithm can well accomplish the task of cross-source, partial data and similar local features of blade point cloud registration with high precision. The total time spent on point cloud alignment of 100,000 order of magnitude blade is about 4.2 s, and meanwhile, the average point cloud alignment error is reduced to below 0.05 mm.Originality/valueAn improved point cloud registration method is proposed and introduced into the calibration process of a robotic system. The online calibration technique improves the accuracy and efficiency of the calibration process and enhances the automation of the robotic grinding and polishing system.

Automatic livestock body measurement based on keypoint detection with multiple depth cameras

The body measurement of livestock is an important task in precision livestock farming. To reduce the cost of manual measurement, an increasing number of studies have proposed non-contact body measurement methods using depth cameras. However, these methods only use 3D data to construct geometric features for body measurements, which is prone to error on incomplete and noisy point clouds. This paper introduces a 2D-3D fusion body measurement method, developed in order to exploit the potential of raw scanned data including high-resolution RGB images and 3D spatial information. The keypoints for body measurement are detected on RGB images with a deep learning model. Then these keypoints are projected onto the surface of livestock point clouds by utilizing the intrinsic parameters of the camera. Combining the process of interpolation and the pose normalization method, 9 body measurements of cattle and 5 body measurements of pig (including body lengths, body widths, body heights, and heart girth) are measured. To verify the feasibility of this method, the experiments are performed on 103 cattle data and 13 pig data. Compared with manual measurements, the MAPEs (mean absolute percentage errors) of 5 cattle body measurements and 1 pig body measurement are reduced to less than 10%. Body widths are more susceptible to non-standard posture. The MAPEs of 2 cattle body widths are larger than 20% and the MAPE of 1 pig body width reaches 30%. In comparison with a previous girth measurement method, the presented method is more accurate and robust for the cattle dataset. The same approach can be adapted and implemented for non-contact body measurement for different livestock species.

Monte Carlo-based 3D surface point cloud volume estimation by exploding local cubes faces

<p class="Abstract">This article proposes a state-of-the-art algorithm for estimating the 3D volume enclosed in a surface point cloud via a modified extension of the Monte Carlo integration approach. The algorithm consists of a pre-processing of the surface point cloud, a sequential generation of points managed by an affiliation criterion, and the final computation of the volume. The pre-processing phase allows a spatial re-orientation of the original point cloud, the evaluation of the homogeneity of its points distribution, and its enclosure inside a rectangular parallelepiped of known volume. The affiliation criterion using the explosion of cube faces is the core of the algorithm, handles the sequential generation of points, and proposes the effective extension of the traditional Monte Carlo method by introducing its applicability to the discrete domains. Finally, the final computation estimates the volume as a function of the total amount of generated points, the portion enclosed within the surface point cloud, and the parallelepiped volume. The developed method proves to be accurate with surface point clouds of both convex and concave solids reporting an average percentage error of less than 7 %. It also shows considerable versatility in handling clouds with sparse, homogeneous, and sometimes even missing points distributions. A performance analysis is presented by testing the algorithm on both surface point clouds obtained from meshes of virtual objects as well as from real objects reconstructed using reverse engineering techniques.</p>

A free‐form surface flattening algorithm that minimizes geometric deformation energy

Based on the analysis of the advantages and disadvantages of existing surface flattening algorithms, this paper proposes a free-form surface flattening algorithm that minimizes geometric deformation by combining the advantages of the geometric flattening method and the mechanical energy flattening method to address the problems of many iterations, large changes in convergence, and weak visualization of deformation. The point cloud surface is meshed using the triangular slice search method, and the 3D surface mesh is wrapped around the surface using geometric mapping relationships. The initial correction and deformation analysis of the ring flatten graphic are carried out according to the average flattening error, and a global geometric deformation energy model is established to obtain the energy-minimizing unfolding conditions and optimize the initial flattening graph by iterative optimization. The verification of the algorithm shows that the algorithm is general, has good robustness, high flattening accuracy, and visualizes the flattening deformation. The method is suitable for some design occasions of machining, such as the flattening calculation of automobile outer cladding parts and sheet metal parts. It is especially suitable for the flattening calculation of ring-like parts, and is a good guide for the design calculation and stamping process of sheet-metal-like parts.

Body size measurement and live body weight estimation for pigs based on back surface point clouds

Body measurement is an important link in monitoring the growth of livestock in breeding management. Conventional measurements are notoriously difficult, unstable, stressful and time-consuming. 3D Computer vision technology has demonstrated its superiority in body size measurement and live body weight estimation because it has advantages of being both non-contact and effective. The objective of this paper was validating the use of single-frame surface point clouds to measure the body size, and develop a model to estimate the pig weight. With a limit bar not applied, a Kinect camera was fixed above the drinking area allowed multiple images of pigs to be captured at one time in a group-house. Body size parameters were extracted from back surface point clouds, and subsequently used as independent variables to build the three regression analysis models: i) stepwise regression analysis, ii) ridge regression analysis, and iii) partial least squares regression analysis. An experiment was conducted on fifty pigs in a group-house, half of which were Landrace gilts, half were Yorkshire shoats. The result of size measurement showed the mean relative absolute errors of body length, height, shoulder width were 0.7%, 1.8%, and 3.3%, respectively. Final ridge regression equation was developed with an R2 of 0.958 and a MAE of 2.961 kg. It was concluded that the proposed methods based on Kinect v2 was shown to be accurate and reliable for body size measurement and live body mass estimation.

FEATURE FUSION-BASED FEATURE POINT EXTRACTION AND MATCHING OF THE THREE-DIMENSIONAL HUMAN VERTEBRAE MODEL

In the positioning and registration of a three-dimensional human vertebrae model, the main difficulty encountered is the extraction and matching of the model feature points. To accurately locate the vertebrae, it is necessary to optimize the existing feature point extraction and matching method. In this context, the present study was envisaged to achieve the purpose of accurate extraction and matching by integrating the features of curvature and Euclidean distance. It was found that this method removed the redundant point cloud data in the 3D model, thereby effectively improving the matching rate. Initially, the feature points were manually picked as the circle center, and the mean value ([Formula: see text]) of average curvature ([Formula: see text]) of all points in the specified infinitesimal radius ([Formula: see text]) sphere neighborhood were calculated. When [Formula: see text], it implied the point [Formula: see text] tended to be flat relative to the whole point cloud surface, and was not regarded as a feature point. When [Formula: see text], the point [Formula: see text] exhibited a greater curvature relative to the whole point cloud surface, that is, a more prominent point. In this case, point [Formula: see text] met the basic conditions for becoming a feature point and was included in the candidate feature point set. Subsequently, the Euclidean distance between the points in the candidate feature point set and the manually picked circle center points was calculated individually. The smaller the Euclidean distance between the two points, the higher the degree of feature similarity between the two points, and the stronger the matching of feature points. Therefore, the manually picked points were replaced with the candidate feature point with the closest distance that accurately reflected the change of the local geometric characteristics of the point. It was found that the accuracy of the improved method of the feature point extraction and matching was about 18% higher than that before the improvement, which verifies the effectiveness and robustness of the proposed method.

Estimation of Crop Height Distribution for Mature Rice Based on a Moving Surface and 3D Point Cloud Elevation

Estimation of rice plant height distribution plays a significant role in keeping the feed rate of rice combine harvesters stable. This is an effective way to guarantee the working stability of the whole machine, as a consequence, improving threshing and cleaning efficiency and reducing loss and damage rates. However, dense growth and leafy and bent branches of mature rice make it difficult to detect the lowest point of aggregated growing plants in three dimensional (3D) point cloud data. Therefore, the objective of this study was to put forward a method to estimate plant height distribution on the basis of a moving surface and 3D point cloud elevation. The statistical outlier removal (SOR) algorithm was used to reduce noise points far away from target point cloud body, and then moving surface fitting elevation was applied to achieve accurate classification of ground and crop point cloud data for plant height estimation. Experiments showed that, compared with the actual value, the average square root error (RMSE) of the estimation results was 8.29, the average absolute percentage error (MAPE) was 9.28%, and the average accuracy was 90%. The proposed method could accurately estimate the height of mature rice and is beneficial to calculating the feed rate in advance, which can provide a reference for further investigation in automatic and intelligent harvesting.

Scale-dependent rock surface characterization using LiDAR surveys

Evaluating topographical roughness along exposed rock surfaces is to a large extent subjective to the intended application and the related measurement scale. In tunneling and mining applications, topographical roughness of rock surfaces is a key factor in assessing the performance of extractive processes such as blasting backbreak, haulage surface condition and excavation geometry. The scale used to measure surface roughness should be “fit for purpose”, both in terms of the scale of the measurements and the scale required for the practical interpretation. This paper presents a scale-dependent method to characterize the overall surface roughness of rock, from 3D point clouds of rock surfaces collected using terrestrial laser scanner (LiDAR) surveys. A power law formulation is used to quantify the overall surface roughness as a function of the scale of measurement. The proposed methodology is described in detail and applied to different operational settings at two underground mines to evaluate various aspects of the extraction processes. Special consideration is given to the scale dependency of the power law relationship describing the main trend for the overall surface roughness, and how this can be linked to the evaluation of specific excavation processes.

Cross-Domain Self-Supervised Complete Geometric Representation Learning for Real-Scanned Point Cloud Based Pathological Gait Analysis.

Accurate lower-limb pose estimation is aprerequisite of skeleton based pathological gait analysis. To achieve this goal in free-living environments for long-term monitoring, single depth sensor has been proposed in research. However, the depth map acquired from a single viewpoint encodes only partial geometric information of the lower limbs and exhibits large variations across different viewpoints. Existing off-the-shelf 3D pose tracking algorithms and public datasets for depth based human pose estimation are mainly targeted at activity recognition applications. They are relatively insensitive to skeleton estimation accuracy, especially at the foot segments. Furthermore, acquiring ground truth skeleton data for detailed biomechanics analysis also requires considerable efforts. To address these issues, we propose a novel cross-domain self-supervised complete geometric representation learning framework, with knowledge transfer from the unlabelled synthetic point clouds of full lower-limb surfaces. The proposed method can significantly reduce the number of ground truth skeletons (with only 1%) in the training phase, meanwhile ensuring accurate and precise pose estimation and capturing discriminative features across different pathological gait patterns compared to other methods.

Study of Damage Quantification of Concrete Drainage Pipes Based on Point Cloud Segmentation and Reconstruction

The urban drainage system is an important part of the urban water cycle. However, with the aging of drainage pipelines and other external reasons, damages such as cracks, corrosion, and deformation of underground pipelines can cause serious consequences such as urban waterlogging and road collapse. At present, the detection of underground drainage pipelines mostly focuses on the qualitative identification of pipeline damage, and it is impossible to quantitatively analyze pipeline damage. Therefore, a method to quantify the damage volume of concrete pipes that combines surface segmentation and reconstruction is proposed. An RGB-D sensor is used to collect the damage information of the drainage pipeline, and the collected depth frame is registered to generate the pipeline’s surface point cloud. Voxel sampling and Gaussian filtering are used to improve data processing efficiency and reduce noise, respectively, and the RANSAC algorithm is used to remove the pipeline’s surface information. The ball-pivoting algorithm is used to reconstruct the surface of the segmented damage data and pipe’s surface information, and finally to obtain the damage volume. In order to evaluate, we conducted our research on real-world materials. The measurement results show that the method proposed in this paper measures an average relative error of 7.17% for the external damage volume of concrete pipes and an average relative error of 5.22% for the internal damage measurements of concrete pipes.

Facial landmark-guided surface matching for image-to-patient registration with an RGB-D camera.

Fiducial marker-based image-to-patient registration is the most common way in image-guided neurosurgery, which is labour-intensive, time consuming, invasive and error prone. We proposed a method of facial landmark-guided surface matching for image-to-patient registration using an RGB-D camera. Five facial landmarks are localised from preoperative magnetic resonance (MR) images using deep learning and RGB image using Adaboost with multi-scale block local binary patterns, respectively. The registration of two facial surface point clouds derived from MR images and RGB-D data is initialised by aligning these five landmarks and further refined by weighted iterative closest point algorithm. Phantom experiment results show the target registration error is less than 3mm when the distance from the camera to the phantom is less than 1000mm. The registration takes less than 10s. The proposed method is comparable to the state-of-the-arts in terms of the accuracy yet more time-saving and non-invasive.

Research on global actual measurement of indoor surface flatness and verticality Based on sparse point cloud

At present, it has become a trend to realize indoor actual measurement by reconstructing indoor 3D information with dense point cloud. Considering the large time consumption of traditional manual measurement and data overload of 3D reconstruction for a construction, the paper presents a global measurement method based on sparse point cloud for indoor actual measurement of flatness and verticality. First, given the point cloud is present with low density, our data size is much smaller than that in 3D reconstruction, which greatly reduces time consumption for data processing. Second, we calculate surface flatness degree by using a tolerance formula in machinery industry as reference after we conduct fitting of surfaces by least square method. As for the measurement of verticality, making use of the more orderly and effective point cloud clustering plus surface fitting based on sparse point cloud, we are able to calculate the verticality degree easily by a mathematical method. Compared with existing methods, this approach might feature a more tailored or specialized measurement method for actual measurement of flatness and verticality in acceptance of construction work in indoor scenes. To validate this approach, we present our experimental results and examine the systematic error by the mathematical assessment modelling. It is proved that the systematic error of the measurement method based on sparse point cloud is nearly ignorable.

PUFA-GAN: A Frequency-Aware Generative Adversarial Network for 3D Point Cloud Upsampling.

We propose a generative adversarial network for point cloud upsampling, which can not only make the upsampled points evenly distributed on the underlying surface but also efficiently generate clean high frequency regions. The generator of our network includes a dynamic graph hierarchical residual aggregation unit and a hierarchical residual aggregation unit for point feature extraction and upsampling, respectively. The former extracts multiscale point-wise descriptive features, while the latter captures rich feature details with hierarchical residuals. To generate neat edges, our discriminator uses a graph filter to extract and retain high frequency points. The generated high resolution point cloud and corresponding high frequency points help the discriminator learn the global and high frequency properties of the point cloud. We also propose an identity distribution loss function to make sure that the upsampled points remain on the underlying surface of the input low resolution point cloud. To assess the regularity of the upsampled points in high frequency regions, we introduce two evaluation metrics. Objective and subjective results demonstrate that the visual quality of the upsampled point clouds generated by our method is better than that of the state-of-the-art methods.

An Earth Mover's Distance Based Multivariate Generalized Likelihood Ratio Control Chart for Effective Monitoring of 3d Point Cloud Surface

An Earth Mover's Distance Based Multivariate Generalized Likelihood Ratio Control Chart for Effective Monitoring of 3d Point Cloud Surface

Mapping Unmanned Aerial System Data onto Building Information Modeling Parameters for Highway Construction Progress Monitoring

The transportation infrastructure management sector lacks automated procedures that can help it find and resolve the performance deviations. The objective of this research is to illustrate the mapping of Unmanned Aerial System (UAS) collected photogrammetric data to building information modeling (BIM) parameters, and their application for automated construction progress monitoring and the generation of as-built models. The goal is to support project managers to estimate project progress during highway construction. As a part of ongoing work, this paper takes into account 4D (3D + time) data that is acquired from 3D surface digital elevation models, point clouds, LiDAR data, and orthographic photos. It maps these 4D data onto BIM parameters to create as-built models of the project at different intervals. A comparison between as-planned and as-built models using the earned value management method is employed to develop metrics that can be used for indicating cost and schedule deviations during construction. The mapping methodology introduced in this paper is illustrated using an ongoing highway construction project case study. The main contribution of this paper is the organization, processing, and integration of UAS data with BIM data structures and project management workflows. The research outcomes will assist project managers in an easy and quick identification of potential performance problems and support the project management decision-making process.

Rice Height Monitoring between Different Estimation Models Using UAV Photogrammetry and Multispectral Technology

Unmanned aerial vehicle (UAV) photogrammetry was used to monitor crop height in a flooded paddy field. Three multi-rotor UAVs were utilized to conduct flight missions in order to capture RGB (RedGreenBlue) and multispectral images, and these images were analyzed using several different models to provide the best results. Two image sets taken by two UAVs, mounted with RGB cameras of the same resolution and Global Navigation Satellite System (GNSS) receivers of different accuracies, were applied to perform photogrammetry. Two methods were then proposed for creating crop height models (CHMs), one of which was denoted as the M1 method and was based on the Digital Surface Point Cloud (DSPC) and the Digital Terrain Point Cloud (DSPT). The other was denoted as the M2 method and was based on the DSPC and a bathymetric sensor. An image set taken by another UAV mounted with a multispectral camera was used for multispectral-based photogrammetry. A Normal Differential Vegetation Index (NDVI) and a Vegetation Fraction (VF) were then extracted. A new method based on multiple linear regression (MLR) combining the NDVI, the VF, and a Soil Plant Analysis Development (SPAD) value for estimating the measured height (MH) of rice was then proposed and denoted as the M3 method. The results show that the M1 method, the UAV with a GNSS receiver with a higher accuracy, obtained more reliable estimations, while the M2 method, the UAV with a GNSS receiver of moderate accuracy, was actually slightly better. The effect on the performance of CHMs created by the M1 and M2 methods is more negligible in different plots with different treatments; however, remarkably, the more uniform the distribution of vegetation over the water surface, the better the performance. The M3 method, which was created using only a SPAD value and a canopy NDVI value, showed the highest coefficient of determination (R2) for overall MH estimation, 0.838, compared with other combinations.