Point Cloud Method Research Articles

Displacement field reconstruction in landslide physical modeling by using a terrain laser scanner – Part 1: Methodology, error analysis and validation

Laser scanning technology has been widely used in landslide aspects. However, the existing deformation analysis based on terrain laser scanners can only provide limited information, which is insufficient for understanding landslide kinematics and failure mechanisms. To overcome this limitation, this paper proposes an automated method for processing point clouds collected in landslide physical modeling. This method allows the acquisition of quantitative three-dimensional (3D) deformation field information. The results show the organized and spatially related point cloud segmentation in terms of spherical targets. The segmented point clouds can be fitted to determine the locations of all preset targets and their corresponding location changes. The proposed method has been validated based on theoretical analysis and numerical and physical tests, which indicates that it can batch-process massive data sets with high computational efficiency and good noise resistance. Compared to existing methods, this method shows a significant potential for understanding landslide kinematics and failure mechanisms and advancing the application of 3D laser scanning in geotechnical modeling.

Extracting cow point clouds from multi-view RGB images with an improved YOLACT++ instance segmentation

Obtaining shape features from a farm environment is the basis for animal size measurement, body condition scoring, weight prediction, and individual identification. To improve the robustness of point cloud segmentation and fully utilize 2D (Two dimension) and 3D (Three dimension) information for segmenting animal shape data, a novel pixel-level segmentation method of cow point clouds was proposed by fusing real images with high resolution and point clouds. First, the 3D point clouds of cows were generated from the multi-view images by using the Structure from Motion (SfM) algorithm, and the corresponding relationship between the point clouds and the real image was established by the camera interior and exterior parameters. Second, for 2D image segmentation, the improved, single-stage instance segmentation algorithm YOLACT++ (You Only Look At CoefficienTs) was applied to improve the segmentation accuracy of cow body regions, and the average precision of multi-view images was 85.9%. Last, in point cloud segmentation, the image segmentation results of the cows were mapped to 3D point clouds by the correspondence between point clouds and images, and the visual points of different viewpoints were fused to obtain the 3D point cloud of the target cow. The average relative error of truncated areas between the proposed method and manual segmentation of the point cloud was 2.18%, and the change in the truncated area caused by the increase in images was less than 5% with the appropriate number of images (20–40). The results demonstrated the effectiveness of the proposed method in 3D point cloud segmentation of cows and its potential application in animal shape acquisition.

Towards Intricate Stand Structure: A Novel Individual Tree Segmentation Method for ALS Point Cloud Based on Extreme Offset Deep Learning

Due to the complex structure of high-canopy-density forests, the traditional individual tree segmentation (ITS) algorithms based on ALS point cloud, which set segmentation threshold manually, is difficult to adequately cover a variety of complex situations, so the ITS accuracy is unsatisfactory. In this paper, a top-down segmentation strategy is adopted to propose an adaptive segmentation method based on extreme offset deep learning, and the ITS set aggregation strategy based on gradient change criterion is designed for the over-segmentation generated by random offset, and the precise ITS is realized. Firstly, the segmentation sub-plot is set as 25 m × 25 m, the regional point cloud and its treetop are marked, and the offset network is trained. Secondly, the extreme offset network is designed to carry out spatial transformation of the original point cloud, and each point is offset to the position near the treetop to obtain the offset point cloud with a high density at the treetop, which enhances the discrimination among individual trees. Thirdly, the self-adaptive mean shift algorithm based on average neighboring distance is designed to divide and mark the offset point cloud. Fourthly, the offset point cloud, after clustering, is mapped back to the original space to complete the preliminary segmentation. Finally, according to the gradient change among different canopies, the ITS aggregation method is designed to aggregate adjacent canopies with a gentle gradient change. In order to investigate the universality of the proposed method on different stand structures, two coniferous forest plots (U1, U2) in the Blue Ridge area of Washington, USA, and two mixed forest plots (G1, G2) in Bretten, Germany, are selected in the experiment. The learning rate of the deep network is set as 0.001, the sampled point number of the sub-plot is 900, the transformer dimension is 512 × 512, the neighboring search number of points is 16, and the number of up-sampling blocks is 3. Experimental results show that in mixed forests (G1, G2) with complex structures, the F-score of the proposed method reaches 0.89, which is about 4% and 7% higher than the classical SHDR and improved DK, respectively. In high-canopy-density areas (U2, G2), the F-score of the proposed method reaches 0.89, which is about 3% and 4% higher than the SHDR and improved DK, respectively. The results show that the proposed method has high universality and accuracy, even in a complex stand environment with high canopy density.

A single plant segmentation method of maize point cloud based on Euclidean clustering and K-means clustering

It is very important to study the correlation between phenotypic information and genetic information of population maize plants for breeding good maize varieties. How to separate single maize plants from population plants to more accurately measure maize phenotypic parameters is still a difficult problem. Therefore, we propose a single plant segmentation method based on Euclidean clustering and K-means clustering. First, three dimensional (3D) point cloud data of two planting density populations of maize (field 1 and field 2) at five-leaf stage (V5) and six-leaf stage (V6) were obtained by terrestrial laser scanning (TLS). Second, the point cloud data of maize plants were preprocessed to extract the point cloud data of the experimental area. Then, plane segmentation, point cloud filtering and Euclidean clustering were used to segment the population maize plants. Finally, the number of plants and center point of point cloud of maize plants after preliminary segmentation were obtained by voxel filtering, and then K-means clustering was used to achieve single segmentation of maize plants. When planting density was sparse, the segmentation results based on Euclidean clustering and K-means clustering showed that the Accuracy* F1 Score (A * F1) of V5 stage and V6 stage plants were 100.00 % and 99.87 %, respectively; and the A * F1 of V5 stage and V6 stage were 99.59 % and 85.69 %, respectively, when planting density was dense. Compared with the results of Euclidean clustering single plant segmentation, maize single plant segmentation A * F1 increased by 0.00 %, 6.02 %, 12.25 % and 73.41 % at V5 stage and V6 stage in field 1 and field 2, respectively. The results showed that the single plant segmentation based on Euclidean clustering and K-means clustering method could solve the problem that Euclidean clustering single plant segmentation method could not segment the cross-leaf plants. This study provides a simple and reliable method for plant segmentation of population maize for breeders and crop phenotypists.

High Accuracy and Low Complexity Lidar Place Recognition Using Unitary Invariant Frobenius Norm

Simultaneous Localization and Mapping (SLAM) is used in solving the problems of localization, navigation, and map construction for autonomous vehicles moving in unknown environments. Place recognition is an inevitable subject in SLAM, and the current lidar-based methods have been popularized for their rising environmental robustness. Currently, the place recognition method with lidar has attracted much attention due to its high environmental robustness. However, many 3D laser point cloud methods for place recognition descriptors construction with the local or global point-cloud data omitting some intrinsic properties of the point clouds. In this article, we propose a place recognition method based on the unitary invariance of the Frobenius norm for utilizing different attributes of ground points and non-ground points. Specifically, the interpretable filtering framework and a dynamic threshold adjustment strategy are raised according to different environments and sensors with intensity and geometric information. As commissioning, the comparative experiments are conducted with CHDloop datasets and public KITTI datasets of different scales and access types. Compared to the original Scan Context (SC) and Intensity Scan Context (ISC) methods, our proposed method achieves higher efficiency while maintaining a improved recall rate and precision. This novel method has been integrated into the existing LiDAR SLAM and formed a new framework FSC_ALOAM that reduces drifts in point-cloud mapping. The entire process improves the accordance of maps at the identical position in auto-driving.

A Simple Point-based Iso-Scallop Tool Path Planning Method for Noisy Point Clouds with High Robustness and Controlled Errors

With the rapid advances in 3D scanning and vision technologies, point clouds have seen a significant increase in their application to CNC machining. As an important way to generate tool paths from point clouds, point-based direct planning methods are, however, now restricted to high-quality point clouds. Most existing methods would have robustness issues if a noisy point cloud is given due to their reliance on the noise-sensitive estimation of differential properties from point clouds. Error control under noisy point clouds is also rarely considered. This paper presents a robust and error-controlled point-based iso-scallop tool path planning method for CNC machining with ball-nosed cutters. It casts the problem of searching for scallop/CL points to another problem of finding circle-sphere intersection points and the maximum/minimum among them. Both circle-sphere intersections and max/min comparisons are easy to implement and robust. It has also been theoretically proved that the intersections and comparisons can ensure an error bound of 2ϵ on the final results if two conditions called ϵ-sampling and ϵ-covering are met. The effectiveness of the method has been demonstrated by a series of examples and comparisons.

MPED: Quantifying Point Cloud Distortion Based on Multiscale Potential Energy Discrepancy.

In this article, we propose a new distortion quantification method for point clouds, the multiscale potential energy discrepancy (MPED). Currently, there is a lack of effective distortion quantification for a variety of point cloud perception tasks. Specifically, in human vision tasks, a distortion quantification method is used to predict human subjective scores and optimize the selection of human perception task parameters, such as dense point cloud compression and enhancement. In machine vision tasks, a distortion quantification method usually serves as loss function to guide the training of deep neural networks for unsupervised learning tasks (e.g., sparse point cloud reconstruction, completion, and upsampling). Therefore, an effective distortion quantification should be differentiable, distortion discriminable, and have low computational complexity. However, current distortion quantification cannot satisfy all three conditions. To fill this gap, we propose a new point cloud feature description method, the point potential energy (PPE), inspired by classical physics. We regard the point clouds are systems that have potential energy and the distortion can change the total potential energy. By evaluating various neighborhood sizes, the proposed MPED achieves global-local tradeoffs, capturing distortion in a multiscale fashion. We further theoretically show that classical Chamfer distance is a special case of our MPED. Extensive experiments show that the proposed MPED is superior to current methods on both human and machine perception tasks. Our code is available at https://github.com/Qi-Yangsjtu/MPED.

A novel solution to planar feature-constrained, dual quaternion-based registration method for point clouds

To solve point cloud registration, a novel solution to planar feature-based registration of point clouds was proposed in this study according to dual quaternion description based on the constraint of planar feature. The normal vector of the homologous feature plane should be kept parallel according to the registration, and the points on the plane should satisfy the planar equation. Moreover, the modified Levenberg-Marquardt method was adopted to complete the registration model, so as to avoid inappropriate initial values from causing non-convergence of the iteration. Lastly, the robustness and accuracy exhibited by the method were verified using simulated and measured data.

P‐2.32: Six‐degree‐of‐freedom Platform Motion Anomaly Prediction Technology Based on Binocular Vision

Binocular stereo vision is based on the principle of parallax and uses imaging equipment to obtain two images of the measured object from different positions, and obtains the three-dimensional geometric information of the object by calculating the position deviation between the corresponding points of the image. As an important execution component of the six-degree-of-freedom platform in the flight simulator, the electric cylinder can undertake the important task of dynamic simulation, and its abnormal working state directly affects the training effect and life safety of the flight simulator users. In this paper, the electric cylinder position pose is obtained by the three-dimensional point cloud method, and the improved active learning anomaly detection algorithm is proposed to predict the electric cylinder position posture, so as to determine whether the platform electric cylinder fails.

Using color-only vegetation indexes to remove vegetation from otherwise mostly mono-material point clouds

Point clouds are now a standard way of describing objects in many engineering disciplines, whether they are man-made objects such as structures, buildings, or various types of structures. Commonly used methods of acquiring such data include ground, UAV, or even aerial photogrammetry, followed by terrestrial, UAV, and aerial scanning. After measurement (by the scanner) or calculation (from photogrammetry), the point cloud goes through extensive processing that essentially transforms the unordered mass of points into a usable data set. One of the important steps is removing points representing obstructing objects and features, including vegetation in particular. Here, many filtering methods based on different principles are available and suitable for application to different scenes. This paper presents a new method of filtering point clouds based on the visible spectrum color principle using vegetation indexes determined from RGB system colors only. Since each sensor has to some extent, an individual interpretation of the colors, it cannot be assumed to determine specific boundaries of what is and is no longer vegetation. Therefore, it was proposed to use means clustering to simplify the operator's work. The method was also designed in such a way that the entire evaluation could be implemented in the freely available CloudCompare software. The procedure was tested on three different sites with different terrain and vegetation characteristics showing, which demonstrated the applicability of this method to data where the color information (green) uniquely identifies vegetation. The selected vegetation filters ExG, ExR, ExB, and ExGr were tested, where ExG was the best. K-means clustering helps an operator to distinguish more easily between vegetation and the rest of the point cloud without compromising the quality of the result. The method is practically implementable using the freely downloadable and usable CloudCompare software.

Semantic Segmentation for Digital Archives of Borobudur Reliefs Based on Soft-Edge Enhanced Deep Learning

Segmentation and visualization of three-dimensional digital cultural heritage are important analytical tools for the intuitive understanding of content. In this paper, we propose a semantic segmentation and visualization framework that automatically classifies carved items (people, buildings, plants, etc.) in cultural heritage reliefs. We also apply our method to the bas-reliefs of Borobudur Temple, a UNESCO World Heritage Site in Indonesia. The difficulty in relief segmentation lies in the fact that the boundaries of each carved item are formed by indistinct soft edges, i.e., edges with low curvature. This unfavorable relief feature leads the conventional methods to fail to extract soft edges, whether they are three-dimensional methods classifying a three-dimensional scanned point cloud or two-dimensional methods classifying pixels in a drawn image. To solve this problem, we propose a deep-learning-based soft edge enhanced network to extract the semantic labels of each carved item from multichannel images that are projected from the three-dimensional point clouds of the reliefs. The soft edges in the reliefs can be clearly extracted using our novel opacity-based edge highlighting method. By mapping the extracted semantic labels into three-dimensional points of the relief data, the proposed method provides comprehensive three-dimensional semantic segmentation results of the Borobudur reliefs.

Maize Ear Height and Ear–Plant Height Ratio Estimation with LiDAR Data and Vertical Leaf Area Profile

Ear height (EH) and ear–plant height ratio (ER) are important agronomic traits in maize that directly affect nutrient utilization efficiency and lodging resistance and ultimately relate to maize yield. However, challenges in executing large-scale EH and ER measurements severely limit maize breeding programs. In this paper, we propose a novel, simple method for field monitoring of EH and ER based on the relationship between ear position and vertical leaf area profile. The vertical leaf area profile was estimated from Terrestrial Laser Scanner (TLS) and Drone Laser Scanner (DLS) data by applying the voxel-based point cloud method. The method was validated using two years of data collected from 128 field plots. The main factors affecting the accuracy were investigated, including the LiDAR platform, voxel size, and point cloud density. The EH using TLS data yielded R2 = 0.59 and RMSE = 16.90 cm for 2019, R2 = 0.39 and RMSE = 18.40 cm for 2021. In contrast, the EH using DLS data had an R2 = 0.54 and RMSE = 18.00 cm for 2019, R2 = 0.46 and RMSE = 26.50 cm for 2021 when the planting density was 67,500 plants/ha and below. The ER estimated using 2019 TLS data has R2 = 0.45 and RMSE = 0.06. In summary, this paper proposed a simple method for measuring maize EH and ER in the field, the results will also offer insights into the structure-related traits of maize cultivars, further aiding selection in molecular breeding.

Hybrid simplification algorithm for unorganized point cloud based on two-level fuzzy decision making

Traditional point cloud simplification algorithms have specific application scenarios. If these well-known methods can be accurately applied to different feature regions of a point cloud, high-quality point cloud feature preservation can be realized. In this paper, a hybrid point cloud simplification method, based on two-level fuzzy decision making, is proposed. Specifically, the number of peak points and bin intervals of each dimension density histogram of the point cloud are counted, and the cluster number and initial centers of fuzzy c-means (FCM) clustering method are determined, hence the autonomous FCM clustering method of point clouds is realized. Based on the expert linguistic inference rules of type-1 fuzzy system, its output is used to accurately identify clusters as flat, transitional and drastic type. According to the attributes of the clusters, predefined thresholds or algorithms are applied in each one, achieving high quality point cloud simplification. The experimental results show that the algorithm can effectively determine the number of clusters, improve the speed of FCM clustering algorithm and achieve feature preserving point cloud simplification, after removing 50–90 % of the points.

Analysis of Maintenance Techniques for a Three-Dimensional Digital Twin-Based Railway Facility with Tunnels

In accordance with the paradigm of the 4th and Industrial Revolution, the introduction of building information modeling is expected in all areas related to railroad construction, operation and management, along with the establishment of a metaverse platform that combines big data, the Internet of Things, and artificial intelligence. The performance of tasks related to the safety and maintenance of railway facilities is aided by the use of digital systems free from physical and temporal constraints. Three-dimensional (3D) modeling and other 4th industrial technologies, such as unmanned aerial vehicles (UAVs) and light detection and ranging (LiDAR), are increasingly implemented in many types of infrastructure. With respect to railroads, the use of these methods to monitor tunnel spaces has been hindered by the limitations of modeling with UAV and inadequate Global Positioning System reception. Here, we conducted the domestic application of 4th industrial technologies to a railway tunnel; we addressed these problems using a BLK360, a fixed LiDAR device that combines two-dimensional panoramic images and a 3D point cloud method. The outcomes of this research will benefit railway operation managers by providing a platform combining a two-dimensional panoramic virtual reality (VR) image and a 3D model developed from a 3D scan framework for the maintenance of existing railway facilities (tunnels). Our approach was optimized for the maintenance and operational management of railroad facilities, as demonstrated for tunnels, because it continuously acquires time-series data that is appropriate for the maintenance of the corresponding space. In the future, this approach can be used for test tracks and operational lines.

A Classification Method of Point Clouds of Transmission Line Corridor Based on Improved Random Forest and Multi-Scale Features.

Classification of airborne laser scanning (ALS) point clouds of power lines is of great importance to their reconstruction. However, it is still a difficult task to efficiently and accurately classify the ground, vegetation, power lines and power pylons from ALS point clouds. Therefore, in this paper, a method is proposed to improve the accuracy and efficiency of the classification of point clouds of transmission lines, which is based on improved Random Forest and multi-scale features. The point clouds are filtered by the optimized progressive TIN densification filtering algorithm, then the elevations of the filtered point cloud are normalized. The features of the point cloud at different scales are calculated according to the basic features of the point cloud and the characteristics of transmission lines. The Relief F and Sequential Backward Selection algorithm are used to select the best subset of features to estimate the parameters of the learning model, then an Improved Random Forest classification model is built to classify the point clouds. The proposed method is verified by using three different samples from the study area and the results show that, compared with the methods based on Support Vector Machines, AdaBoost or Random Forest, our method can reduce feature redundancy and has higher classification accuracy and efficiency.

Keyframe image processing of semantic 3D point clouds based on deep learning.

With the rapid development of web technologies and the popularity of smartphones, users are uploading and sharing a large number of images every day. Therefore, it is a very important issue nowadays to enable users to discover exactly the information they need in the vast amount of data and to make it possible to integrate their large amount of image material efficiently. However, traditional content-based image retrieval techniques are based on images, and there is a "semantic gap" between this and people's understanding of images. To address this "semantic gap," a keyframe image processing method for 3D point clouds is proposed, and based on this, a U-Net-based binary data stream semantic segmentation network is established for keyframe image processing of 3D point clouds in combination with deep learning techniques.

Fast clustering method of LiDAR point clouds from coarse-to-fine

LiDAR has become an indispensable sensor for autonomous vehicles due to its unique properties. The clustering of non-ground point clouds, as an essential step of the perceptual processing pipeline, is crucial to the safe driving of autonomous vehicles. However, some existing methods are prone to over-clustering, insufficient clustering, or poor real-time performance. Therefore, we propose a coarse-to-fine clustering strategy to balance clustering accuracy and speed. We first propose a clustering method based on angle and distance judgment to rough process the point cloud and then use the clustering method based on breakpoint detection to refine the point cloud. We evaluate the proposed method on two public data sets to prove the effectiveness of the proposed method. Through quantitative comparison, we find that the average processing time of each frame in the two data sets is about 14 ms, which can meet the real-time requirements of automatic driving.

A Fast Ground Segmentation Method of LiDAR Point Cloud From Coarse-to-Fine

Light detection and ranging (LiDAR) has become an indispensable sensor for autonomous vehicles due to its unique properties. Segmentation of raw point cloud is a fundamental step in the perception processing pipeline, which is critical to the safe driving of autonomous vehicles. However, some existing methods are prone to over-segmentation, insufficient segmentation, or poor real-time performance in the process of point cloud segmentation. Therefore, we propose a coarse-to-fine segmentation strategy to balance segmentation accuracy and speed. The ground segmentation includes applying robust locally weighted regression (RLWR) to the nonuniformly divided bins of polar grids in concentric model to achieve coarse ground segmentation. The processing results are mapped to a range image, and the improved range image convolution method is used to refine the ground segmentation. We evaluate the proposed method on two public datasets, and the precision, recall, and harmonic mean of the ground segmentation are all above 93%, which is sufficient to demonstrate the robustness of our method. Using the segmentation strategy from coarse to fine, the average processing time per frame of the SemanticKITTI dataset and the Nagoya dataset are 39.49 and 42.63 ms, respectively, with an average of 41.06 ms, which can meet the real-time requirements of autonomous driving.

Automatic Registration of Homogeneous and Cross-Source TomoSAR Point Clouds in Urban Areas.

Building reconstruction using high-resolution satellite-based synthetic SAR tomography (TomoSAR) is of great importance in urban planning and city modeling applications. However, since the imaging mode of SAR is side-by-side, the TomoSAR point cloud of a single orbit cannot achieve a complete observation of buildings. It is difficult for existing methods to extract the same features, as well as to use the overlap rate to achieve the alignment of the homologous TomoSAR point cloud and the cross-source TomoSAR point cloud. Therefore, this paper proposes a robust alignment method for TomoSAR point clouds in urban areas. First, noise points and outlier points are filtered by statistical filtering, and density of projection point (DoPP)-based projection is used to extract TomoSAR building point clouds and obtain the facade points for subsequent calculations based on density clustering. Subsequently, coarse alignment of source and target point clouds was performed using principal component analysis (PCA). Lastly, the rotation and translation coefficients were calculated using the angle of the normal vector of the opposite facade of the building and the distance of the outer end of the facade projection. The experimental results verify the feasibility and robustness of the proposed method. For the homologous TomoSAR point cloud, the experimental results show that the average rotation error of the proposed method was less than 0.1°, and the average translation error was less than 0.25 m. The alignment accuracy of the cross-source TomoSAR point cloud was evaluated for the defined angle and distance, whose values were less than 0.2° and 0.25 m.

Using Fractional Super-Resolution to Improve Lossy Compression of Point Cloud Geometry

We present a method for post-processing point clouds’ geometric information by applying a previously proposed fractional super-resolution technique to clouds compressed and decoded with MPEG’s G-PCC codec. In some sense, this is a continuation of that previous work, which requires only a downscaled point cloud and a scaling factor, both of which are provided by the G-PCC codec. For non-solid point clouds, an a priori down-scaling is required for improved efficiency. The method is compared to the GPCC itself, as well as machine-learning-based techniques. Results show a great improvement in quality over GPCC and comparable performance to the latter techniques, with the advantage of not needing any kind of previous training.