Three-dimensional Point Research Articles

Biaxial Piezoelectric MEMS Mirrors with Low Absorption Coating for 1550 nm Long-Range LIDAR.

This paper presents the fabrication and characterization of a biaxial MEMS (MicroElectroMechanical System) scanner based on PZT (Lead Zirconate Titanate) which incorporates a low-absorption dielectric multilayer coating, i.e., a Bragg reflector. These 2 mm square MEMS mirrors, developed on 8-inch silicon wafers using VLSI (Very Large Scale Integration) technology are intended for long-range (>100 m) LIDAR (LIght Detection And Ranging) applications using a 2 W (average power) pulsed laser at 1550 nm. For this laser power, the use of a standard metal reflector leads to damaging overheating. To solve this problem, we have developed and optimised a physical sputtering (PVD) Bragg reflector deposition process compatible with our sol-gel piezoelectric motor. Experimental absorption measurements, performed at 1550 nm and show up to 24 times lower incident power absorption than the best metallic reflective coating (Au). Furthermore, we validated that the characteristics of the PZT, as well as the performance of the Bragg mirrors in terms of optical scanning angles, were identical to those of the Au reflector. These results open up the possibility of increasing the laser power beyond 2W for LIDAR applications or other applications requiring high optical power. Finally, a packaged 2D scanner was integrated into a LIDAR system and three-dimensional point cloud images were obtained, demonstrating the scanning stability and operability of these 2D MEMS mirrors.

Multi-focusing surface-emitting lasers.

Complete control of a beam pattern requires not only projecting a two-dimensional (2D) pattern but also focusing on a three-dimensional (3D) point cloud, which is typically achieved utilizing holography under the framework of diffraction. We previously reported direct focusing from on-chip size surface-emitting lasers that utilize a holographically modulated photonic crystal cavity based on 3D holography. However, this demonstration was of the simplest 3D hologram with a single point and single focal length, and the more typical 3D hologram with multiple points and multiple focal lengths has not yet been examined. Toward direct generation of a 3D hologram from the on-chip size surface-emitting laser, we here examined a simple 3D hologram featuring two different focal lengths with a single off-axis point in each to reveal the fundamental physics. Two types of holography, one based on superimposition and the other on random tiling, successfully demonstrated the desired focusing profiles. However, both types caused a spot noise beam in the far field plane due to interference between focusing beams with different focal lengths, especially in the case of the superimposing method. We also found that the 3D hologram based on the superimposing method consisted of higher order beams including the original hologram due to the manner of the holography. Secondly, we demonstrated a typical 3D hologram with multiple points and focal lengths and successfully showed the desired focusing profiles by both methods. We believe our findings will bring innovation to mobile optical systems and pave the way to developing compact optical systems in areas such as material processing, micro fluidics, optical tweezers, and endoscopy.

Automatic Branch-Leaf Segmentation and Leaf Phenotypic Parameter Estimation of Pear Trees Based on Three-Dimensional Point Clouds.

The leaf phenotypic traits of plants have a significant impact on the efficiency of canopy photosynthesis. However, traditional methods such as destructive sampling will hinder the continuous monitoring of plant growth, while manual measurements in the field are both time-consuming and laborious. Nondestructive and accurate measurements of leaf phenotypic parameters can be achieved through the use of 3D canopy models and object segmentation techniques. This paper proposed an automatic branch-leaf segmentation pipeline based on lidar point cloud and conducted the automatic measurement of leaf inclination angle, length, width, and area, using pear canopy as an example. Firstly, a three-dimensional model using a lidar point cloud was established using SCENE software. Next, 305 pear tree branches were manually divided into branch points and leaf points, and 45 branch samples were selected as test data. Leaf points were further marked as 572 leaf instances on these test data. The PointNet++ model was used, with 260 point clouds as training input to carry out semantic segmentation of branches and leaves. Using the leaf point clouds in the test dataset as input, a single leaf instance was extracted by means of a mean shift clustering algorithm. Finally, based on the single leaf point cloud, the leaf inclination angle was calculated by plane fitting, while the leaf length, width, and area were calculated by midrib fitting and triangulation. The semantic segmentation model was tested on 45 branches, with a mean Precisionsem, mean Recallsem, mean F1-score, and mean Intersection over Union (IoU) of branches and leaves of 0.93, 0.94, 0.93, and 0.88, respectively. For single leaf extraction, the Precisionins, Recallins, and mean coverage (mCoV) were 0.89, 0.92, and 0.87, respectively. Using the proposed method, the estimated leaf inclination, length, width, and area of pear leaves showed a high correlation with manual measurements, with correlation coefficients of 0.94 (root mean squared error: 4.44°), 0.94 (root mean squared error: 0.43 cm), 0.91 (root mean squared error: 0.39 cm), and 0.93 (root mean squared error: 5.21 cm2), respectively. These results demonstrate that the method can automatically and accurately measure the phenotypic parameters of pear leaves. This has great significance for monitoring pear tree growth, simulating canopy photosynthesis, and optimizing orchard management.

A critical review of automated extraction of rock mass parameters using 3D point cloud data

Abstract This paper presents a comprehensive review aimed at gaining insights into the current state of research on quantification technology used for acquiring three-dimensional (3D) point cloud data of rock masses and extracting vital structural information. Addressing this significant challenge and problem within the field of rock engineering is the main objective of this study. The timely and accurate acquisition of rock mass data and fine characterization of rock mass parameters can avoid unnecessary personnel injury and property damage. First, the methods of point cloud information acquisition and structural information extraction are systematically summarized and classified. Then, various existing methods are analyzed for their advantages and disadvantages. Based on this analysis, the future development direction of relevant technologies is proposed to improve the level of acquisition of 3D information of rock mass and the level of extraction of key information of rock mass. The results indicate that rock mass point cloud information acquisition technology can be classified into two types: laser point cloud acquisition and image reconstruction based on structure from motion (SfM) algorithm. Rock mass structural information can be classified into rock mass structural planes and their attitudes, rock mass traces and their geometric parameters, and other rock mass parameters, including structural plane roughness, spacing, and block characteristics, etc. Different acquisition technologies and feature extraction methods have their own advantages, disadvantages, and applicable ranges. Therefore, a comprehensive selection of various evaluation methods should be made based on specific engineering characteristics and existing data situations in practice.

GL-Net: Semantic segmentation for point clouds of shield tunnel via global feature learning and local feature discriminative aggregation

has gradually become the first choice of modern urban public transportation due to its advantages of safety and high-efficiency. Shield tunnel is an important type of subway tunnel, and its structural stability and safety play an important role in subway operation. The shield tunnels are prone to problems such as water leakage and tunnel collapse, which affect the safe operation of subways. Efficient monitoring methods are required to detect the status of subway tunnels. The data collection and accurate segmentation of key components of shield tunnels are the basis and key to the automatic monitoring of subway tunnels. This research presents a novel semantic segmentation method of three-dimensional (3-D) point clouds of typical structural elements (e.g., longitudinal joint, circumferential joints, bolt hole and grouting hole) in shield tunnel based on deep learning. In this method, we focus on how to make the network learn robust global features and complex local distribution patterns. Further, we propose a global and local feature encoding block (namely GL-block) to discriminatively aggregate local features while learning global representation. After multiple encodings by the GL-block, we design a global correlation modeling (GCM) module to establish a global awareness of each point. Finally, a weighted cross-entropy loss function is designed to solve the problem of unbalanced number of samples in each category of shield tunnel. In the experiments, we make a dataset of shield tunnel point clouds with a length of about 1,000 m collected by CNU-TS-1 (DU et al., 2018) mobile tunnel monitoring system, and use the dataset to train and test the segmentation ability of our method on the typical structural elements of shield tunnels. Experiments verify the effectiveness of our method by comparing with the other state-of-the-art 3-D point cloud semantic segmentation methods, and our method has an mIoU score of 73.02 %, which is at least 14.54 % higher than the other compared state-of-the-art networks. Also, we further verify the adaptability of our method to different tunnels and different laser scanning equipment, such as FARO, Leica and Z + F, and achieve very advanced performance.

Development of Autonomous Moving Robot Using Appropriate Technology for Tsukuba Challenge

We have been participating in the Tsukuba Challenge, an open experiment involving autonomous robots, since 2014. The technology of our robot has stabilized, and our robot has continued to win the Tsukuba Mayor Prize from 2018 to 2021 without changing the basic configuration of the body and navigation software. Here, we report the robot’s structure as the project’s current completed form. Our robot is designed with the policy of selecting the most rational technology (appropriate technology) to achieve the purpose, even if it is not the latest. For example, we used image-like two-dimensional data instead of a three-dimensional point cloud in map matching for robot positioning. For pedestrian signal recognition, which was required to perform an optional task, we did not use deep learning but rather conventional color image processing. These techniques are advantageous for balancing the execution time and accuracy required in the challenge.

Learning Representative Features by Deep Attention Network for 3D Point Cloud Registration.

Three-dimensional point cloud registration, which aims to find the transformation that best aligns two point clouds, is a widely studied problem in computer vision with a wide spectrum of applications, such as underground mining. Many learning-based approaches have been developed and have demonstrated their effectiveness for point cloud registration. Particularly, attention-based models have achieved outstanding performance due to the extra contextual information captured by attention mechanisms. To avoid the high computation cost brought by attention mechanisms, an encoder-decoder framework is often employed to hierarchically extract the features where the attention module is only applied in the middle. This leads to the compromised effectiveness of the attention module. To tackle this issue, we propose a novel model with the attention layers embedded in both the encoder and decoder stages. In our model, the self-attentional layers are applied in the encoder to consider the relationship between points inside each point cloud, while the decoder utilizes cross-attentional layers to enrich features with contextual information. Extensive experiments conducted on public datasets prove that our model is able to achieve quality results on a registration task.

Development and evaluations of an approach with full utilization of point cloud for measuring the angle of repose

The angle of repose (AOR) is an indicator of the fluidity of granular materials. The uncertainty for determining the material AOR includes the deviation in the global shape of repeatedly generated heaps and measurement uncertainty for each heap. A suitable method of measuring the AOR helps to decrease the latter. For this aim, a measurement method with full utilization of a three-dimensional (3D) point cloud was developed. About the approach, there was a sector-shaped domain for batch detection in a cylindrical coordinate system. The progressively rotating domain could calculate the slope AORs. Then the average of total slope AORs was determined as the heap AOR. A series of simulated heaps with varying particle shapes, heap sizes, particle sizes, and known AORs were measured by the approach for evaluating its performance. The heap 3D data from a publicly available dataset was used, which contained fresh corn grains, lignite, wood chips, and limestone with 4 replicate heaps for each material. The two other published 3D methods for calculating the AORs were used as comparison items for the developed method. Factors of measurement uncertainty depended on these three methods. And the first heap data of each material was configured with a gradient of 6 for this factor. The coefficient of variation (CV) of the measurements by each method for 6 gradients of a heap was used to assess its measurement uncertainty. The replicate heaps of each material were measured using these methods. The evaluation of the measurement uncertainty showed that the CV of the measurements by the developed method for 6 gradients of a heap was significantly lower than it was by the two other methods. This article gives and evaluates a method for measuring the AOR, and it can decrease the measurement uncertainty.

A novel solution for extracting individual tree crown parameters in high-density plantation considering inter-tree growth competition using terrestrial close-range scanning and photogrammetry technology

Automatic tree crown mapping, crown width (CW) and crown projection area (CPA) extraction are the basis for high-precision estimation of forest productivity and carbon stock. Terrestrial close-range observation (TCRO) technology, mainly including terrestrial laser scanning (TLS) and close-range photogrammetry (CRP), measures individual tree structural parameters by generating detailed three-dimensional (3D) points, making it a potential replacement for labor-intensive manual forest inventories. However, accurate extraction of crown parameters using TCRO in high-density forests remain a challenge due to different degrees of missing crown information caused by occlusion between trees and limitations of computer 3D modeling techniques and segmentation methods. Here, we propose a model for estimating crown parameters using tree spacing, diameter at breast height (DBH) and stand age, which collaboratively considers the advantages of TCRO data and the bioecological mechanisms of inter-tree competition. First, an approach to construct a tree triangulation network with tree position and tree spacing is designed based on the analysis of the bioecological mechanisms of inter-tree competition relationships. Second, a model to quantify inter-tree competition using tree spacing, DBH and stand age is developed to extract CW and CPA. We demonstrate the generality of our model to high-density forests in subtropical China with TLS data. It shows a good performance in Chinese fir, Eucalyptus and Spingbract Chinkapin-dominated plots with estimation accuracy (EA) ≥ 90.22 % and relative root mean square error (rRMSE) ≤ 0.1286 for CW and EA ≥ 84.51 % and rRMSE ≤ 0.1861 for CPA. In addition, the performance in estimating CW (EA = 89.80 %) and CPA (EA = 86.13 %) using CRP point clouds demonstrate the applicability of our model to CRP. The proposed model collaboratively considers inter-tree competition mechanism and TCRO data characteristics, therefore it is interpretable in terms of bioecological principle and universal for different forest environments (e.g., coniferous, broad-leaved and mixed coniferous species). The results show that the method provides an efficient solution for automatic and accurate measurements of the crown parameters in the forest sample plot investigation, which can support the fine forest management and carbon stock estimation.

Automatic Aluminum Alloy Surface Grinding Trajectory Planning of Industrial Robot Based on Weld Seam Recognition and Positioning

In this paper, we propose a novel method for planning grinding trajectories on curved surfaces to improve the grinding efficiency of large aluminum alloy surfaces with welds and defect areas. Our method consists of three parts. Firstly, we introduce a deficiency positioning method based on a two-dimensional image and three-dimensional point cloud, which enables us to accurately and quickly locate the three-dimensional defective areas. Secondly, we propose a 2D weld positioning method based on the defect area and obtain the spatial position of the 3D weld by combining the relationship between 2D and 3D images. Additionally, we propose an orthogonal projection method from the point cloud to the aluminum alloy surface to calculate the weld reinforcement. Thirdly, we present a space spiral grinding trajectory planning method for complex curved surfaces based on the characteristics of the weld reinforcement, spatial position, and spatial position information of the defect area. This method shortens the grinding time of the defect area and improves efficiency. Simulation and experimental results show that our grinding trajectory planning method is more efficient than other grinding methods in removing defects from the surface of aluminum alloys. Moreover, the defect area after grinding is smoother than before.

Three-Dimensional Point Cloud Data Pre-Processing for the Multi-Source Information Fusion in Aircraft Assembly

Wing-body assembly is a key part of aircraft manufacturing, and during the process of wing assembly, the 3D point cloud data of the components are an important basis for attitude adjustment. The large amount of measured point cloud data and the obvious noise affect the quality and efficiency of the final assembly. To address this problem, research on the pre-processing method of the component point cloud data is carried out. Firstly, a feature-enhanced point cloud resampling method is proposed to preserve key features such as part contours in the resampling process. Then, a multi-scale point cloud data noise filtering method is proposed, which can effectively filter out the outliers. The experimental results show that the proposed method improves the speed and accuracy of the subsequent point cloud analysis effectively and is successfully applied to the assembly process of a large passenger aircraft, laying the foundation for high-quality assembly.

Multimodal deep learning for point cloud panoptic segmentation of railway environments

The demand for transportation asset digitalisation has significantly increased over the years. For this purpose, mobile mapping systems (MMSs) are among the most popular technologies that allow capturing high precision three-dimensional point clouds of the infrastructure. In this paper, a multimodal deep learning methodology is presented for panoptic segmentation of the railway infrastructure. The methodology takes advantage of image rasterisation of the point clouds to perform a rough segmentation and discard more than 80% of points that are not relevant to the infrastructure. With this approach, the computational requirements for processing the remaining point cloud are highly reduced, allowing the process of dense point clouds in short periods of time. A 90 km-long railway scenario was used for training and testing. The proposed methodology is two times faster than the current state-of-the-art for the same point cloud density, and pole-like object segmentation metrics are improved.

Correction to: robust two-phase registration method for three-dimensional point set under the bayesian mixture framework

Correction to: robust two-phase registration method for three-dimensional point set under the bayesian mixture framework

A Localization and Mapping Algorithm Based on Improved LVI-SAM for Vehicles in Field Environments.

Quickly grasping the surrounding environment's information and the location of the vehicle is the key to achieving automatic driving. However, accurate and robust localization and mapping are still challenging for field vehicles and robots due to the characteristics of emptiness, terrain changeability, and Global Navigation Satellite System (GNSS)-denied in complex field environments. In this study, an LVI-SAM-based lidar, inertial, and visual fusion using simultaneous localization and mapping (SLAM) algorithm was proposed to solve the problem of localization and mapping for vehicles in such open, bumpy, and Global Positioning System (GPS)-denied field environments. In this method, a joint lidar front end of pose estimation and correction was designed using the Super4PCS, Iterative Closest Point (ICP), and Normal Distributions Transform (NDT) algorithms and their variants. The algorithm can balance localization accuracy and real-time performance by carrying out lower-frequency pose correction based on higher-frequency pose estimation. Experimental results from the complex field environment show that, compared with LVI-SAM, the proposed method can reduce the translational error of localization by about 4.7% and create a three-dimensional point cloud map of the environment in real time, realizing the high-precision and high-robustness localization and mapping of the vehicle in complex field environments.

P‐2.32: Six‐degree‐of‐freedom Platform Motion Anomaly Prediction Technology Based on Binocular Vision

Binocular stereo vision is based on the principle of parallax and uses imaging equipment to obtain two images of the measured object from different positions, and obtains the three-dimensional geometric information of the object by calculating the position deviation between the corresponding points of the image. As an important execution component of the six-degree-of-freedom platform in the flight simulator, the electric cylinder can undertake the important task of dynamic simulation, and its abnormal working state directly affects the training effect and life safety of the flight simulator users. In this paper, the electric cylinder position pose is obtained by the three-dimensional point cloud method, and the improved active learning anomaly detection algorithm is proposed to predict the electric cylinder position posture, so as to determine whether the platform electric cylinder fails.

Method for real-time reconstruction of a transmission line based on the LiDAR point cloud data of a partial line segment

In a light detection and ranging (LiDAR) anti-external force damage monitoring system, the LiDAR installed on an electric power tower can only collect some point cloud data from the transmission line within a few tens of meters. Consequently, its anti-break system fails to calculate the distance between the transmission line and ground objects. In this study, a real-time reconstruction method for a transmission line based on a catenary mathematical model is proposed. Based on the three-dimensional (3D) point cloud data of the local power line, a 3D point cloud is mapped to a two-dimensional (2D) plane. In the 3D plane coordinate system, the catenary equation is fitted and calculated using the catenary model, and the 2D point cloud dataset of the transmission line is calculated according to this equation. Subsequently, the transmission line of the 2D plane is restored to the 3D space, and finally, the entire transmission line is restored to the 3D point cloud data. The field test results reveal that the proposed method can be used to effectively calculate and reconstruct the 3D transmission line model in real time and solve the spatial distance calculation problem of the anti-external force damage monitoring system.

"Canopy fingerprints" for characterizing three-dimensional point cloud data of soybean canopies.

Advances in imaging hardware allow high throughput capture of the detailed three-dimensional (3D) structure of plant canopies. The point cloud data is typically post-processed to extract coarse-scale geometric features (like volume, surface area, height, etc.) for downstream analysis. We extend feature extraction from 3D point cloud data to various additional features, which we denote as 'canopy fingerprints'. This is motivated by the successful application of the fingerprint concept for molecular fingerprints in chemistry applications and acoustic fingerprints in sound engineering applications. We developed an end-to-end pipeline to generate canopy fingerprints of a three-dimensional point cloud of soybean [Glycine max (L.) Merr.] canopies grown in hill plots captured by a terrestrial laser scanner (TLS). The pipeline includes noise removal, registration, and plot extraction, followed by the canopy fingerprint generation. The canopy fingerprints are generated by splitting the data into multiple sub-canopy scale components and extracting sub-canopy scale geometric features. The generated canopy fingerprints are interpretable and can assist in identifying patterns in a database of canopies, querying similar canopies, or identifying canopies with a certain shape. The framework can be extended to other modalities (for instance, hyperspectral point clouds) and tuned to find the most informative fingerprint representation for downstream tasks. These canopy fingerprints can aid in the utilization of canopy traits at previously unutilized scales, and therefore have applications in plant breeding and resilient crop production.

Development of an Omnidirectional AGV by Applying ORB-SLAM for Navigation Under ROS Framework

The development of an automated guided vehicle with omni-wheels for autonomous navigation under a robot operating system framework is presented in this paper. Specifically, a laser rangefinder-constructed two-dimensional environment map is integrated with a three-dimensional point cloud map to achieve real-time robot positioning using the oriented features from accelerated segment test and rotated binary robust independent elementary feature detector-simultaneous localisation and mapping algorithm. In the path planning for autonomous navigation of the omnidirectional mobile robot, we applied the A* global path search algorithm, which uses a heuristic function to estimate the robot position difference and searches for the best direction. Moreover, we employed the time-elastic-band method for local path planning, which merges the time interval of two locations to realise time optimisation for dynamic obstacle avoidance. The experimental results verified the effectiveness of the applied algorithms for the omni-wheeled mobile robot. Further, the results showed a superior performance over the adaptive Monte Carlo localisation for robot localisation and dynamic window approach for local path planning.

Development and Verification of an Autonomous and Controllable Mobile Robot Platform

In this paper, we design a mobile robot platform, which employs a fully autonomous mechanical structure and electrical control system. Two driving wheels realize flexible steering movement with four universal wheels. A variety of sensors are built on the mobile robot platform, including the Inertial Measurement Unit (IMU) used to establish the inertial navigation coordinate system and the Velodyne’s Puck lidar sensor (VLP-16) used to obtain the three-dimensional (3D) point cloud information of the environment. Then, we build a software control architecture based on the Robot Operating System (ROS), using multi-node communication to perform positioning, environment perception, dynamic obstacle avoidance, path planning and motion control. Furthermore, a method of actively exploring the environment and constructing a map is proposed, using multi-path evaluation for real-time path planning and obstacle avoidance. In the end, we conduct autonomous exploration experiments to verify the performance of the designed mobile robot platform in indoor multi-obstacle scenes.

Chinese Digital Arm (CDA): A High-Precision Digital Arm for Electrical Stimulation Simulation.

To effectively analyze the diffusion and accumulation of signals on the surface and inside the human body under electrical stimulation, we used the gray threshold of the Chinese Digital Human image dataset to segment an arm image and reconstruct the tissue to obtain its three-dimensional cloud point dataset. Finally, a semirefined digital arm entity model with the geometric characteristics of the actual human arm tissue was constructed using reverse engineering technology. Further input of the current signal stimulation under tDCS and tACS with additional analysis of the signal diffusion in the transient mode via model calculation revealed that DC electrical stimulation is likely to cause high-voltage burns. The effective depth achieved using the AC stimulation signal is considerable, and provides reference for the electrical stimulation selection. Simultaneously, in the digital arm model, the signal diffusion and tissue damage inside the arm can be analyzed by changing the field, which provides a theoretical basis for the experimental study of the human body.