Three-dimensional Point Cloud Research Articles

LiDAR-Based Dense Pedestrian Detection and Tracking

LiDAR-based pedestrian detection and tracking (PDT) with high-resolution sensing capability plays an important role in real-world applications such as security monitoring, human behavior analysis, and intelligent transportation. The problem of LiDAR-based PDT suffers from the complex gathering movements and the phenomenon of self- and inter-object occlusions. In this paper, the detection and tracking of dense pedestrians using three-dimensional (3D) real-measured LiDAR point clouds in surveillance applications is studied. To deal with the problem of undersegmentation of dense pedestrian point clouds, the kernel density estimation (KDE) is used for pedestrians center estimation which further leads to a pedestrian segmentation method. Three novel features are defined and used for further PDT performance improvements, which takes advantage of the pedestrians’ posture and body proportion. Finally, a new track management strategy for dense pedestrians is presented to deal with the tracking instability caused by dense pedestrians occlusion. The performance of the proposed method is validated with experiments on the KITTI dataset. The experiment shows that the proposed method can significantly increase F1 score from 0.5122 to 0.7829 compared with the STM-KDE. In addition, compared with AB3DMOT and EagerMOT, the tracking trajectories from the proposed method have the longest average survival time of 36.17 frames.

Laser point cloud registration method based on iterative closest point improved by Gaussian mixture model considering corner features

ABSTRACT As a fundamental process of three-dimensional lidar point cloud data (3D LPCD) processing, numerous registration methods are time consuming and easily fall into local optimum. A 3D LPCD registration method based on the iterative closest point (ICP) algorithm, which is improved by the Gaussian mixture model (GMM) considering corner features, is proposed in this article to address these limitations. The GMM method is used for coarse registration, and the input original 3D LPCD is replaced by corner features extracted by the improved 3D Harris algorithm to improve the efficiency of coarse registration. In addition, a satisfactory initial position between the reference and the moving 3D LPCD is prepared for ICP fine registration by coarse registration; thus, the accuracy of fine registration can be improved. The registration accuracy and efficiency of the new method is proved to be higher than those of four common ICP-based registration methods (3DSC-RANSACICP, 3DSC-SAC-IAICP, FPFH-RANSACICP, and FPFH-SAC-IAICP), and GMM registration methods, and the local optimum problem is effectively addressed.

Estimating Individual Tree Above-Ground Biomass of Chinese Fir Plantation: Exploring the Combination of Multi-Dimensional Features from UAV Oblique Photos

Chinese fir (Cunninghamia lanceolata (Lamb.) Hook) is one of the important tree species in plantation in southern China. Rapid and accurate acquisition of individual tree above-ground biomass (IT-AGB) information is of vital importance for precise monitoring and scientific management of Chinese fir forest resources. Unmanned Aerial Vehicle (UAV) oblique photogrammetry technology can simultaneously obtain high-density point cloud data and high spatial resolution spectral information, which has been a main remote sensing source for obtaining forest fine three-dimensional structure information and provided possibility for estimating IT-AGB. In this study, we proposed a novel approach to estimate IT-AGB by introducing the color space intensity information into a regression-based model that incorporates three-dimensional point cloud and two-dimensional spectrum feature variables, and the accuracy was evaluated using a leave-one-out cross-validation approach. The results demonstrated that the intensity variables derived from the color space were strongly correlated with the IT-AGB and obviously improved the estimation accuracy. The model constructed by the combination of point cloud variables, vegetation index and RGB spatial intensity variables had high accuracy (R2 = 0.79; RMSECV = 44.77 kg; and rRMSECV = 0.25). Comparing the performance of estimating IT-AGB models with different spatial resolution images (0.05, 0.1, 0.2, 0.5 and 1 m), the model was the best at the spatial resolution of 0.2 m, which was significantly better than that of the other four. Moreover, we also divided the individual tree canopy into four directions (East, West, South and North) to develop estimation models respectively. The result showed that the IT-AGB estimation capacity varied significantly in different directions, and the West-model had better performance, with the estimation accuracy of 67%. This study indicates the potential of using oblique photogrammetry technology to estimate AGB at an individual tree scale, which can support carbon stock estimation as well as precision forestry application.

Method of Classification the Mobile Robot Workspace Based on the Analysis of a 3D Point Cloud

One of the main and most difficult tasks in the development of automotive systems is the classification of the workspace of a mobile robot. Based on the classification results, a local map of the area is built, with the help of which, then, the robot trajectory is planned. The article describes a method of classification the working area of an autonomous mobile robot moving in rough terrain. The developed classification method is based on the analysis of a three-dimensional point cloud obtained by a laser scan- ning 3D rangefinder. Using a scanning laser rangefinder allows you to classify the robot  s motion zone at any time of the day or year. A set of classification features is proposed, the calculation of which is carried out using the least square method and elements of probability theory and mathematical statistics. The classification of the robot workspace is carried out by four classes: "Flat surface", "Small roughness", "Large roughness" and "Obstacle". Each class characterizes the degree of passability of the robot movement surface. Classification results are saved as a local map of passability. In each cell of such a map a number is written that characterizes the passability of the region of the working area bounded by this cell. The developed classifier is integrated into the on-board control system of the wheeled mobile robot. The results of experimental studies confirming the efficiency and effectiveness of the proposed classification method are presented. The accuracy of class recognition of the mobile robot workspace has been determined. The developed classifier successfully operates in various conditions, including in winter and at dusk, but at the same time it has limitations when working in conditions of natural noise, such as rain, snow. The average classification accuracy with the minimal influence of natural noise is 92.3 %, and the execution time of each iteration does not exceed 0.085 s, which allows using developed classifier as a part of on-board control systems of autonomous mobile robots.

An improved computation method for asphalt pavement texture depth based on multiocular vision 3D reconstruction technology

The depth of an asphalt pavement structure is an important index used to reflect the anti-skid performance of the pavement, which directly affects the driving safety of vehicles. In the current specifications (e.g., China, European and American standards), the mean texture depth (MTD) of an asphalt pavement is generally measured by the sand-patch method (SPM). However, SPM is easily affected by the operator’s subjective experience and experimental environment, resulting in low accuracy and high scatter in the data. In view of such shortcomings, a fast and accurate method to obtain the texture depth of asphalt pavement was proposed by designing a new test tool and a computer-aided calculation method. The multiocular vision theory was adopted to reconstruct a three-dimensional (3D) point cloud model of pavement texture. To reflect the concavity and convexity of pavement texture and eliminate the influence of pavement slope, the iterative closest point (ICP) algorithm was employed to obtain the datum reference plane (DRP). On this basis, the pentahedral volume calculation method suitable for evaluating the texture depth of an asphalt pavement was proposed. Laboratory experiment results using SPM were inverted by equal volume calculation to obtain the texture reference plane (TRP) and MTD'. Meanwhile, a 3D average maximum elevation (Avg.EL) algorithm was obtained by the evolution of two-dimensional mean profile depth (MPD). Furthermore, three parameters, namely MTD, MTD', and Avg.EL, were used to measure the texture depth of two field pavement sections in service. The experimental results showed that the correlation between MTD' and MTD was better than that between Avg.EL and MTD, and the degree of scatter was smaller. MTD' could be used directly as a reference value for MTD. In the present work, the 3D volume calculation method of asphalt pavement texture depth was proposed, which integrates the technical flow of image acquisition, model reconstruction, and algorithm analysis. The developed method provides a technical reference for the application of computer vision technology in the analysis of pavement texture depth.

Direct-focusing surface-emitting laser.

Focusing is a fundamental optical technique that has been widely implemented via lenses. Here, we demonstrate direct focusing from a band-edge surface-emitting laser, whose emission area is 200 µm × 200 µm, without any lenses. To achieve this, a phase-modulating layer is incorporated into the laser cavity. This layer acts simultaneously as a lasing cavity similar to that of a photonic crystal laser and as a holographic spatial-phase modulator, which transforms the output beam into a focusing beam by slightly shifting the positions of holes from a periodic square lattice. Beam profiles along the surface normal clearly show that direct focusing occurs with a focal length and focal spot size of 310 µm and 6.1 µm, respectively. The focal length agrees well with the theoretical value, and the focal spot size is 2.0 times the diffraction-limited size, which indicates that the higher transverse modes are sufficiently suppressed. In addition, the power density at the focus is 540 times higher than that at the near-field plane. Interestingly, a focus pattern is also observed in the opposite direction at the near-field plane, which indicates that a converging beam and a diverging beam are simultaneously emitted because of the nature of the in-plane band-edge laser. The conventional beam patterns of semiconductor laser cavities are limited to the regime of two-dimensional projection based on a Fourier hologram. In contrast, we demonstrate the simplest form of a three-dimensional point cloud based on a Fresnel hologram, which is quite useful for micro-sensing applications such as microfluidics, flow cytometry, blood sensors, and endoscopy.

Visual enhancement of single-view 3D point cloud reconstruction

3D reconstruction from a single image is one of the core computer vision problems. Thanks to the development of deep learning, 3D reconstruction of a single image has demonstrated impressive progress in recent years. Existing researches use Chamfer distance as a loss function to guard the training of the neural network. However, the Chamfer loss will give equal weights to all points inside the 3D point clouds. It tends to sacrifice fine-grained and thin structures to avoid incurring a high loss, which will lead to visually unsatisfactory results. This paper proposes a framework that can recover a detailed three-dimensional point cloud from a single image by focusing more on boundaries (edge and corner points). Experimental results demonstrate that the proposed method outperforms existing techniques significantly, both qualitatively and quantitatively, and has fewer training parameters.

IoT-based low-cost 3D mapping using 2D Lidar for different materials

Three-dimensional (3D) mapping is the technology of profiling objects in three dimensions to map them in real life. 3D mapping is an excellent tool for analysis, surveying, and other engineering domains. 3D light detection and ranging (Lidar) technology usually does 3D mapping, but the cost of 3D Lidar is very high, which constrains its applicability in the Industry. The need for a low-cost 3D mapping has increased widely. A cost-effective 3D mapping based on the Internet of Things (IoT) using 2D Lidar is proposed. 2D Lidar is lower in cost than 3D Lidar, so it completely replaces the 3D Lidar for applications that need moderate or low accuracy. This research sought to confirm the applicability of this low-cost IoT-based advanced sensing technology, i.e., Lidar, in the engineering domain. A two-dimensional Lidar mapping system prototype is constructed, and the same is used to generate three-dimensional (3D) point cloud models. Lidar measurements are greatly dependent on the materials. Every material has different absorbance and reflectivity, which alter the Lidar measurements. In this research, we have shown how different material affects Lidar accuracy.

Real-time multiscale prediction of structural performance in material extrusion additive manufacturing

The material extrusion additive manufacturing (AM) has been extensively used in fabricating structures with complex geometries. However, geometric defects often exist in an AM structure, which could compromise its final performance. In this paper, a real-time multiscale performance evaluation method is developed for material extrusion-based honeycomb structures. The representative cell boundary is extracted from three-dimensional (3D) point clouds obtained via an in-situ monitoring approach. The cell boundary is then used to generate the digital twin of the unit cell of the printed layer based on the finite element (FE) method. A physics-based multiscale modeling approach called mechanics of structure genome (MSG) is then employed to predict the effective material properties of the printed layer and plate stiffness matrix of the final structure. The proposed approach provides a highly efficient way to predict the real-time performance of the as-manufactured products. Moreover, the numerical example shows that the geometric defects could result in complex mechanical behaviors in the defected parts, which cannot be captured by the conventional approaches based on the shape deviations. The numerical results are validated by the three-point bending tests. The proposed method can be used in the closed-loop control of material extrusion-based manufacturing systems.

Safe Walking Route Recommender Based on Fall Risk Calculation Using a Digital Human Model on a 3D Map

The recent increase in sedentary lifestyles has highlighted the importance of physical activity in our daily lives. Although fall accidents are a common cause of injuries among the elderly, walking is one of the simplest physical activities available. Therefore, walking route selection and identification of walking behavior in different scenarios remains a widely researched area. This paper is about developing a walking-route recommender system that proposes a safe route to the user and reduces the probability of fall accidents. We determine route candidates after the start and endpoints are fed to the system. Then the tripping risk involved in each route candidate is identified by performing several walking simulations of a digital human model on a three-dimensional laser-scanned point cloud. Finally, a safe route is determined after considering the user’s age and the tripping risk. It was confirmed that tripping risk could be competently estimated according to the considered age group.

3D Reconstruction System Based on Multi Sensor

Abstract In the 3d dense map construction system of indoor scene by mobile robot, the existing single sensor method cannot improve the positioning accuracy and reconstruction accuracy of robot, as well as the requirement of rapidity. Therefore, it is applied to THE ORB-SLAM with three parallel threads of track tracking, map reconstruction and loopback detection. Through depth camera pose to splice point cloud of building three-dimensional dense point cloud, in the 3 d reconstruction, a computer can not rely on GPU parallel computing, using only the CPU recovery environment three-dimensional dense scene map method, further reduce the time of the map construction, improve the efficiency of the reconstruction, thus improve the overall performance. Since only the ORB features were retained in the map during the construction of ORB-SLAM2, sparse point cloud map was established. Fortunately, the framework structure of ORB-SlAM2 was relatively clear. Only one thread needed to be added for the maintenance of point cloud map, and the key frames generated by ORB-SLAM2 were passed into the point cloud map construction thread. Use incoming keyframes to generate a map with dense point clouds.

A Novel 3-D Local DAISY-Style Descriptor to Reduce the Effect of Point Displacement Error in Point Cloud Registration

Three-dimensional (3-D) point clouds are widely considered for applications in different fields. Various methods have been proposed to generate point cloud data: LIDAR and image matching from static and mobile platforms, including, e.g., terrestrial laser scanning. With multiple point clouds from stationary platforms, point cloud registration is a crucial and fundamental issue. A standard approach is a point-based registration, which relies on pairs of corresponding points in two-point clouds. Therefore, a necessary step in point-based registration is the construction of 3-D local descriptors. One of the (many) challenges that will specifically affect the performance of local descriptors with local spatial information is the point displacement error. This error is caused by the difference in the distributions of points surrounding a (potentially) corresponding center point in the two-point clouds. It can occur for various reasons such as 1) distortions caused by the sensors recording the data, 2) moving objects, 3) varying density of point cloud, 4) change of viewing angle, and 5) different of the sensors. The purpose of this article is to develop a new 3-D local descriptor reducing the effect of this type of error in point cloud coarse registration. The approach includes an improved local reference frame and a new geometric arrangement in point cloud space for the 3-D local descriptor. Inspired by the 2-D DAISY descriptor, a geometric arrangement is created to reduce the effect of the point displacement error. in addition, directional histograms are considered as features. Investigations are performed for point clouds from challenging environments, which are publicly available. The results of this study show the high performance of the proposed approach for point cloud registration, especially in more challenging and noisy environments.

A Three-dimensional Point Cloud Denoising Method Based on Adaptive Threshold

A Three-dimensional Point Cloud Denoising Method Based on Adaptive Threshold

Research on 3D finite element simulation model of aluminium beam with lamination defects based on 2D Fourier transform

In order to solve the problems of low detection accuracy and poor modelling effect when building the laminated defect aluminium beam model, a 3D finite element simulation model of laminated defect aluminium beam based on 2D Fourier transform is proposed. The two-dimensional Fourier transform method was used to obtain the full wave field signal and spatial position wave number curve of aluminium beam structure. The three-dimensional point cloud image was generated by MATLAB software, and the solid model of aluminium beam with delamination defect was established. The three-dimensional finite element simulation model of aluminium beam with laminated defects was obtained by introducing it into the worktable. The experimental results show that the accuracy of the method is 98.91%, the average texture number and frame rate are 17.105 ms and 95.507 frames/second, respectively. It has a good effect on the establishment of 3D finite element simulation model of aluminium beam.

Experimental study on the effectiveness of using 3D scanning and 3D engraving technology to accurately assess shale fracture conductivity

Accurately evaluating fracture conductivity is key to optimizing the shale fracturing process. However, the influence of fracture morphology on conductivity remains unknown due to the non-replicability of rough samples in most laboratory-based studies. Thus, accurately duplicating rock samples with identical surface morphology and mechanical properties in the laboratory is a major concern for future fracture conductivity studies. Herein, a new method of reproducing rock samples for conductivity testing was proposed. Essentially, a laser scanner was used to obtain split rough fracture surface three-dimensional point cloud data for reverse reconstruction, which were subsequently combined with 3D engraving technology. This method successfully produced a significant number of conductivity test samples with uniform surface morphology and similar mechanical properties that can be used in experimental research on shale fracture conductivity. Comparison of the split shale sample with that created using the proposed method showed an area tortuosity error of only ±0.35%, a comparable elevation distribution frequency, and a high degree of similarity between the surfaces. Furthermore, due to the influence of the fracture surface roughness, rock samples with a proppant concentration below a certain “critical proppant concentration” may exhibit higher conductivity than samples with higher proppant concentrations under low closure pressure. Moreover, the “critical proppant concentration” was positively correlated with large-scale waviness, but did not show obvious correlation with small-scale unevenness. • A new method was proposed to reproduce shale samples with uniform surface morphology by integrating 3D scanning and 3D engraving techniques. • Under low closure stress, higher conductivity may be produced at a proppant concentration below a certain critical value, namely “critical proppant concentration". • The “critical proppant concentration” was positively correlated with large-scale waviness, which determines the proppant distribution and thus affects conductivity.

Self-Supervised Depth Estimation Leveraging Global Perception and Geometric Smoothness

Self-supervised depth estimation has drawn much attention in recent years as it does not require labeled data but image sequences. Moreover, it can be conveniently used in various applications, such as autonomous driving, robotics, realistic navigation, and smart cities. However, extracting global contextual information from images and predicting a geometrically natural depth map remain challenging. In this paper, we present DLNet for pixel-wise depth estimation, which simultaneously extracts global and local features with the aid of our depth Linformer block. This block consists of the Linformer and innovative soft split multi-layer perceptron blocks. Moreover, a three-dimensional geometry smoothness loss is proposed to predict a geometrically natural depth map by imposing the second-order smoothness constraint on the predicted three-dimensional point clouds, thereby realizing improved performance as a byproduct. Finally, we explore the multi-scale prediction strategy and propose the maximum margin dual-scale prediction strategy for further performance improvement. In experiments on the KITTI and Make3D benchmarks, the proposed DLNet achieves performance competitive to those of the state-of-the-art methods, reducing time and space complexities by more than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$62\%$</tex-math> </inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$56\%$</tex-math> </inline-formula> at a resolution of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$416 \times 128$</tex-math> </inline-formula> , respectively. Extensive testing on various real-world situations further demonstrates the strong practicality and generalization capability of the proposed model.

Fast and Lite Point Cloud Semantic Segmentation for Autonomous Driving Utilizing LiDAR Synthetic Training Data

In the autonomous car, perception with point cloud semantic segmentation helps obtain a wealth of information about the surrounding road environment. Despite the massive progress of recent researches, the existing machine learning networks are still insufficient for online applications of autonomous driving due to too subdivided classes, the lack of training data, and their heavy computing load. To solve these problems, we propose a fast and lite point cloud semantic segmentation network for autonomous driving, which utilizes LiDAR synthetic data to improve the performance by transfer learning. First, we modify the labeling classes and generate the LiDAR synthetic data-set for additional training to alleviate the lack of training data of the realistic data-set. Then, to lower the computing load, we adopt a projection-based method and apply a lightweight segmentation network to projected LiDAR images, which has drastically reduced computing. Finally, we verified and evaluated the proposed network in this paper through experiments. Experimental results show that the proposed network can perform the three-dimensional point cloud semantic segmentation in an online way, in which the inference speed overwhelms the existing algorithms.

Привязка данных мобильного лазерного сканирования к результатам аэрофотосъемки на основе определения взаимного положения массивов точек

One of the most rapid surveying methods of terrestrial survey is mobile laser scanning (MLS). This method allows collecting data for objects of the environment with high detail and accuracy in a short period of time. A condition for obtaining accurate MLS data is receiving a satellite signal of high quality. When scanning built-up urban areas, the quality of signal deteriorates, which leads to a decrease in accuracy. The use of additional control data, one of which may be aerial photography results, helps to compensate for the loss of accuracy. For this purpose, MLS data are registered using aerial photog-raphy data with various automatic methods, which are carried out in two- or three-dimensional space. Therefore, the three-dimensional MLS data are previously transformed into the form of a two-dimensional image or a group of aerial images into the form of a three-dimensional point cloud. Regis-tration in 3D space is preferred with low quality of aerial images. The technique of automatic MLS data registration to point clouds generated from aerial images using the ICP method is discussed. It is proposed to predivide the MLS data into fragments. It is shown that the size of fragments for calculating local corrections in horizontal position can differ from the size of fragments for calculating correc-tions in vertical one.

Estimating Tree Structural Parameters via Automatic Tree Segmentation From LiDAR Point Cloud Data

In this article, we proposed an automated tree segmentation method using light detection and ranging (LiDAR) point cloud data. Tree segmentation was performed accurately even with bumpy ground, and was validated on more than 1000 samples. For example, 371 out of 374 trees were detected from dataset 2, and the error was caused by the trees with low point densities located in the area far from the LiDAR. Segmentation was accurately performed, including the branches, leading to the retrieval of high-level parameters such as the leaf areas. To obtain the parameters regarding the leaf area from the segmented trees, a method for classifying the leaf and branch points in the three-dimensional point clouds obtained using a terrestrial LiDAR method was proposed. After preprocessing the input point cloud, such as by voxelization, the fast point feature histogram (FPFH) features were calculated. Then, the classifier for classification into leaves and branches was trained using the training dataset to calculate the test accuracy with the test data. Moreover, an unsupervised method for classification using the FPFH feature and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -means algorithm was also performed. Consequently, the recall and precision values of the classification were determined as 98.14% and 96.03%, respectively, with the supervised approach.

Extraction and Simplification of three-dimensional Point Cloud Topological Features Using Piecewise Linear Morse Theory

Extraction and Simplification of three-dimensional Point Cloud Topological Features Using Piecewise Linear Morse Theory