3D Point Cloud Image Research Articles

Physical characteristics identification of the concrete interfacial transition zone via 3D image scanning

The interfacial transition zone (ITZ) is the weakest link in concrete materials. The physical characteristics of the ITZ, such as its microhardness and thickness, significantly influence the mechanical properties and durability of concrete. Typically, microhardness testing is employed to identify the physical features of the ITZ through line measurements with a certain degree of randomness. In order to accurately describe ITZ physical characteristics, this study proposes a novel physical characteristics identification method via three-dimensional (3D) image scanning. The fundamental principle of this method lies in the varying hardness of each concrete phase, resulting in different surface elevations after polishing. As the polishing abrasiveness changes from coarse (240 mesh) to fine (800 mesh), the ITZ can be clearly observed in the 3D point cloud images of the experimental concrete. The ITZ geometrical information (thickness and different surface heights) can be easily identified using 3D scanned images. It was also found that there was a considerable linear regression relationship between the height difference and the relative microhardness variation; thus, the ITZ microhardness could be indirectly recognized through surface measurements. The physical characteristics of ITZ can be statistically analysed, effectively mitigating the bias inherent in traditional test results and significantly improving identification efficiency and accuracy.

A subway tunnel image stitching method based on point cloud mapping relationships and high-resolution image

The paper presents an intelligent tunnel 3D laser image acquisition system. The system can quickly acquire the 3D point cloud and tunnel surface image of the whole tunnel section. The paper proposes the subway tunnel image stitching method based on point cloud mapping relationships and high-resolution image. Firstly, the method extracts the homonymous points from the mapped image generated from the 3D point cloud and the camera image based on the maximum information coefficient method. Secondly, the 3D point cloud positions corresponding to the homonymous points are determined. The optimization of camera parameters is achieved based on the gradient descent method. Then, the tunnel point cloud is expanded to generate a gray scale image based on spherical projection. Based on the optimized parameters, the point cloud is projected to the camera image to determine the depth value of each camera image pixel point. Finally, the pixel values of the camera image are projected based on the spherical projection to obtain the stitching result images. This method stitches individual images together to form a more complete tunnel surface image. The larger tunnel surface image helps in identifying tunnel defects.

Enhancing angular resolution in single-photon LIDAR for daytime long-range 3D imaging

Single photon LIDAR exhibits exceptional detection sensitivity and finds extensive applications in long-distance measurement and three-dimensional imaging domains. However, it is susceptible to background noise interference, leading to a trade-off between scanning imaging efficiency and angular resolution, thereby limiting its overall performance. In this paper, we propose a novel ‘compression-resampling’ high-resolution imaging method for single-photon scanning LIDAR based on the spatial correlation of the target scene. This approach significantly enhances the system's angular resolution without compromising imaging efficiency. By employing this method, we successfully conducted 3D imaging over distances exceeding 26 km even under challenging conditions with high levels of background noise during daylight hours. Within a target field of view measuring 1° × 1°, we reconstructed a high-resolution 3D point cloud image comprising 1000 × 1410 pixels. Notably, this achievement yielded an impressive angular resolution of 17.4 urad while maintaining an acceptable imaging time of approximately 20 minutes. The results unequivocally demonstrate the effectiveness of our proposed method in improving the quality of three-dimensional imaging and enhancing angular resolution while simultaneously reducing imaging time requirements, thus expanding potential application scenarios for single-photon LIDAR technology.

End-effector path tracking of a 14 DOF Rover Manipulator System in CG-Space framework

AbstractIn order to resolve redundancy and path planning of a high DOF mobile manipulator using conventional approaches like Jacobian and a pseudoinverse method, researchers face the limitation of computational load and delay in response. If such kind of mobile manipulator is traversing the rough terrain, then conventional methods become too costly to implement due to the handling of redundant joints, obstacles, and wheel-terrain interaction. A few optimization-based redundancy resolution approaches try incorporating wheel-terrain interaction but fail in real-time response. This paper describes a 14 DOF Rover Manipulator System’s end-effector path-tracking approach using CG-Space framework to incorporate wheel-terrain interaction. CG-Space means the center of gravity (CG) locus of the Rover. The Rover’s CG is calculated while traversing over 3D terrain using a multivariable optimization method and a 3D point cloud image of the actual terrain and stored as CG-Space over the given terrain. First of all, we decide which part of the system moves to track the path, that is, arm or Rover, depending upon the manipulator’s work volume and manipulability measure restrictions. The next task is to obtain the Rover pose according to the end-effector path using a simple arm’s inverse kinematic solution between the CG-Space and end-effector task space without resolving redundancy. Meanwhile, obstacles and non-traversable regions are avoided in CG-Space. On diverse 3D terrains, simulation and experimental results support the suggested CG-Space framework for end-effector tracking.

High-precision surface profilometry on a micron-groove based on dual-comb electronically controlled optical sampling.

We demonstrate an optical method for 3D profilometry of micro-nano devices with large step structures. The measurement principle is based on a dual-comb direct time-of-flight detection. An electronically controlled optical sampling (ECOPS) approach is used to improve the acquisition rate. In a proof-of-principle distance measurement experiment, the measurement precision reaches 15nm at 4000-times averages. The method has been used to characterize the profile of a large aspect-ratio rectangular micron-groove with 10µm width and 62.3µm depth. By point-by-point scanning, a 3D point cloud image is obtained, and the 3D profile of the micro-structure is quantitatively reconstructed with sub-micrometer precision. The proposed high-precision, high-speed surface 3D profile measurement technology could be applied to profilometry and inspection of complex microelectronics devices in the future.

RETRACTED ARTICLE: Unmanned Vehicle Fusion Positioning Technology Based on “5G + Beidou” and 3D Point Cloud Image

Unmanned vehicles need to know their location and direction information accurately to plan and navigate their paths. However, the positioning system is susceptible to interference from a variety of factors, which leads to increased positioning errors, thereby affecting the accuracy of unmanned vehicle positioning. An unmanned vehicle fusion positioning technology based on the "5G + Beidou" integrated positioning system was proposed. While using the "5G + Beidou" base station for positioning, the 3D point cloud image was fused, and the high-precision real-time positioning was carried out through the vehicle's autonomous navigation algorithm. This paper first analyzed the current situation and characteristics of GNSS technology and studied the key technologies and principles of the "5G + Beidou" integrated positioning system. Then, aiming at the difficulty of 5G base station deployment, the GNSS system parameter optimization scheme based on a multidimensional fusion structure was designed. Finally, in the experiment, it was verified that the fusion system could achieve higher precision positioning results compared with traditional single-dimensional GNSS and multi-dimensional GNSS. The technical advantages of "5G + Beidou" were used for data fusion processing of unmanned vehicles, and a positioning method based on the combination of 3D point cloud image and high-precision map was proposed. Through some experiments, it was concluded that the fusion location method could control the error below 0.1, which showed the accuracy of the fusion location.

Detection and characterization of spike architecture based on deep learning and X-ray computed tomography in barley

BackgroundSpike is the grain-bearing organ in cereal crops, which is a key proxy indicator determining the grain yield and quality. Machine learning methods for image analysis of spike-related phenotypic traits not only hold the promise for high-throughput estimating grain production and quality, but also lay the foundation for better dissection of the genetic basis for spike development. Barley (Hordeum vulgare L.) is one of the most important crops globally, ranking as the fourth largest cereal crop in terms of cultivated area and total yield. However, image analysis of spike-related traits in barley, especially based on CT-scanning, remains elusive at present.ResultsIn this study, we developed a non-invasive, high-throughput approach to quantitatively measuring the multitude of spike architectural traits in barley through combining X-ray computed tomography (CT) and a deep learning model (UNet). Firstly, the spikes of 11 barley accessions, including 2 wild barley, 3 landraces and 6 cultivars were used for X-ray CT scanning to obtain the tomographic images. And then, an optimized 3D image processing method was used to point cloud data to generate the 3D point cloud images of spike, namely ‘virtual’ spike, which is then used to investigate internal structures and morphological traits of barley spikes. Furthermore, the virtual spike-related traits, such as spike length, grain number per spike, grain volume, grain surface area, grain length and grain width as well as grain thickness were efficiently and non-destructively quantified. The virtual values of these traits were highly consistent with the actual value using manual measurement, demonstrating the accuracy and reliability of the developed model. The reconstruction process took 15 min approximately, 10 min for CT scanning and 5 min for imaging and features extraction, respectively.ConclusionsThis study provides an efficient, non-invasive and useful tool for dissecting barley spike architecture, which will contribute to high-throughput phenotyping and breeding for high yield in barley and other crops.

Individual Pig Identification Using Back Surface Point Clouds in 3D Vision.

The individual identification of pigs is the basis for precision livestock farming (PLF), which can provide prerequisites for personalized feeding, disease monitoring, growth condition monitoring and behavior identification. Pig face recognition has the problem that pig face samples are difficult to collect and images are easily affected by the environment and body dirt. Due to this problem, we proposed a method for individual pig identification using three-dimension (3D) point clouds of the pig's back surface. Firstly, a point cloud segmentation model based on the PointNet++ algorithm is established to segment the pig's back point clouds from the complex background and use it as the input for individual recognition. Then, an individual pig recognition model based on the improved PointNet++LGG algorithm was constructed by increasing the adaptive global sampling radius, deepening the network structure and increasing the number of features to extract higher-dimensional features for accurate recognition of different individuals with similar body sizes. In total, 10,574 3D point cloud images of ten pigs were collected to construct the dataset. The experimental results showed that the accuracy of the individual pig identification model based on the PointNet++LGG algorithm reached 95.26%, which was 2.18%, 16.76% and 17.19% higher compared with the PointNet model, PointNet++SSG model and MSG model, respectively. Individual pig identification based on 3D point clouds of the back surface is effective. This approach is easy to integrate with functions such as body condition assessment and behavior recognition, and is conducive to the development of precision livestock farming.

Multi-Sensor Data Fusion for 3D Reconstruction of Complex Structures: A Case Study on a Real High Formwork Project

As the most comprehensive document types for the recording and display of real-world information regarding construction projects, 3D realistic models are capable of recording and displaying simultaneously textures and geometric shapes in the same 3D scene. However, at present, the documentation for much of construction infrastructure faces significant challenges. Based on TLS, GNSS/IMU, mature photogrammetry, a UAV platform, computer vision technologies, and AI algorithms, this study proposes a workflow for 3D modeling of complex structures with multiple-source data. A deep learning LoFTR network was used first for image matching, which can improve matching accuracy. Then, a NeuralRecon network was employed to generate a 3D point cloud with global consistency. GNSS information was used to reduce search space in image matching and produce an accurate transformation matrix between the image scene and the global reference system. In addition, to enhance the effectiveness and efficiency of the co-registration of the two-source point clouds, an RPM-net was used. The proposed workflow processed the 3D laser point cloud and UAV low-altitude multi-view image data to generate a complete, accurate, high-resolution, and detailed 3D model. Experimental validation on a real high formwork project was carried out, and the result indicates that the generated 3D model has satisfactory accuracy with a registration error value of 5 cm. Model comparison between the TLS, image-based, data fusion 1 (using the common method), and data fusion 2 (using the proposed method) models were conducted in terms of completeness, geometrical accuracy, texture appearance, and appeal to professionals. The results denote that the generated 3D model has similar accuracy to the TLS model yet also provides a complete model with a photorealistic appearance that most professionals chose as their favorite.

Application of 3D point cloud map and image identification to mobile robot navigation

This paper presents application of the combination of mapping, image identification, and indoor navigation for the omnidirectional wheeled mobile robot (WMR). The mapping part uses the point clouds converted from the depth map of the LiDAR camera to construct the indoor 3D point cloud map. Combining this map with the You Only Look Once ver.4 (YOLOv4) network, the door number and the doorknob can be displayed on the map by image identification. The navigation part applies 3D point cloud map, image, and depth map to an omnidirectional WMR so that it can know the current position and plan the path to the designated location. The proposed control scheme can make the robot perform indoor navigation and reach designated location.

Object defect detection based on data fusion of a 3D point cloud and 2D image

In the process of defect detection, there can be interference factors such as poor image quality and missing point cloud data as a result of the complexity of the acquisition environment. Certain limitations can be found relying solely on point cloud data processing or image feature detection. Therefore, this paper tries a more intuitive and effective exploration. Firstly, an algorithm named hole boundary points detection of point cloud, based on multi-scale principal component analysis, is proposed, which can achieve the preliminary detection of hole boundary points while calculating the normal vector of each point in the point cloud. Then the boundary contour of each hole is constructed by a polygon growth algorithm. Finally, we use the complementary information of the 3D point cloud and 2D image to explore the origin of holes and realize the ‘true’ and ‘false’ classification of holes. The experimental results show that our algorithm can successfully detect point cloud holes and can also distinguish them from object defects, providing data support for subsequent hole filling and defect measurement.

Construction of Edge Computing Platform Using 3D LiDAR and Camera Heterogeneous Sensing Fusion for Front Obstacle Recognition and Distance Measurement System

This research aims to utilise heterogeneous sensor fusion using 3D Light Detection and Ranging (LiDAR) and cameras, combined with an object recognition system and a ranging system, to construct an edge computing platform such that a vehicle equipped with the platform can perform computations offline in real time. This work comprises two main sections: the first is heterogeneous fusion, and the second is obstacle recognition and ranging detection. To achieve heterogeneous sensor fusion, 3D–3D point matching was used to find rigid body transformation between two sensors and finally project the LiDAR 3D point cloud image onto the 2D image. For object recognition, YOLOv4-Tiny was used as the detection network. A lightweight network architecture and high computational speed could be effectively used on edge computing hardware with limited performance. Further, by drawing the bounding box, we could detect the point cloud within the bounding box to estimate the distance to the obstacle. For detecting distance, we conducted experiments in two ways: ‘minimum point in box’ and ‘median point in box’ and compared the results. With heterogeneous sensor fusion, object recognition and the ranging system, detecting the category and distance of obstacles ahead of the vehicle was possible in real time. Furthermore, integrating the edge computing platform architecture enabled moving the entire system offline, making it an independent system that returns results in real time. Finally, a dynamic test was conducted on a road. The experiment showed that the detection speed of YOLOv4-Tiny in the dynamic test was higher than 60 FPS, and the accuracy rate surpassed 70%. Furthermore, the distance detection error of the 3D LiDAR was less than 3 cm, which is sufficiently accurate to be applied to complex environments on roads.

Time-of-flight measurement-based three-dimensional profiler system employing a lightweight Fresnel-type Risley prism scanner

A time-of-flight measurement-based three-dimensional (3D) profiler system employing a lightweight scanning system is demonstrated. To reduce the weight of the scanning system, and thereby achieve faster scanning speeds, two Fresnel prism sheets were employed as the scanning optics and installed to work as a pair of Risley prisms. Each Fresnel prism sheet has a diameter of 102 mm and mass of 15 g, which is about 12 times lighter than ordinary bulky prism. By scanning the laser beam with the developed scanning system, a 3D point cloud image of a target object located 8 m away could be successfully obtained. The image distortion was removable by correcting six geometrical parameters of the scanner using a simple optimization algorithm. It was confirmed by the experiment that once the distortion has been corrected, it is valid for other scanning speeds (and trajectories), enabling 3D profile measurements that do not require postprocessing of measured data. Measurement results for a standard target composed of square extrusions were in good agreement with the reference values, with deviations of <1 mm.

Design and Realization of Wide Field-of-View 3D MEMS LiDAR

Light detection and ranging (LiDAR) sensors are promising for automated transportation to detect the surrounding environment. However, most LiDAR solutions are complex and bulky. By designing a MEMS-mirror-based LiDAR, we can improve the volume constraints, but MEMS mirrors could limit scanning angles. In this work, we simulated and demonstrated a MEMS LiDAR system to solve the current obstacles. Combining a MEMS mirror and a wide-angle lens into the system, small-volume and large field-of-view (FOV) LiDAR systems can be realized. We use ray tracing optical simulation software to design a pair of aspherical lenses to expand the scanning angle. After the laser beam passes through the wide-angle lens, the FOV can be increased to 104 degrees. The distortion of the wide-angle lens is controlled below 3%, making the scanned image precise to the actual situation. In order to experimentally demonstrate the small-volume MEMS scanning LiDAR, a modular laser rangefinder is used with a MEMS mirror. The entire system of the LiDAR scanner is around 15 cm <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times5$ </tex-math></inline-formula> cm <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times2.5$ </tex-math></inline-formula> cm. In the natural light environment for wide-angle LiDAR measurement, the maximum error is less than 2%. Finally, an image processing program is written to convert the scanned data into a 3D point cloud image, and the generated image proves the complete function of the proposed LiDAR.

Classification Using 3D Point Cloud and 2D Image on Abstract Objects

While classification using machine learning is exceptionally successful with 2D images, it is more challenging to classify 3D objects. However, 3D objects classification is critical because of its application in autonomous vehicles and robotics. This paper compared neural networks with similar structures using 3D point clouds and 2D images on the same objects. We also generated objects with abstract design and input them into the neural networks we created. We find clear disadvantages with classifying abstract objects compared to ordinary objects for both neural networks. We believe having contextual information will help to address this problem. We also observed that the neural network based on images performs worse than that based on point clouds. However, image based classification takes less time to train compared to point cloud based classification.

Achieve accurate recognition of 3D point cloud images by studying the scattering characteristics of typical targets

Three-dimensional imaging lidar has been widely used in various fields due to its high measurement accuracy, strong directionality, and three-dimensional visualization. At present, the three-dimensional point cloud imaging method of lidar mainly depends on the distance information of the target, but the intensity information of the target can play an auxiliary role in the recognition of the target, and it has also received extensive attention. The combination of intensity information and distance information enables the system to make more accurate decisions and improve the redundancy of the system. When scanning non-cooperative targets, if there is only distance information, the target will not be accurately identified, which may cause errors in decision-making. This paper studies and analyzes the scattering characteristics of typical lidar targets. The intensity of the scattered echo signal of the target is related to the target material, target distance, gray value and detection angle, so the detector's receiving scattering rate is different. For the extraction of the intensity and the distance signal, we propose a double-scale intensity-weighted centroid algorithm to achieve it, which ensures the accuracy of the signal. To this end, we build a database of the scattering characteristics of related targets, combined with the original distance information to realize the accurate recognition of the target in the lidar 3D point cloud image.

Interactive Holographic Display for Real-Time Drawing and Erasing of 3D Point-Cloud Images With a Fingertip

We demonstrate an interactive holographic display system for real-time drawing and erasing of 3D point-cloud images. This system detects the position of the observer's fingertip with a motion sensor, which it uses to calculate the complex amplitude distribution for the hologram. Because the calculated distribution is added to or subtracted from the complex amplitude distribution of the previous frame, the calculation time per frame is independent of the number of points comprising the 3D point cloud. Then, the system can quickly update the hologram even when the number of points comprising the 3D point cloud is increased. Because the 3D point-cloud image reconstructed from the calculated hologram is projected at the same position as the observer's fingertip with almost no latency, the system can draw and erase a 3D point-cloud image through interactions with the observer's fingertip. We experimentally confirmed that the system successfully draws and erases a 3D point-cloud image.

Sensors Fusion and Multidimensional Point Cloud Analysis for Electrical Power System Inspection.

Thermal inspection is a powerful tool that enables the diagnosis of several components at its early stages. One critical aspect that influences thermal inspection outputs is the infrared reflection from external sources. This situation may change the readings, demanding that an expert correctly define the camera position, which is a time consuming and expensive operation. To mitigate this problem, this work proposes an autonomous system capable of identifying infrared reflections by filtering and fusing data obtained from both stereo and thermal cameras. The process starts by acquiring readings from multiples Observation Points (OPs) where, at each OP, the system processes the 3D point cloud and thermal image by fusing them together. The result is a dense point cloud where each point has its spatial position and temperature. Considering that each point’s information is acquired from multiple poses, it is possible to generate a temperature profile of each spatial point and filter undesirable readings caused by interference and other phenomena. To deploy and test this approach, a Directional Robotic System (DRS) is mounted over a traditional human-operated service vehicle. In that way, the DRS autonomously tracks and inspects any desirable equipment as the service vehicle passes them by. To demonstrate the results, this work presents the algorithm workflow, a proof of concept, and a real application result, showing improved performance in real-life conditions.

Performance of Polar-Coded 3D Image Transmission over Fading Channel

Any signal transmitted over an air-to-ground channel is corrupted by fading, noise, and interference. In this paper, a Polar-coded 3D point cloud image transmission system with fading channel is modeled, and also the simulation is performed to verify its performance in terms of 3D point cloud image data transmission over Rician channel with Gaussian white noise and overlap of Gaussian white noise + periodic pulse jamming separately. The comparison of Polar-coded scheme with RS-coded scheme in the same scenario indicates that Polar-coded system gives far better performance against AWGN noise and fading than the RS-coded system does in the case of short block length. But RS-coded scheme shows better performance on antipulse jamming than that of Polar-coded scheme, while there is no interleaving between codewords.

Identifying rice seedling bands based on slope virtualization clustering

In this paper, a method for identifying rice seedling band tracks by using LIDAR data is proposed, and the method provides navigation information for straight-line driving to minimize seedling injuries when paddy field management machinery enters a rice field. In this paper, the reconstruction of seedling band tracks is realized by constructing a three-dimensional (3D) point cloud image by collecting LIDAR data. Based on a two-dimensional servo platform, a mean-shift clustering algorithm is designed with slope virtualization, iterative zone clustering and linearization processes. The point cloud data of rice field on the 35th day of transplanting were simulated by Matlab. In the environment 2–3 m in the forward-looking direction and a standard plant spacing of 21 cm, the reconstructed seedling trajectory was compared with the real seedling trajectory. The maximum parallelism of the seedling belt trajectory was 45 mm, and the maximum median deviation was 7 mm. These values meet the needs of paddy field management machinery for achieving nondestructive seedling driving trajectories and verify the feasibility of the slope-blurred mean-shift clustering algorithm for seedling identification.