Point Cloud Generation Research Articles

A Coarse-to-Fine Transformer-Based Network for 3D Reconstruction from Non-Overlapping Multi-View Images

Reconstructing 3D structures from non-overlapping multi-view images is a crucial task in the field of 3D computer vision, since it is difficult to establish feature correspondences and infer depth from overlapping parts of views. Previous methods, whether generating the surface mesh or volume of an object, face challenges in simultaneously ensuring the accuracy of detailed topology and the integrity of the overall structure. In this paper, we introduce a novel coarse-to-fine Transformer-based reconstruction network to generate precise point clouds from multiple input images at sparse and non-overlapping viewpoints. Specifically, we firstly employ a general point cloud generation architecture enhanced by the concept of adaptive centroid constraint for the coarse point cloud corresponding to the object. Subsequently, a Transformer-based refinement module applies deformation to each point. We design an attention-based encoder to encode both image projection features and point cloud geometric features, along with a decoder to calculate deformation residuals. Experiments on ShapeNet demonstrate that our proposed method outperforms other competing methods.

Three-Dimensional Point Cloud Denoising for Tunnel Data by Combining Intensity and Geometry Information

At present, three-dimensional laser scanners are used to scan subway shield tunnels and generate point cloud data as the basis for extracting a variety of information about tunnel defects. However, there are obstacles in the tunnel such as pipelines, tracks, and signaling systems that cause noise in the point cloud. Usually, the data of the tunnel point cloud are huge, and the efficiency of artificial denoising is low. Faced with this problem, based on the respective characteristics of the geometric shape and reflection intensity of the tunnel point cloud and their correlation, this paper proposes a tunnel point cloud denoising method. The method includes the following three parts: reflection intensity threshold denoising, joint shape and reflection intensity denoising, and shape denoising. Through the experiment on the single-ring segment point cloud of a shield tunnel, the method proposed in this paper takes 2 min to remove 99.77% of the noise in the point cloud. Compared with manual denoising, the method proposed in this paper takes two fifteenths of the time to achieve the same denoising effect. The method proposed in this paper meets the requirements of a tunnel point cloud data survey. Thus, it provides support for the efficient, accurate, and automatic daily maintenance and surveys of tunnels.

DPR-GAN: Dual-Stream Progressive Refinement for Adversarial 3D Point Cloud Generation

Point cloud generation aims to transfer a latent code to realistic 3D shapes through generative models. However, most of progressive generative methods ignore the spatial relationship among different stages and suffer from the loss of contextual information. To address this issue, we propose dual-stream progressive refinement adversarial network (DPR-GAN), which utilizes a dual-stream structure to establish the relationship between two adjacent stages. Such a mechanism can learn the spatial context and preserve more spatial details of point clouds at different stages. In addition, DPR-GAN adopts 3D gridding transformation to guide the shape deformation. In this way, 3D gridding transformation can learn a reasonable correspondence between the local regions of 3D shapes and latent codes. Benefiting from the uniformity and adaptability of the 3D grids, our proposed DPR-GAN can improve the quality and consistency of generated point clouds. We conduct comprehensive experiments to demonstrate that the proposed DPR-GAN is capable of generating pluralistic point clouds, as compared with state-of-the-art generation methods in terms of both visual and quantitative evaluations.

COMPARISON OF STATE-OF-THE-ART MULTI-VIEW STEREO SOLUTIONS FOR CLOSE RANGE HERITAGE DOCUMENTATION

Abstract. In recent years novel 3D reconstruction methods have been developed to improve the conventional image-based point cloud generation techniques. These novel methods generally attempt to address various challenges encountered in conventional methods, namely, the reconstruction of reflective surfaces and the amount of processing time required, both of which are major bottlenecks in heritage documentation and especially those related to large and complex objects. In this paper, we identified three types of 3D image-based reconstruction techniques and tested their usage on heritage datasets, namely (1) conventional multi-view stereo (MVS), (2) learning-based MVS, and (3) neural radiance fields (NeRF). The aim of this study is to determine the capabilities of these methods in reconstruction of three different heritage-related datasets with different challenges. Our results show that conventional MVS is nowadays a reliable solution for 3D reconstruction, in many instances recording good results relative to the reference terrestrial laser scans (TLS) when properly deployed. When applied to a challenging highly reflective scene, conventional MVS fared well using the PatchMatch algorithm (reaching an object completeness rate of 99.05%), while NeRF’s best performance was 99.98%. However, NeRF suffered from noisy data, some of which may stem from its radiance field-to-point cloud conversion method. The results show that there is great potential in using specific methods for specific cases, and research in combining them may yield interesting results in the future.

Deep learning framework for three-dimensional surface reconstruction of object of interest in photoacoustic tomography.

Photoacoustic tomography (PAT) is a non-ionizing hybrid imaging technology of clinical importance that combines the high contrast of optical imaging with the high penetration of ultrasonic imaging. Two-dimensional (2D) tomographic images can only provide the cross-sectional structure of the imaging target rather than its overall spatial morphology. This work proposes a deep learning framework for reconstructing three-dimensional (3D) surface of an object of interest from a series of 2D images. It achieves end-to-end mapping from a series of 2D images to a 3D image, visually displaying the overall morphology of the object. The framework consists of four modules: segmentation module, point cloud generation module, point cloud completion module, and mesh conversion module, which respectively implement the tasks of segmenting a region of interest, generating a sparse point cloud, completing sparse point cloud and reconstructing 3D surface. The network model is trained on simulation data sets and verified on simulation, phantom, and in vivo data sets. The results showed superior 3D reconstruction performance both visually and on the basis of quantitative evaluation metrics compared to the state-of-the-art non-learning and learning approaches. This method potentially enables high-precision 3D surface reconstruction from the tomographic images output by the preclinical PAT system without changing the imaging system. It provides a general deep learning scheme for 3D reconstruction from tomographic scanning data.

Dynamic Scene Graph Generation of Point Clouds with Structural Representation Learning

Dynamic Scene Graph Generation of Point Clouds with Structural Representation Learning

Selection of an Algorithm for Assessing the Verticality of Complex Slender Objects Using Semi-Automatic Point Cloud Analysis

Remote technologies, including laser scanning, are frequently employed to acquire data describing the geometric condition of engineering objects. The automation of point cloud processing becomes essential for promptly and reliably monitoring changes in their current shape. The article introduces a methodology for generating point clouds, focusing on detecting the shape of the object’s cross profiles and subsequently determining its inclination through simulations and real data recorded using terrestrial laser scanning technology. The simulations enabled the identification of variations in the characteristics of changes in the course of the axis of a slender structure, depending on the adopted calculation method. Point clouds derived from measurements of complex engineering objects facilitated the validation of the assumptions of the proposed methodology. The suggested solution enables the semi-automatic extraction of data from point clouds and the assessment of the geometric state of engineering object axes based on multi-temporal point clouds.

A Non-Contact Measurement of Animal Body Size Based on Structured Light

To improve the accuracy of non-contact measurements of animal body size and reduce costs, a new monocular camera scanning equipment based on structured light was built with a matched point cloud generation algorithm. Firstly, using the structured light 3D measurement model, the camera intrinsic matrix and extrinsic matrix could be calculated. Secondly, the least square method and the improved segment–facet intersection method were used to implement and optimize the calibration of the light plane. Then, a new algorithm was proposed to extract gray- centers as well as a denoising and matching algorithm, both of which alleviate the astigmatism of light on animal fur and the distortion or fracture of light stripes caused by the irregular shape of an animal’s body. Thirdly, the point cloud was generated via the line–plane intersection method from which animal body sizes could be measured. Finally, an experiment on live animals such as rabbits and animal specimens such as fox and the goat was conducted in order to compare our equipment with a depth camera and a 3D scanner. The result shows that the error of our equipment is approximately 5%, which is much smaller than the error of the other two pieces of equipment. This equipment provides a practicable option for measuring animal body size.

Research on micro/nano scale 3D reconstruction based on scanning electron microscope

Scanning electron microscopy (SEM) has an important application in the petroleum field, which is often used to analyze the microstructure of reservoir rocks, etc. Most of these analyses are based on two-dimensional images. In fact, SEM can carry out micro-nano scale three-dimensional measurement, and three-dimensional models can provide more accurate information than two-dimensional images. Among the commonly used SEM 3D reconstruction methods, parallax depth mapping is the most commonly used method. Multiple SEM images can be obtained by continuously tilting the sample table at a certain Angle, and multiple point clouds can be generated according to the parallax depth mapping method, and a more complete point clouds recovery can be achieved by combining the point clouds registration. However, the root mean square error of the point clouds generated by this method is relatively large and unstable after participating in point clouds registration. Therefore, this paper proposes a new method for generating point clouds. Firstly, the sample stage is rotated by a certain angle to obtain two SEM images. This operation makes the rotation matrix a known quantity. Then, based on the imaging model, an equation system is constructed to estimate the unknown translation parameters, and finally, triangulation is used to obtain the point clouds. The method proposed in this paper was tested on a publicly available 3D SEM image set, and the results showed that compared to the disparity depth mapping method, the point clouds generated by our method showed a significant reduction in root mean square error and relative rotation error in point clouds registration.

Investigating tooth surface reconstruction principle of a gear rack and pinion system with variable transmission ratio pinion

Rack-and-pinion gear sets with variable transmission ratios have gradually become a standard core component in steering systems, as the harmony between portability and sensitivity in vehicle steering can be achieved with such applications. When the conventional meshing theory is adopted to calculate point clouds of variable-ratio tooth surfaces, drawbacks will occur with respect to unevenness in point cloud density, inaccessible tooth surface boundaries, and point cloud fusion at the interface between the working tooth surface and fillet. Thus, the generated point cloud of variable ratio tooth surfaces is inapplicable for computer-aided engineering (CAE) and plastic forming. In this study, the point cloud reconstruction of pinion tooth surfaces with variable transmission ratio was investigated. Meanwhile, based on the analysis of transient length in contact lines that occur during gear meshing, various criteria were proposed to determine the validity of the variable-ratio rack-and-pinion design in terms of whether continuous and reverse driving can be achieved. By adopting the proposed methods, the designed variable-ratio pinion tooth surfaces demonstrate point clouds with equidistant distributions and clear boundaries. CAE analysis results also indicate a good quality in gear meshing. The real-time gear meshing status of variable-ratio rack-and-pinion sets is characterized by the transient length of contact lines. The measured variation curves for the transmission ratios of a prototype demonstrate significant consistency with the theoretical curves, indicated by attaining transmission ratio errors within a reasonable range. The validity of the design methods proposed in this study has been verified by the aforementioned results.

Examples of using laser scanning as a support for traditional measuring methods in hard coal mining

In recent years, the growing interest in new techniques for spatial data acquisition, processing and analysis has also found applications in surveying and geological departments, where it is used in the process of developing documentation. In the conditions of mining operations, surveying measurements must be carried out quickly, safely and with sufficient accuracy. The use of laser scanning allows the collection of a significant amount of data, which requires time-consuming processing, but is necessary for accurate modelling of objects in 3D. The legal basis for the use of laser scanning in mining is the acts that regulate activities in the field of surveying, geological survey documentation and technical standards for surveying. The process of implementing laser scanning in underground mining requires compliance with occupational safety regulations and technical standards, as well as proper training of personnel. The laser scanners used in practice are equipped with the SLAM algorithm, enabling precise measurement and point cloud generation. Appropriate software is crucial for processing and analysing the collected data. The use of laser scanning makes it possible to efficiently perform non-standard measurements, which contributes to streamlining work at mining facilities. However, to realize its full potential, it is necessary to train personnel and raise the level of use of this technology.

High-Precision Intracity Temperature Estimation Based on Generated Point Clouds

High-Precision Intracity Temperature Estimation Based on Generated Point Clouds

An adaptive algorithm for generating 3D point clouds of the human body based on 4D millimeter-wave radar.

The traditional algorithms for generating 3D human point clouds often face challenges in dealing with issues such as phantom targets and target classification caused by electromagnetic multipath effects, resulting in a lack of accuracy in the generated point clouds and requiring manual labeling of the position of the human body. To address these problems, this paper proposes an adaptive method for generating 3D human point clouds based on 4D millimeter-wave radar (Self-Adaptive mPoint, SA-mPoint). This method estimates the rough human point cloud by considering micro-motion and respiration characteristics while combining the echo dynamic with static information. Furthermore, it enhances the density of point cloud generation. It reduces interference from multipath noise through multi-frame dynamic fusion and an adaptive density-based clustering algorithm based on the center points of humans. The effectiveness of the SA-mPoint algorithm is verified through experiments conducted using the TI Millimeter Wave Cascade Imaging Radar Radio Frequency Evaluation Module 77G 4D cascade radar to collect challenging raw data consisting of single-target and multi-target human poses in an open classroom setting. Experimental results demonstrate that the proposed algorithm achieves an average accuracy rate of 97.94% for generating point clouds. Compared to the popular TI-mPoint algorithm, it generates a higher number of point clouds on average (increased by 87.94%), improves the average accuracy rate for generating point clouds (increased by 78.3%), and reduces the running time on average (reduced by 11.41%). This approach exhibits high practicality and promising application prospects.

4D High-Resolution Imagery of Point Clouds for Automotive mmWave Radar

In the community of automotive millimeter wave radar, the recently developed concept of four-dimensional (4D) radar can provide high-resolution point clouds image with enhanced imaging performance. Currently, the density of point clouds for single-frame image is usually too sparse to satisfy the demands of target classification and recognition due to the limitation of Doppler and angle resolutions. To address the aforementioned issues, a novel algorithm is proposed for 4D high-resolution imagery generation of point clouds with extremely high Doppler and angle resolutions in this paper. For high Doppler resolution with high-dynamic, a novel velocity ambiguity resolution algorithm is proposed using a dual pulse repetition frequency (dual-PRF) waveform design embedded in an innovative time-division multiplexing & Doppler-division multiplexing MIMO (TDM-DDM-MIMO) framework. Meanwhile, an attractive complex-valued deep convolutional network (CV-DCN) of super-resolution direction-of-arrival (DOA) estimation is proposed only using single-frame data. To be specific, a spatial smoothing operator on array data is applied as input of the network, and a CV-DCN is designed to learn the transformation of the spatial spectrum from the end-to-end to effectively protect the spectrum extraction. Furthermore, experimental analysis is performed to confirm the effectiveness of the proposed super-resolution DOA estimation algorithm. Finally, the 4D high-resolution imagery of point clouds is obtained by experiments in the parking lot.

Efficient 3D Reconstruction Through Enhanced PatchMatch Techniques for Accelerated Point Cloud Generation

Efficient 3D Reconstruction Through Enhanced PatchMatch Techniques for Accelerated Point Cloud Generation

3D Point Cloud Generation Based on Low-Cost and Portable FMCW Radar for Poor-Visibility Environments

3D Point Cloud Generation Based on Low-Cost and Portable FMCW Radar for Poor-Visibility Environments

Advancements in Radar Point Cloud Generation and Usage in Context of Healthcare and Assisted Living Domain: A Review

Advancements in Radar Point Cloud Generation and Usage in Context of Healthcare and Assisted Living Domain: A Review

From Noise Addition to Denoising: A Self-Variation Capture Network for Point Cloud Optimization.

Point clouds obtained from 3D scanners are often noisy and cannot be directly used for subsequent high-level tasks. In this article, we propose a novel point cloud optimization method capable of denoising and homogenizing point clouds. Our idea is based on the assumption that the noise is generally much smaller than the effective signal. We perform noise perturbation on the noisy point cloud to get a new noisy point cloud, called self-variation point cloud. The noisy point cloud and self-variation point cloud have different noise distribution, but the same point cloud distribution. We compute the potential commonality between two noisy point clouds to obtain a clean point cloud. To implement our idea, we propose a Self-Variation Capture Network (SVCNet). We perturb the point cloud features in the latent space to obtain self-variation feature vectors, and capture the commonality between two noisy feature vectors through the feature aggregation and averaging. In addition, an edge constraint module is introduced to suppress low-pass effects during denoising. Our denoising method does not take into account the noise characteristics, and can filter the drift noise located on the underlying surface, resulting in a uniform distribution of the generated point cloud. The experimental results show that our algorithm outperforms the current state-of-the-art algorithms, especially in generating more uniform point clouds. In addition, extended experiments demonstrate the potential of our algorithm for point clouds upsampling.

FOREST FEATURE LIDAR SLAM (F2-LSLAM) AND INTEGRATED SCAN SIMULTANEOUS TRAJECTORY ENHANCEMENT AND MAPPING (IS2-TEAM) FOR ACCURATE FOREST INVENTORY USING BACKPACK SYSTEMS

Abstract. Under-canopy mapping is desired to derive critical forest biometrics, such as diameter at breast height (DBH), merchantable height, and debris volume. The main challenge of such under-canopy mapping is the intermittent access to the global navigation satellite system (GNSS) signal, which is crucial to deriving accurately georeferenced mapping products. In this study, we propose two frameworks – Forest Feature LiDAR Simultaneous Localization and Mapping (F2-LSLAM) and Integrated Scan Simultaneous Trajectory Enhancement and mapping (IS2-TEAM) – for 3D LiDAR unit mounted on backpack systems to achieve accurate forest inventory. In the F2-LSLAM strategy, ground/tree trunk features are extracted from individual LiDAR scans. On the other hand, when trajectory information provided by navigation sensors is available, these semantic features are derived from LiDAR points within several scans (i.e., integrated scan) for the IS2-TEAM strategy. Then, local/global least squares adjustment (LSA) using derived features is performed to register LiDAR scans to a common reference frame for both strategies. To evaluate the performance of the proposed strategies, three in-house developed backpack systems with varying specifications were used to collect data in complicated forest environments. Through the comparison with point clouds acquired by a commercial backpack LiDAR system, the proposed frameworks are capable of generating point clouds with satisfactory intra-dataset alignment quality (in the range of 2–4 cm) for all backpack systems in natural forest areas with relatively flat terrain. However, for more challenging areas with dense undergrowth vegetation and/or large height differences, F2-LSLAM framework cannot extract sufficient features, while IS2-TEAM still exhibits good performance.

Use of Smartphone Lidar Technology for Low-Cost 3D Building Documentation with iPhone 13 Pro: A Comparative Analysis of Mobile Scanning Applications

Laser scanning technology has long been the preferred method for capturing interior scenes in various industries. With a growing market, smaller and more affordable scanners have emerged, offering end products with sufficient accuracy. While not on par with professional scanners, Apple has made laser scanning technology accessible to users with the introduction of the new iPhone Pro models, democratizing 3D scanning. Thus, this study aimed to assess the performance of the iPhone’s lidar technology as a low-cost solution for building documentation. Four scanning applications were evaluated to determine the accuracy, precision, and user experience of the generated point clouds compared with a terrestrial laser scanner. The results reveal varying performances on the same device, highlighting the influence of software. Notably, there is room for improvement, particularly in tracking the device’s position through software solutions. As it stands, the technology is well suited for applications such as indoor navigation and the generation of quick floor plans in the context of building documentation.