Point Cloud Reconstruction Research Articles

A Novel Filtering Method of 3D Reconstruction Point Cloud from Tomographic SAR

With the development of airborne synthetic aperture radar (SAR) technology, the 3D SAR point cloud reconstruction has emerged as a crucial development trend in the current SAR community. However, due to measurement errors, environmental interference, radar decoherence, and other noises associated with the SAR system, the reconstructed tomogram is often deteriorated by numerous noisy scatterers. As a result, it becomes challenging to obtain high-quality 3D point clouds of the observed object, making it difficult to further process the point cloud and realize target identification. To address these issues, we propose a K nearest neighbor comprehensive weighted filtering algorithm. The filtered point cloud is evaluated quantitatively using three-dimensional entropy. In this study, we adopted various filtering methods for simulated data, P-band data of Genhe, and Ku-band data of Yuncheng to refine the tomogram and compare their performances. Both qualitative and quantitative analyses demonstrate the superiority of the filtering algorithm proposed in this paper.

Application of 2D Images in Visual and Tactile Dimensions of Fiber Art Design

With people’s higher and higher spiritual requirements, fiber art is more common in people’s lives. From ordinary fiber materials to handicrafts and daily necessities, there are shadows of fiber art. This paper aims to study the application of two-dimensional images in the visual and tactile dimensions of fiber art design. This paper proposes a three-dimensional simulation of two-dimensional images of art based on deep learning technology to achieve the purpose of convenient display and also analyzes the feature extraction in fiber art. In order to comprehensively compare the performance of the algorithms, this paper compares the performance of four categories: reconstruction speed, number of reconstructed point clouds, diversity of reconstructable categories, and reconstruction stability. The experimental results show that the number of point clouds obtained by the algorithm proposed in this paper far exceeds the other two reconstruction processes, and the average time is 34.4 min. This also shows that such algorithmic processes are more robust to clothing diversity.

Variable Rate Point Cloud Geometry Compression Method

With the development of 3D sensors technology, 3D point cloud is widely used in industrial scenes due to their high accuracy, which promotes the development of point cloud compression technology. Learned point cloud compression has attracted much attention for its excellent rate distortion performance. However, there is a one-to-one correspondence between the model and the compression rate in these methods. To achieve compression at different rates, a large number of models need to be trained, which increases the training time and storage space. To address this problem, a variable rate point cloud compression method is proposed, which enables the adjustment of the compression rate by the hyperparameter in a single model. To address the narrow rate range problem that occurs when the traditional rate distortion loss is jointly optimized for variable rate models, a rate expansion method based on contrastive learning is proposed to expands the bit rate range of the model. To improve the visualization effect of the reconstructed point cloud, a boundary learning method is introduced to improve the classification ability of the boundary points through boundary optimization and enhance the overall model performance. The experimental results show that the proposed method achieves variable rate compression with a large bit rate range while ensuring the model performance. The proposed method outperforms G-PCC, achieving more than 70% BD-Rate against G-PCC, and performs about, as well as the learned methods at high bit rates.

DDL-MVS: Depth Discontinuity Learning for Multi-View Stereo Networks

We propose an enhancement module called depth discontinuity learning (DDL) for learning-based multi-view stereo (MVS) methods. Traditional methods are known for their accuracy but struggle with completeness. While recent learning-based methods have improved completeness at the cost of accuracy, our DDL approach aims to improve accuracy while retaining completeness in the reconstruction process. To achieve this, we introduce the joint estimation of depth and boundary maps, where the boundary maps are explicitly utilized for further refinement of the depth maps. We validate our idea by integrating it into an existing learning-based MVS pipeline where the reconstruction depends on high-quality depth map estimation. Extensive experiments on various datasets, namely DTU, ETH3D, “Tanks and Temples”, and BlendedMVS, show that our method improves reconstruction quality compared to our baseline, Patchmatchnet. Our ablation study demonstrates that incorporating the proposed DDL significantly reduces the depth map error, for instance, by more than 30% on the DTU dataset, and leads to improved depth map quality in both smooth and boundary regions. Additionally, our qualitative analysis has shown that the reconstructed point cloud exhibits enhanced quality without any significant compromise on completeness. Finally, the experiments reveal that our proposed model and strategies exhibit strong generalization capabilities across the various datasets.

Research on 3D reconstruction method of cattle face based on image

As the basis of intelligent breeding management and animal husbandry insurance, the identification of individual cattle is important in animal husbandry management. Given the difficulty of data acquisition caused by the non-rigid and lacking cooperation of cattle, this study proposes a method for cattle face image acquisition and processing that can efficiently adapt to the harsh environment of cattle barns. When processing the non-rigid cow face, the method of approximating the cow face to a rigid body is used to establish the cow face image data set., and the cattle face image data set is established. The Three Dimensional(3D) reconstruction method of cattle face uses a 3D image reconstruction method based on multiple perspectives. First, the scale-invariant feature transform algorithm is used to extract the image feature points. The fast library for approximate nearest neighbors algorithm is used to match feature points. The matching results are selected via random sampling consensus. Second, the structure of the motion method is used for the sparse reconstruction of point clouds, and the dense point cloud is then generated using the three-dimensional multi-view stereo vision algorithm. Finally, the Poisson surface reconstruction method is used for surface reconstruction. The results indicate that this method can effectively realize the three-dimensional reconstruction of cattle faces; the reconstructed images have obvious color, clear texture, and complete shape features.

LiDAR Vehicle Point Cloud Reconstruction Framework for Axle-Based Classification

As Light Detection and Range (LiDAR) technology rapidly advances, it is becoming increasingly viable as a solution to collect vehicle classification data. The main challenge of LiDAR compared with video in vehicle classification lies in its resolution, which limits the ability of LiDAR-based models to classify vehicles in detail from a single captured frame. This paper proposes a novel framework for reconstructing the vehicle point clouds generated by a roadside LiDAR sensor, by considering ground plane constraints and developing a bootstrap aggregating deep neural network (bagging DNN) model to classify the reconstructed vehicle point clouds, according to the US Federal Highway Administration (FHWA) axle-based vehicle classification scheme. First, a probabilistic-based registration algorithm is used to estimate the transformation matrix between consecutive frames of each vehicle point cloud. Then, a multiway registration is conducted to fine-tune the estimated transformation matrices to rebuild the 3D model of each moving vehicle. Second, key features are extracted from the reconstructed vehicle models and fed into a bagging DNN model to provide classifications based on the FHWA scheme. The classification model with the reconstruction framework outperforms the latest developed LiDAR-based FHWA classification model in terms of both accuracy and robustness. The model has an 83 percent average correct classification rate (CCR) on the test set. Remarkably, the proposed model can accurately distinguish Classes 5 and 8 trucks, which have overlapping axle configurations, with a 97 percent and a 90 percent CCR, respectively.

Preliminary exploration of geometrical regularized SAR tomography

Synthetic Aperture Radar Tomography (TomoSAR) is a popular technique in the SAR imaging community due to its ability to solve the problem of layover and obtain a three-dimensional model of observed scene. However, current TomoSAR imaging methods rely on estimating the reflectivity distribution in the elevation direction pixel by pixel based on multi-baseline observations. This approach requires tens or even hundreds of repeated-track observations to achieve satisfactory results, which limits its applicability in time-sensitive applications. With the knowledge that humans can infer three-dimensional structures from a single two-dimensional SAR image, we propose a geometrical regularized TomoSAR imaging approach that leverages three-dimensional clues. The approach first extracts geometrical structures, which are then used to estimate the expected scatterer distribution using the initial reconstructed point cloud. This reduces uncertainty in the scatterer’s position and allows for an adaptive restriction of the search range based on different metrics, such as signal-to-noise level. As a result, the proposed method yields better performance with fewer observations and lower signal-to-noise levels. Simulation results demonstrate the superior performance of our proposed method over classical TomoSAR imaging methods in several aspects. Additionally, experiments with our own multi-baseline systems show that the proposed method outperforms traditional approaches with respect to 3D entropy, and outliers are significantly restricted to the estimated geometrical structures, which confirms the effectiveness of our proposed method.

A pose estimation method of space non-cooperative target based on ORBFPFH SLAM

In this paper, to improve the pose measurement accuracy of the Time-of-Flight (ToF) camera for space non-cooperative targets, a pose estimation method based on ORBFPFH Simultaneous Localization and Mapping (SLAM) with effective integration of strength and depth measurement information is proposed. The primary process of the method is as follows: First, a unique ORBFPFH Bag of Words (BoW) model for space targets is trained based on the ToF image dataset of spacecraft; Secondly, the pose of the non-cooperative space target is tracked based on the ORBFPFH feature and optimized using the pose graph; Then, based on the ORBFPFH BoW model, loop closure is detected to reduce the cumulative error. Finally, the proposed method is tested on the ToF image dataset: compared with the advanced ORB-SLAM2 algorithm, the proposed method has a minor translation error and rotation error on the test data, with a mean translation error less than 0.144 m and mean rotation error less than 0.642°. The test results show that the proposed method can improve the pose estimation accuracy of space non-cooperative targets, achieve a better 3D point cloud reconstruction effect, and provide technical support for space applications such as rendezvous and docking of non-cooperative spacecraft. © 2001 Elsevier Science. All rights reserved

MOBILE MAPPING FOR DIGITAL DOCUMENTATION OF ANCIENT ROMAN ROADS WITHIN A MULTI-SENSOR FRAMEWORK: THE APPIAN WAY IN ROME

Abstract. Digitization of complex Cultural Heritage sites requires the integration of several strategies to achieve a comprehensive description of the site. This is the case of the digitization project of the Appian Way in Rome involving a section of 12 km from Capo di Bove to Frattocchie under the superintendence of the Parco Archeologico dell’Appia Antica. This paper presents some first results of the project with a specific focus on the acquisition of the primary data needed for the further steps of the project. In particular, the paper discusses the integration of different sensors such as lidar-based Mobile Mapping Systems, spheric (360°) cameras and digital (frame) calibrated cameras, for the reconstruction of point clouds to be used as input data for the digitization process. A specific attention is paid both on acquisition speed and to accuracy of the obtained results. The paper presents also some results showing the transformation of the acquired data into a GIS system to be used as a support tool for decision making of the public administration.

Quantitative estimation of organ-scale phenotypic parameters of field crops through 3D modeling using extremely low altitude UAV images

Plant phenotypic parameters provide key information in modern crop breeding. However, the rapid and accurate estimation of organ-scale phenotypic parameters remains a challenge. In this context, the present study proposed a novel methodology for the automatic quantification of the organ-scale parameters of field crops using the unmanned aerial vehicle (UAV) platform. First, a lightweight UAV was employed to capture the multi-view and high-resolution image sequences of field crops at an extremely-low flight altitude. Subsequently, based on these image sequences, point cloud reconstruction of the canopy was conducted. Next, the geometrical model of individual leaves was reconstructed using the modified Crust algorithm and optimized by abnormal facet elimination and leaf surface repair. Finally, individual leaf phenotypic parameters were calculated based on the reconstructed geometrical models. The method was evaluated by comparing the calculated parameters with actual measurements. The calculated values for leaf length, maximum leaf width, and leaf area were in good agreements with the measured values (maize: R2 > 0.97 for all parameters, RMSE for length, width, and leaf area was 2.6 cm, 0.4 cm, and 33.2 cm2, respectively; soybean: R2 > 0.85 for all parameters, RMSE for the counterparts was 0.3 cm, 0.5 cm, and 2.3 cm2, respectively; tobacco: R2 > 0.89 for all parameters, RMSE for the counterparts was 4.1 cm, 1.4 cm, and 42.6 cm2, respectively). The methodology based on extremely-low altitude UAV images has promising prospects in the crop breeding program for the automatic acquisition of fine organ-scale parameters with high efficiency.

Method of unwrapped phase error correction in phase-measuring profilometry based on adaptive local window

Phase measurement profilometry (PMP) is widely used to reconstruct three-dimensional (3D) shape of objects. However, errors are introduced during standard phase unwrapping process, which considerably brings down measurement accuracy and lowers method reliability. This paper proposes a novel method to identify the errors, and subsequently correct these errors through erecting adaptive local windows. Under proposal, two robust algorithms are developed to correct these errors in initial rudimentary correction as well as in subsequent refined error removal. The adaptive local window is designed as such that the fringe order corresponding to each period of the wrapping phase is independent of each other. Therefore, it avoids the error transmission in the correction process. Experiments are carried out on absolute phase error correction and 3D point cloud reconstruction. Experimental results verified the developed method in terms of reducing the stratification of 3D point cloud and improving the quality of 3D measurement.

Zero-reference deep learning for low-light image enhancement of underground utilities 3D reconstruction

Image-based 3D reconstruction has become one of the most promising as-built construction modeling methods for its high cost-efficiency and outstanding performance. However, the quality performance of image-based 3D reconstruction is very sensitive to the illumination conditions. To date, the image-based 3D reconstruction in low-light environment is mainly optimized by traditional approaches that are time-consuming and manual parameters required. And the supervised deep learning methods request suitable paired image data (low-light images and the paired reference images). Therefore, a Zero-reference Deep learning model for the low-light image Enhancement for underground utilities 3D reconstruction (ZDE3D) is proposed in this paper. ZDE3D improved the 3D reconstruction performance of low-light images by unsupervised loss functions design without paired or unpaired training datasets. Field experiments implemented confirms that the capability of ZDE3D for increasing the quantity of sparse reconstruction point cloud by 13.19% on average and the reconstruction accuracy reached 98.58%.

High-Capacity Spatial Structured Light for Robust and Accurate Reconstruction

Spatial structured light (SL) can achieve three-dimensional measurements with a single shot. As an important branch in the field of dynamic reconstruction, its accuracy, robustness, and density are of vital importance. Currently, there is a wide performance gap of spatial SL between dense reconstruction (but less accurate, e.g., speckle-based SL) and accurate reconstruction (but often sparser, e.g., shape-coded SL). The central problem lies in the coding strategy and the designed coding features. This paper aims to improve the density and quantity of reconstructed point clouds by spatial SL whilst also maintaining a high accuracy. Firstly, a new pseudo-2D pattern generation strategy was developed, which can improve the coding capacity of shape-coded SL greatly. Then, to extract the dense feature points robustly and accurately, an end-to-end corner detection method based on deep learning was developed. Finally, the pseudo-2D pattern was decoded with the aid of the epipolar constraint. Experimental results validated the effectiveness of the proposed system.

MPED: Quantifying Point Cloud Distortion Based on Multiscale Potential Energy Discrepancy.

In this article, we propose a new distortion quantification method for point clouds, the multiscale potential energy discrepancy (MPED). Currently, there is a lack of effective distortion quantification for a variety of point cloud perception tasks. Specifically, in human vision tasks, a distortion quantification method is used to predict human subjective scores and optimize the selection of human perception task parameters, such as dense point cloud compression and enhancement. In machine vision tasks, a distortion quantification method usually serves as loss function to guide the training of deep neural networks for unsupervised learning tasks (e.g., sparse point cloud reconstruction, completion, and upsampling). Therefore, an effective distortion quantification should be differentiable, distortion discriminable, and have low computational complexity. However, current distortion quantification cannot satisfy all three conditions. To fill this gap, we propose a new point cloud feature description method, the point potential energy (PPE), inspired by classical physics. We regard the point clouds are systems that have potential energy and the distortion can change the total potential energy. By evaluating various neighborhood sizes, the proposed MPED achieves global-local tradeoffs, capturing distortion in a multiscale fashion. We further theoretically show that classical Chamfer distance is a special case of our MPED. Extensive experiments show that the proposed MPED is superior to current methods on both human and machine perception tasks. Our code is available at https://github.com/Qi-Yangsjtu/MPED.

A 3D shape measurement method for high-reflective surface based on dual-view multi-intensity projection

Phase shifting profilometry has been commonly used in three-dimensional shape measurement with the advantages of high-precision and non-contact. However, it is still challenging to measure high-reflective surface because image saturation will lead to absolute phase errors and reconstruction errors. In this paper, a dual-view multi-intensity projection method was proposed. Compared with the single-view method, the proposed method can reconstruct more points at each projection intensity especially for pixels around the specular angle and reduce the number of projections to reduce the time consumption. First, we established the dual-view structured light system consisting of two monocular systems that share the same projector. Subsequently, a dual-view saturated pixel judging method was proposed that enables the reconstruction results under two views to be combined without duplicate points. The multi-intensity projection method was adopted by reducing the input projection intensity step by step and reconstructing the remaining pixels around the specular angle. Finally, the reconstruction result can be obtained by stitching point clouds at each projection intensity. Experiments verified that the proposed method could improve the integrity of reconstructed point clouds and measurement efficiency.

A Low-Cost 3-D Imaging Device Using 2-D LiDAR and Reflectors

Laser techniques are widely used to perform topographic scenario surveys. However, in the absence of calibration in the Two-Dimensional (2-D)-based low-cost Three-Dimensional (3-D) scanning devices, a nonnegligible distortion between the ground truth and the reconstructed point cloud exists because of the imperfect structure of the 3-D scanning devices. Meanwhile, the real-time imaging performance is poor and insufficient to capture the shape of moving targets. Therefore, a simple but efficient 3-D scanning structure is developed to decrease the distortion. For real-time processing, the backward emitted laser beams are reflected by the reflectors and the then the reflected beams are available to detect the objects in front of the LiDAR. In the direction where objects of interest exist, the scanning frequency can be multiplied. Through a careful theoretical analysis, the spatial relationship between the points of the 2-D LiDAR, the placement of reflectors, and the 3-D objects is deduced. A solid experiment is performed in which several 3-D scanning devices are compared. It verifies the superiority of the proposed device in reducing the distortion and feasibility to alleviate the real-time processing problem.

3D-CDRNet: Retrieval-based dense point cloud reconstruction from a single image under complex background

3D reconstruction technique based on deep learning is gaining increasing attention from researchers. The majority of current 3D reconstruction techniques require a simple background, which limit their applications on complex background image. Extracting point cloud features comprehensively is also extremely difficult. This paper design a novel 3D reconstruction network to overcome these limitations. Firstly, we get the image and the retrieved point cloud that is the most similar to the input image. Secondly, to learn the features of the retrieved point cloud, the network encodes and decodes the single image and the retrieved point cloud to generate sparse point cloud. Finally, the proposed dense module densifies the sparse point cloud into the dense point cloud. We use single image of complex background and public dataset to evaluate our network. The reconstruction results indicate that the network surpasses previous reconstruction networks.

Local-to-global structure prior guided high-precision point cloud registration framework based on FPP

As one of the popular point cloud reconstruction technologies, fringe projection profilometry (FPP) has great application prospects in point cloud registration from different viewing angles. While the existing point cloud registration algorithm that focuses on FPP is limited to high-speed fringe projection system. Besides, the deep learning methods based on the global feature retrieval mechanism can hardly mine the multi-modal data features of FPP, and are difficult to meet the accuracy requirements of FPP. To address the above issue, we design a local–global structure prior guided high-precision registration framework that focuses on the FPP data characteristics. Specifically, the consistent clustering matching (CCM) module is proposed firstly to obtain the structure priors of cluster correspondences in the overlap regions by analyzing the FPP multi-modal data. Then the local-cluster interaction network (LCINet) guided by the structure priors is introduced for feature extraction and interaction. Finally, the spatial alignment module based on cluster voting(SACV) is proposed to select cluster correspondences with high confidence to calculate the transformation matrix and align point clouds. Experiments show that our method achieves state-of-the-art performance on the dataset collected by FPP, and our framework can also greatly improve the training efficiency and performance of other deep learning models.

SE-MD: a single-encoder multiple-decoder deep network for point cloud reconstruction from 2D images

SE-MD: a single-encoder multiple-decoder deep network for point cloud reconstruction from 2D images

A novel method for dense point cloud reconstruction and weld seam detection for tubesheet welding robot

Welding is not yet fully automated, which is mainly limited by the manual drawing of the welding model. A novel method of automatically establishing a dense point cloud model of the tubesheet and detecting the weld seam was proposed to solve the issue in this paper. The multi-sensor system was calibrated with a fast calibration method, and a laser filter algorithm was then applied to fuse the multi-sensor data. The vocabulary tree method was carried out for tubesheet image similarity analysis, followed by the Random Sample Consensus (RANSAC) algorithm used for feature matching, aiming to establish a point cloud model based on epipolar constraint, EPNP and PMVS algorithm. Finally, a weld seam detection algorithm based on voxel point cloud density was proposed to detect the weld seam in the point cloud. After comparison, the reconstruction method has better robustness than the reference. The point cloud measurement results showed that the average row and column length errors of the reconstructed point cloud were both less than 1 %, which can meet the requirements in current welding applications. And the proposed weld seam detection method can reduce the detection error rate from 20.83 % to 9.03 %.