3D Point Cloud Reconstruction Research Articles

Three-Dimensional X-Ray Computed Tomography Image Segmentation and Point Cloud Reconstruction for Internal Defect Identification in Laser Powder Bed Fused Parts

Abstract Defects shape, volume, and orientation all have a direct impact on the mechanical properties of Laser Powder Bed Fused (L-PBF-ed) parts. Therefore, it is necessary to evaluate and analyze the three-dimensional (3D) geometrical characteristics of these defects. X-ray Computed Tomography (XCT) can reveal an object's internal structure by volumetric scanning through its building direction. Point clouds are 3D data that can be extracted from the stack of XCT images taken from a part to perform further analysis. This study presents a novel approach for 3D segmentation and geometrical analysis of L-PBF defect structures from XCT images. The proposed method integrates Voronoi labeling and 3D point cloud reconstruction to reveal individual defect characteristics from the XCT image stack of a part. A case study showed the proposed methodology's effectiveness in identifying and characterizing defect regions in L-PBF-ed Cobalt-Chrome (CoCr) parts.

Enhancing images for vision measurement in hazy tunnel construction

To address the negative impacts of image degradation on vision measurement in hazy tunnel construction, this paper presents a unified enhancement method of color and grayscale images in tunnel haze environments. Firstly, a tunnel haze environment light scattering model is introduced utilizing geometric optics principles and the conventional atmospheric scattering model. Secondly, a model for perceiving image haze is established within the image feature space to estimate ambient light and transmission. The efficacy of the method is assessed by constructing a tunnel haze simulation chamber and a platform for measurement of tunnel boring machine (TBM) cutter wear. Comparative experiments demonstrate the method reduces the error in 3D point cloud reconstruction by 77% and improves the measurement precision of TBM cutter wear by 62%. The results demonstrate the potential application of this method in automated tunnel construction. However, further research is needed to enhance its efficacy in dense haze images.

Enhancing Building Point Cloud Reconstruction from RGB UAV Data with Machine-Learning-Based Image Translation.

The performance of three-dimensional (3D) point cloud reconstruction is affected by dynamic features such as vegetation. Vegetation can be detected by near-infrared (NIR)-based indices; however, the sensors providing multispectral data are resource intensive. To address this issue, this study proposes a two-stage framework to firstly improve the performance of the 3D point cloud generation of buildings with a two-view SfM algorithm, and secondly, reduce noise caused by vegetation. The proposed framework can also overcome the lack of near-infrared data when identifying vegetation areas for reducing interferences in the SfM process. The first stage includes cross-sensor training, model selection and the evaluation of image-to-image RGB to color infrared (CIR) translation with Generative Adversarial Networks (GANs). The second stage includes feature detection with multiple feature detector operators, feature removal with respect to the NDVI-based vegetation classification, masking, matching, pose estimation and triangulation to generate sparse 3D point clouds. The materials utilized in both stages are a publicly available RGB-NIR dataset, and satellite and UAV imagery. The experimental results indicate that the cross-sensor and category-wise validation achieves an accuracy of 0.9466 and 0.9024, with a kappa coefficient of 0.8932 and 0.9110, respectively. The histogram-based evaluation demonstrates that the predicted NIR band is consistent with the original NIR data of the satellite test dataset. Finally, the test on the UAV RGB and artificially generated NIR with a segmentation-driven two-view SfM proves that the proposed framework can effectively translate RGB to CIR for NDVI calculation. Further, the artificially generated NDVI is able to segment and classify vegetation. As a result, the generated point cloud is less noisy, and the 3D model is enhanced.

Development of a 3D point cloud reconstruction-based apple canopy liquid sedimentation model

China leads the world in apple production, contributing to over half of the global total. Annually, extensive pesticide use is employed in apple cultivation, but the uncertain quantities of pesticide deposition and indiscriminate application have resulted in significant pesticide loss and residue. This poses a considerable potential risk to human health. This paper focuses on ground-based plant protection and stratifies the canopy horizontally, using 3D point cloud reconstruction technology and tree deposition experiments to explore the relationship between canopy leaf area index and pesticide deposition. By calculating the droplet penetration coefficient based on the amount of solution deposited at various canopy layers, a functional model linking leaf area index and droplet penetration coefficient has been established, with a correlation coefficient (R2) of 0.92. Based on this model, and by integrating the canopy volume with the amount of solution deposited on the outermost layer of the canopy, the total solution deposition on the entire tree can be estimated. According to the deposition model, there is a negative correlation between the amount of solution deposited on the canopy and the leaf area index of the canopy. Moreover, the experiments have shown that when the spray vehicle travels at a speed of 1.5 m/s, the maximum amount of solution is deposited while still ensuring an effective deposition rate. The establishment of the deposition estimation model allows for the visualization of canopy solution deposition and can guide fruit farmers in determining the appropriate frequency of spraying based on the severity of pest and disease infestations.

Digital Restoration of Historical Buildings by Integrating 3D PC Reconstruction and GAN Algorithm

Historical architecture is an important carrier of cultural and historical heritage in a country and region, and its protection and restoration work plays a crucial role in the inheritance of cultural heritage. However, the damage and destruction of buildings urgently need to be repaired due to the ancient age of historical buildings and the influence of natural environment and human factors. Therefore, an artificial intelligence repair technology based on three-dimensional (3D) point cloud reconstruction and generative adversarial networks was proposed to improve the precision and efficiency of repair work. Firstly, in-depth research on the principles and algorithms of 3D point cloud data processing and generative adversarial networks should be conducted. Secondly, a digital restoration framework was constructed by combining these two artificial intelligence technologies to achieve precise and efficient restoration of historical buildings through continuous adversarial learning processes. The experimental results showed that the errors in the restoration of palace buildings, defense walls, pagodas, altars, temples, and mausoleums were 0.17, 0.12, 0.13, 0.11, and 0.09, respectively. The technique can significantly reduce the error while maintaining the high precision repair effect. This technology with artificial intelligence as the core has excellent accuracy and stability in the digital restoration. It provides a new technical means for the digital restoration of historical buildings and has important practical significance for the protection of cultural heritage.

Limited environmental information path planning based on 3D point cloud reconstruction

Limited environmental information path planning based on 3D point cloud reconstruction

RESEARCH ON 3D MEASUREMENT TECHNOLOGY BASED ON MACHINE VISION

Measurement is an important means for humans to understand and transform the world, and it is the technological foundation for breaking through the forefront of science and solving major problems in economic and social development. The three-dimensional measurement system serves primary national needs, leads national economic development, and ensures national defense security. With the arrival of a new industrial revolution, major industrial countries worldwide have begun to accelerate the strategic deployment of intelligent manufacturing. As an essential component of the construction of smart factories and lighthouse factories, the 3D measurement system will play a vital role in deepening the implementation of the manufacturing power strategy and promoting the high-end, intelligent, and green manufacturing process. In recent years, optical 3D measurement technology represented by surface structured light has developed rapidly and has been widely applied in multiple material processing fields such as forging, casting, and sheet metal. To analyze the manufacturing accuracy of complex parts using surface structured light 3D measurement technology, it is necessary to first scan and reconstruct the overall 3D surface of the part. The reconstructed 3D point cloud data of the part surface should be roughly and precisely matched with the design model. Finally, based on this, data comparison and accuracy analysis should be carried out according to the actual detection requirements of different types of parts. In this process, the accuracy of 3D reconstruction of complex parts and the alignment accuracy between the reconstruction results and the design model directly determine the reliability and accuracy of the final part manufacturing and machining accuracy analysis. This article uses cases to illustrate the application of machine vision in 3D online measurement. The application of three-dimensional measurement technology in online quality inspection and grinding of blades can not only improve the measurement accuracy of online quality inspection of blades but also maintain the relative stability of robot grinding force and improve the grinding effect in robot grinding of aircraft engine blades. Applied to online quality inspection of train wheel hubs, three-dimensional measurement of train wheel adapters and high-speed train wheel size online inspection has been achieved.

Robust structured light 3D measurement method based on polarization-encoded projection patterns.

Fringe projection profilometry (FPP) is widely used in 3D vision measurement because of its high robustness and measurement accuracy. In the case of HDR objects, due to the problem of surface reflectivity, the obtained image will be overexposed. This will cause the sinusoidality of the fringes projected on the surface of the object in the acquired image to be interfered, resulting in a phase error in the calculated wrapped phase. Therefore, a polarization-encoded sinusoidal structured light is proposed to enhance the sinusoidality of the fringe. The phase information contained in the polarized sinusoidal structured light fringe is only related to the polarization state, not to the light intensity. A polarization coding assisted structured light measurement strategy (PASM) is proposed. This method uses polarization coding assisted polarization phase-shifting fringes for phase unwrapping. The angle of the linear polarizer is set to zero in this method, and it does not require rotating the polarizer. It only needs a single exposure to improve the fringe quality and obtain a more stable unwrapping phase. The experimental results show that the obtained polarization fringes have better sinusoidality, and the phase unwrapping can be more accurate. The reconstructed 3D point cloud also does not appear missing and has better accuracy. It is a reliable method for vision measurement of HDR objects.

Three-Dimensional Surface Reconstruction from Point Clouds Using Euler’s Elastica Regularization

Euler’s elastica energy regularizer, initially employed in mathematical and physical systems, has recently garnered much attention in image processing and computer vision tasks. Due to the non-convexity, non-smoothness, and high order of its derivative, however, the term has yet to be effectively applied in 3D reconstruction. To this day, the industry is still searching for 3D reconstruction systems that are robust, accurate, efficient, and easy to use. While implicit surface reconstruction methods generally demonstrate superior robustness and flexibility, the traditional methods rely on initialization and can easily become trapped in local minima. Some low-order variational models are able to overcome these issues, but they still struggle with the reconstruction of object details. Euler’s elastica term, on the other hand, has been found to share the advantages of both the TV regularization term and the curvature regularization term. In this paper, we aim to address the problems of missing details and complex computation in implicit 3D reconstruction by efficiently using Euler’s elastica term. The main contributions of this article can be outlined in three aspects. Firstly, Euler’s elastica is introduced as a regularization term in 3D point cloud reconstruction. Secondly, a new dual algorithm is devised for the proposed model, significantly improving solution efficiency compared to the commonly used TV model. Lastly, numerical experiments conducted in 2D and 3D demonstrate the remarkable performance of Euler’s elastica in enhancing features of curved surfaces during point cloud reconstruction. The reconstructed point cloud surface adheres more closely to the initial point cloud surface when compared to the classical TV model. However, it is worth noting that Euler’s elastica exhibits a lesser capability in handling local extrema compared to the TV model.

Concrete spalling damage detection and seismic performance evaluation for RC shear walls via 3D reconstruction technique and numerical model updating

Current three-dimensional (3D) reconstruction technique-assisted damage detection research focuses on identifying, classifying, and locating surface damage on concrete components, challenging to quantify the effect of the identified damage on structural capacity. This paper attempts to present a novel method based on the 3D reconstruction technique and numerical model updating to detect concrete spalling damage and evaluate the adverse effects of the detected damage on the seismic performance of reinforced concrete (RC) shear wall components. Through a new concept of information transition point matrix, the mapping relationship between the defective information concealed in the reconstructed 3D point cloud model of the inspected wall and the performance variation of its corresponding finite element model is established. Experimental results demonstrate that the newly proposed method can successfully locate the concrete spalling damage and quantify the bearing capacity variation of the inspected specimen, which has excellent potential for future applications in civil engineering.

A point cloud registration method based on multiple-local-feature matching

A dual-view point cloud registration method has been proposed to enhance the accuracy of point-cloud registration by utilizing multiple local feature descriptors. The method focuses on studying the eigenvalues of the point cloud set at various scales and the impact of descriptors that consist of corresponding curvature radii in reducing misregistration in fast point feature histogram (FPFH). The optimal scale size and number were determined through experimental analysis. SIFT-3D key points of point clouds were extracted to calculate their FPFH and complete initial correspondence matching. Next, a multi-neighborhood local feature descriptor was designed to filter correspondence. A large number of incorrect corresponding point pairs were removed after the initial FPFH matching, and the remaining correct point pairs were used for rough registration. Finally, the improved ICP algorithm was used to reduce registration errors and achieve more accurate dual-view registration. According to the experiments of the public point cloud data set, the dual-view point cloud registration algorithm implemented by the method has good accuracy, high robustness, and more reliable point cloud registration results, which provides a theoretical basis for 3D point-cloud reconstruction.

Photogrammetry-based computational fluid dynamics

Computational fluid dynamics (CFD) is the cornerstone of the design and analysis process in many engineering applications. Not only has it been applied in the design phase, but it has also been employed for analyzing the fluid flow phenomena during the operation phase for many in-use structures, such as vehicles, buildings, and landscapes. However, creating a 3D mesh-based model of in-use structures that can be used by conventional boundary-fitted CFD methods is labor-intensive, time-consuming, and sometimes impossible. Due to the challenges introduced by geometry complexity and lack of design information, it is often difficult to perform an accurate and efficient CFD analysis of these objects. This paper aims to overcome such challenges by proposing a novel photogrammetry-based CFD framework for simulating in-use structures whose design models and analysis meshes are hard to obtain. The proposed framework integrates machine learning-based 3D point cloud reconstruction of structures from 2D images obtained from portable devices (e.g., cell phones and drones) and an immersogeometric approach that can carry out flow analysis directly on reconstructed point clouds. We first present the technical details of point cloud reconstruction techniques and the immersogeometric analysis method. We then simulate the flow past a standard 12 oz soda can reconstructed using photogrammetry and compare the results with reference solutions to assess the accuracy of the approach. Finally, the proposed photogrammetry-based CFD is applied to simulations of a bell tower and the Kavita and Lalit Bahl Smart Bridge on the University of Illinois Urbana-Champaign (UIUC) campus to demonstrate the robustness of the framework and its applicability to real-world in-use civil structures.

Research on 3D Reconstruction of Binocular Vision Based on Thermal Infrared.

Thermal infrared imaging is less affected by lighting conditions and smoke compared to visible light imaging. However, thermal infrared images often have lower resolution and lack rich texture details, making them unsuitable for stereo matching and 3D reconstruction. To enhance the quality of infrared stereo imaging, we propose an advanced stereo matching algorithm. Firstly, the images undergo preprocessing using a non-local mean noise reduction algorithm to remove thermal noise and achieve a smoother result. Subsequently, we perform camera calibration using a custom-made chessboard calibration board and Zhang's camera calibration method to obtain accurate camera parameters. Finally, the disparity map is generated using the SGBM (semi-global block matching) algorithm based on the weighted least squares method, enabling the 3D point cloud reconstruction of the object. The experimental results demonstrate that the proposed algorithm performs well in objects with sufficient thermal contrast and relatively simple scenes. The proposed algorithm reduces the average error value by 10.9 mm and the absolute value of the average error by 1.07% when compared with the traditional SGBM algorithm, resulting in improved stereo matching accuracy for thermal infrared imaging. While ensuring accuracy, our proposed algorithm achieves the stereo reconstruction of the object with a good visual effect, thereby holding high practical value.

Research on Deformation Monitoring of Invert Uplifts in Soft Rock Tunnels Based on 3D Laser Scanning

The soft surrounding rock in tunnels has the characteristics of low strength, easy softening after soaking, and poor self-stability, which makes the inverted arch structure in soft rock tunnels prone to uplift deformations. Therefore, a deformation monitoring method for inverted arch uplifts of soft rock tunnels based on 3D laser scanning technology is studied to improve deformation monitoring. A Leica Scan Station2 3D laser scanner is used to collect 3D point cloud data of soft rock tunnel inverts. Using the automatic matching method of public landmarks and the improved Rodrigues parameter method, the collected 3D point cloud data are spliced and through the Mallat algorithm, the 3D point cloud data are reconstructed and processed after splicing. The whole least square method is used to fit the reconstructed 3D point cloud data. Through the principal component analysis method, the normal vector of the fitting plane is estimated and the best datum plane of the soft rock tunnel invert is found. By calculating and extracting the geometric parameters of the slice point cloud, the monitoring of inverted arch uplift deformations in soft rock tunnels is completed. The experiment shows that this method can effectively collect 3D point cloud data of soft rock tunnel inverts, and complete point cloud stitching and reconstruction. This method can effectively monitor the uplift deformation of inverted arches at different grouting depths.

TIPS: A three-dimensional phenotypic measurement system for individual maize tassel based on TreeQSM

Maize tassel is a crucial pollen producing organ that plays an important role in maize production. It is challenging to investigate the morphological and structural traits of maize tassels for breeding programs, since traditional manual measurements of organ-scale phenotypic traits are labor-intensive and prone to human errors. It is, therefore, urgent to design and develop new phenotyping systems for maize tassels to improve throughput and measurement accuracy. This study first introduced the TreeQSM in crop phenotyping at organ scale and developed TIPS (TreeQSM based Image Phenotyping System for maize tassels). The system mainly consists of three digital cameras for acquiring multi-view images of individual maize tassel. These cameras were vertically arranged with different angles of view. The acquired images were used to reconstruct the 3D point cloud data of individual maize tassel, which were fed into the TreeQSM to extract four tassel phenotypic parameters including trunk length, branch number, branch length, and branch angle. The performance analyses of the developed system were conducted on 52 tassel samples from 37 maize materials with different canopy geometric structures. The experimental results showed that the TIPS could achieve accurate tassel parameters estimation with the R2 of 0.964, 0.973, 0.935, and 0.857, and the RMSE of 1.72, 1.28, 19.94, and 2.73 for the trunk length, branch number, total branch length, and first node branch angle, respectively. This study also investigated the advantages of the data acquisition with the TIPS and the effects of different shooting angles, number of images, lighting conditions and tassel types on the four tassel parameters estimation accuracy. The comparison results indicated that the four influencing factors reduced the estimation accuracy to varying degree. Compared with the other three parameters, the estimation accuracy of branch angles less than 20 were largely affected. The higher the degree of compactness, the worse the estimation accuracy. Compared with shooting angles, the reduction of the number of images in a certain range had less impact on the quality of 3D point cloud reconstruction. The influence of low light was obviously greater than that of strong light. This study provides a valuable guide to the collection and quantitative analysis of high-throughput phenotype information of maize tassels in breeding program.

3D visualization technology for rubber tree forests based on a terrestrial photogrammetry system

IntroductionRubber trees are an important cash crop in Hainan Province; thus, monitoring sample plots of these trees provides important data for determining growth conditions. However, existing monitoring technology and rubber forest sample plot analysis methods are relatively simple and present widespread issues, such as limited monitoring equipment, transportation difficulties, and relatively poor three-dimensional visualization effects in complex environments. These limitations have complicated the development of rubber forest sample plot monitoring.MethodThis study developed a terrestrial photogrammetry system combined with 3D point-cloud reconstruction technology based on the structure from motion with multi-view stereo method and sample plot survey data. Deviation analyses and accuracy evaluations of sample plot information were performed in the study area for trees to explore the practical significance of this method for monitoring rubber forest sample plots. Furthermore, the relationship between the height of the first branch, diameter at breast height (DBH), and rubber tree volume was explored, and a rubber tree standard volume model was established.ResultsThe Bias, relative Bias, RMSE, and RRMSE of the height of the first branch measured by this method were −0.018 m, −0.371%, 0.562 m, and 11.573%, respectively. The Bias, relative Bias, RMSE, and RRMSE of DBH were −0.484 cm, −1.943%, −2.454 cm, and 9.859%, respectively, which proved that the method had high monitoring accuracy and met the monitoring requirements of rubber forest sample plots. The fitting results of rubber tree standard volume model had an R2 value of 0.541, and the estimated values of each parameter were 1.745, 0.115, and 0.714. The standard volume model accurately estimated the volume of rubber trees and forests using the first branch height and DBH.DiscussionThis study proposed an innovative planning scheme for a terrestrial photogrammetry system for 3D visual monitoring of rubber tree forests, thus providing a novel solution to issues observed in current sample plot monitoring practices. In the future, the application of terrestrial photogrammetry systems to monitor other types of forests will be explored.

Quantification of Structural Defects Using Pixel Level Spatial Information from Photogrammetry.

Aging infrastructure has drawn increased attention globally, as its collapse would be destructive economically and socially. Precise quantification of minor defects is essential for identifying issues before structural failure occurs. Most studies measured the dimension of defects at image level, ignoring the third-dimensional information available from close-range photogrammetry. This paper aims to develop an efficient approach to accurately detecting and quantifying minor defects on complicated infrastructures. Pixel sizes of inspection images are estimated using spatial information generated from three-dimensional (3D) point cloud reconstruction. The key contribution of this research is to obtain the actual pixel size within the grided small sections by relating spatial information. To automate the process, deep learning technology is applied to detect and highlight the cracked area at the pixel level. The adopted convolutional neural network (CNN) achieves an F1 score of 0.613 for minor crack extraction. After that, the actual crack dimension can be derived by multiplying the pixel number with the pixel size. Compared with the traditional approach, defects distributed on a complex structure can be estimated with the proposed approach. A pilot case study was conducted on a concrete footpath with cracks distributed on a selected 1500 mm × 1500 mm concrete road section. Overall, 10 out of 88 images are selected for validation; average errors ranging from 0.26 mm to 0.71 mm were achieved for minor cracks under 5 mm, which demonstrates a promising result of the proposed study.

A Novel Filtering Method of 3D Reconstruction Point Cloud from Tomographic SAR

With the development of airborne synthetic aperture radar (SAR) technology, the 3D SAR point cloud reconstruction has emerged as a crucial development trend in the current SAR community. However, due to measurement errors, environmental interference, radar decoherence, and other noises associated with the SAR system, the reconstructed tomogram is often deteriorated by numerous noisy scatterers. As a result, it becomes challenging to obtain high-quality 3D point clouds of the observed object, making it difficult to further process the point cloud and realize target identification. To address these issues, we propose a K nearest neighbor comprehensive weighted filtering algorithm. The filtered point cloud is evaluated quantitatively using three-dimensional entropy. In this study, we adopted various filtering methods for simulated data, P-band data of Genhe, and Ku-band data of Yuncheng to refine the tomogram and compare their performances. Both qualitative and quantitative analyses demonstrate the superiority of the filtering algorithm proposed in this paper.

A pose estimation method of space non-cooperative target based on ORBFPFH SLAM

In this paper, to improve the pose measurement accuracy of the Time-of-Flight (ToF) camera for space non-cooperative targets, a pose estimation method based on ORBFPFH Simultaneous Localization and Mapping (SLAM) with effective integration of strength and depth measurement information is proposed. The primary process of the method is as follows: First, a unique ORBFPFH Bag of Words (BoW) model for space targets is trained based on the ToF image dataset of spacecraft; Secondly, the pose of the non-cooperative space target is tracked based on the ORBFPFH feature and optimized using the pose graph; Then, based on the ORBFPFH BoW model, loop closure is detected to reduce the cumulative error. Finally, the proposed method is tested on the ToF image dataset: compared with the advanced ORB-SLAM2 algorithm, the proposed method has a minor translation error and rotation error on the test data, with a mean translation error less than 0.144 m and mean rotation error less than 0.642°. The test results show that the proposed method can improve the pose estimation accuracy of space non-cooperative targets, achieve a better 3D point cloud reconstruction effect, and provide technical support for space applications such as rendezvous and docking of non-cooperative spacecraft. © 2001 Elsevier Science. All rights reserved

Method of unwrapped phase error correction in phase-measuring profilometry based on adaptive local window

Phase measurement profilometry (PMP) is widely used to reconstruct three-dimensional (3D) shape of objects. However, errors are introduced during standard phase unwrapping process, which considerably brings down measurement accuracy and lowers method reliability. This paper proposes a novel method to identify the errors, and subsequently correct these errors through erecting adaptive local windows. Under proposal, two robust algorithms are developed to correct these errors in initial rudimentary correction as well as in subsequent refined error removal. The adaptive local window is designed as such that the fringe order corresponding to each period of the wrapping phase is independent of each other. Therefore, it avoids the error transmission in the correction process. Experiments are carried out on absolute phase error correction and 3D point cloud reconstruction. Experimental results verified the developed method in terms of reducing the stratification of 3D point cloud and improving the quality of 3D measurement.