High-precision 3D Reconstruction Research Articles

Three-dimensional reconstruction method for layered structures based on a frequency modulated continuous wave terahertz radar

The feasibility of employing a continuous-wave terahertz detection system for non-contact and non-destructive testing (NDT) in multi-layered bonding structures is assessed in this study. The paper introduces the detection principle of terahertz frequency modulated continuous wave (FMCW) radar and outlines the two-dimensional (2D) scanning platform, which integrates optical lenses, three linear actuators, a control platform, and data acquisition units. Experimental results on two types of insulation with prefabricated defects demonstrate the capability of terahertz waves for transparent inspection imaging. These results confirm the viability of terahertz FMCW detection technology as an advanced NDT tool for multi-layered bonding structures. However, the inherent limitations of terahertz wavelength and hardware systems pose challenges in discriminating reflection peaks on upper and lower surfaces. To address this issue, a local adaptive empirical wavelet coefficient modal decomposition (LAEWCMD) method is proposed to enhance the longitudinal discrimination ability of terahertz detection. The proposed method involves segmenting the 2D terahertz detection image into regions to differentiate between defective and non-defective areas. Continuous wavelet transforms (CWT) are then applied to the range signals of each region to derive continuous wavelet coefficients (CWCs). Subsequently, empirical mode decomposition (EMD) is performed on the CWCs to decompose them into intrinsic mode functions (IMFs) and residual signals. The 1st IMF is utilized for three-dimensional (3D) reconstruction, and the regions are fused to generate the final output. The effectiveness of the proposed method is validated on aircraft thermal protection structures (TPS), achieving high-precision 3D reconstruction. This offers a novel approach for the application of terahertz computed tomography imaging and NDT.

High-Precision 3D reconstruction and quantitative structure description: Linking microstructure to macroscopic heat transfer of aerogels

The intricate interplay between microstructure and macroscopic transfer processes in porous materials has long been recognized, yet achieving a definitive link between them remains a formidable challenge. Herein, we propose an effective approach to address this challenge by introducing a high-precision 3D reconstruction and structure descriptor database analysis method of nanoporous aerogels, which enables deeper understanding and accurate predictions of structure-directed properties. By employing focused ion beam-scanning electron microscope (FIB-SEM) technique supplemented with image preprocessing and image segmentation, we successfully built a 3D reconstruction model that exhibits high consistency with experimental results. Then, a structure descriptor database from the view of original porous structure, structure connectivity, and topological information is established, and 10 descriptors are preliminarily filtered relevance with thermal conductivity. Consequently, the quantitative relationship between descriptor and thermal conductivity is evaluated via a back propagation (BP) neural network, and three categories structure descriptors which have the most significant impact are determined as (i) the description of gaseous heat transfer pathways, (ii) the description of three-dimensional connectivity, and (iii) the description of interface complexity. With the profound understanding of structure descriptor, the present work offers a novel, structure-focused strategy that closely link the microstructure to macroscopic properties of porous materials.

Three-dimentional reconstruction of underwater side-scan sonar images based on shape-from-shading and monocular depth fusion

Modern marine research requires high-precision three-dimensional (3D) underwater data. Underwater environments experience severe visible light attenuation, which leads to inferior imaging compared with air. In contrast, sound waves are less affected underwater; hence side-scan sonar is used for underwater 3D reconstruction. Typically, the shape-from-shading algorithm (SfS) is widely used to reconstruct surface normal or heights from side-scan sonar images. However, this approach has challenges because of global information loss and noise. To address these issues, this study introduces a surface-normal fusion method. Specifically, we propose a frequency separation SfS algorithm using a discrete cosine transform, which provides a surface-normal map with less noise. We then fuse the surface-normal map with a novel depth estimation network to achieve high-precision 3D reconstruction of underwater side-scan sonar images. We conducted experiments on synthetic, NYU-depth-v2, and real side-scan sonar datasets to demonstrate the effectiveness of the proposed method.

Disparity Refinement for Stereo Matching of High-Resolution Remote Sensing Images Based on GIS Data

With the emergence of the Smart City concept, the rapid advancement of urban three-dimensional (3D) reconstruction becomes imperative. While current developments in the field of 3D reconstruction have enabled the generation of 3D products such as Digital Surface Models (DSM), challenges persist in accurately reconstructing shadows, handling occlusions, and addressing low-texture areas in very-high-resolution remote sensing images. These challenges often lead to difficulties in calculating satisfactory disparity maps using existing stereo matching methods, thereby reducing the accuracy of 3D reconstruction. This issue is particularly pronounced in urban scenes, which contain numerous super high-rise and densely distributed buildings, resulting in large disparity values and occluded regions in stereo image pairs, and further leading to a large number of mismatched points in the obtained disparity map. In response to these challenges, this paper proposes a method to refine the disparity in urban scenes based on open-source GIS data. First, we register the GIS data with the epipolar-rectified images since there always exists unignorable geolocation errors between them. Specifically, buildings with different heights present different offsets in GIS data registering; thus, we perform multi-modal matching for each building and merge them into the final building mask. Subsequently, a two-layer optimization process is applied to the initial disparity map based on the building mask, encompassing both global and local optimization. Finally, we perform a post-correction on the building facades to obtain the final refined disparity map that can be employed for high-precision 3D reconstruction. Experimental results on SuperView-1, GaoFen-7, and GeoEye satellite images show that the proposed method has the ability to correct the occluded and mismatched areas in the initial disparity map generated by both hand-crafted and deep-learning stereo matching methods. The DSM generated by the refined disparity reduces the average height error from 2.2 m to 1.6 m, which demonstrates superior performance compared with other disparity refinement methods. Furthermore, the proposed method is able to improve the integrity of the target structure and present steeper building facades and complete roofs, which are conducive to subsequent 3D model generation.

Accurate and semantic 3D reconstruction of maize leaves

The rapid advances in equipment and technology for crop three-dimensional (3D) data acquisition have enabled the high-throughput and high-quality acquisition of crop 3D data. However, the challenge now lies in comprehending unstructured crop 3D data and extracting phenotypes and knowledge. Extracting semantic feature points from plant leaves and achieving high-precision 3D reconstruction is a prevalent and formidable issue in this field. In this study, we present an accurate method for semantic 3D reconstruction of maize leaves. The algorithm utilizes 3D point cloud data of maize leaves as the initial input. Subsequently, the semantic 3D reconstruction of maize leaves is achieved by means of point cloud preprocessing, initial triangular meshing, hole filling, remeshing, mesh parameterization, two-dimensional (2D) edge detection, 2D semantic point interpolation, 3D semantic point selection, mesh generation, and optimization of feature-point position. A total of 56 leaves from five different maize cultivars were reconstructed utilizing this method. The results indicate that the average distance error index dCDMP between the reconstructed mesh vertices and the input point cloud is less than 0.3 cm2 for all five cultivars, and the mean of the maximum distance error index dHDMP for each cultivar is within the range of 0.5–0.8 cm, indicating it is high-accuracy. The visualization of leaves shows good consistency between the reconstructed mesh and corresponding point cloud. Additionally, the reconstructed 3D mesh of the leaves exhibits enhanced triangular facet consistency. Each vertex can be precisely identified as a leaf vein point, leaf edge point, leaf tip point, or leaf surface point, and includes semantic data such as the position on the leaf blade, which can be directly employed for phenotype estimations. High-precision reconstruction of 3D models of crop organs that include semantic information and cultivar resolution yields significant benefits. This technology can improve the resolution and accuracy of crop morphological and structural research, and it is also crucial for crop phenomics, functional-structural plant modelling, and fusion research between plant phenotyping and crop models.

Retracted: High-Precision 3D Reconstruction of Cooperative Markers under Motion Blur

Retracted: High-Precision 3D Reconstruction of Cooperative Markers under Motion Blur

Robot trajectory planning for autonomous 3D reconstruction of cockpit in aircraft final assembly testing

The trend towards automation and intelligence in aircraft final assembly testing has led to a new demand for autonomous perception of unknown cockpit operation scenes in robotic collaborative airborne system testing. To address this demand, a robotic automated 3D reconstruction cell which enables to autonomously plan the robot end-camera's trajectory is developed for image acquisition and 3D modeling of the cockpit operation scene. A continuous viewpoint path planning algorithm is proposed that incorporates both 3D reconstruction quality and robot path quality into optimization process. Smoothness metrics for viewpoint position paths and orientation paths are introduced together for the first time in 3D reconstruction. To ensure safe and effective movement, two spatial constraints, Domain of View Admissible Position (DVAP) and Domain of View Admissible Orientation (DVAO), are implemented to account for robot reachability and collision avoidance. By using diffeomorphism mapping, the orientation path is transformed into 3D, consistent with the position path. Both orientation and position paths can be optimized in a unified framework to maximize the gain of reconstruction quality and path smoothness within DVAP and DVAO. The reconstruction cell is capable of automatic data acquisition and fine scene modeling, using the generated robot C-space trajectory. Simulation and physical scene experiments have confirmed the effectiveness of the proposed method to achieve high-precision 3D reconstruction while optimizing robot motion quality.

A novel method for 3D reconstruction in tight and restricted space based on improved line-structured light

Three-dimensional (3D) reconstruction techniques have been widely employed in many fields. But precise endoscopic 3D measurement inside of objects, especially in tight and restricted spaces, is still a challenge since there is no easy access for measurement tools. Endoscope with 2D-view function has been widely used. However, the precise measurement technology of the inner surface of small hollow objects has not been widely researched. The components of the existing endoscope three-dimensional measuring probe are complex, expensive and not performed well in accuracy. Thus, a novel method and a miniaturized endoscopic 3D sensor is introduced. To be suit in tight and restricted space, the sensor probe prototype has a maximum 6 mm in thickness. The miniatured sensor prototype is mainly based on improved line-structured light, which performs better in robustness and measurement accuracy than coded projection. Especially, the components are inexpensive and easily available, which reduces the manufacturing cost thus making it easily to popularize in industry. Correspondingly, to ensure the sensor measurement accuracy, we also proposed a calibration method for a miniature line-structured light system based on specially designed planar targets. It should be noted that the 3D extension of the line-structured light system relies on the movement of the translation stage. Taking full account of the characteristics of the sensor prototype, a high-precision direction calibration method of the translation stage based on joint multi-calibration position noise estimation is proposed. Finally, the accuracy experiment is carried out on the gauge block to verify that the measurement method proposed can achieve high-precision 3D reconstruction in a restricted space, and that the 3D reconstruction error is less than 10µm.

Flexible and fast calibration method for uni-directional multi-line structured light system

Multi-line Shift Profilometry (MLSP) is widely used to measure the high-reflectivity object by projecting high-contrast binary stripe image. However, in the calibration of a MLSP system, the lens distortion of the projector makes the most existing calibration methods fail in obtaining high accurate result. Therefore, to avoid the influence of lens distortion, we propose a novel flexible and fast calibration method for MLSP, which requiring only uni-directional patterns. In this method, for each bending light plane, lens distortion is avoided by establishing a polynomial model between three-dimensional (3D) coordinates and camera sub-pixel coordinates. Because the position of calibration planar targets is placed freely without any constraint, this method is flexible and convenient to operate. Experimental results demonstrate that the proposed method achieves high-precision 3D reconstruction.

High-precision 3D reconstruction of multiple magnetic targets based on center weighting method

A high-precision three-dimensional reconstruction method of multiple magnetic targets is studied in this paper. The inversion space is divided into cubic grid cells, and various complex shapes of magnetic targets are simulated by the combination of grid cells. The forward calculation of the magnetic gradient tensor in the spatial domain is performed. Then, the correlation imaging method is used to calculate cross correlation between the observed magnetic gradient tensor and the theoretical magnetic gradient at each grid cell tensor due to a magnetic dipole. The spatial distribution and center position of magnetic targets are preliminarily estimated by the resultant correlation coefficients, and use them to construct the center weighting function as a constraint to the inversion equations. The solution is completed by using the conjugate gradient method to estimate the magnetic susceptibility of each grid cell, and the 3D outline of the magnetic target is depicted by using the visualization technique. The experimental results show that the proposed method can achieve the 3D reconstruction of multiple simple magnetic targets with high precision.

Motion Error Estimation and Compensation of Airborne Array Flexible SAR Based on Multi-Channel Interferometric Phase

Airborne array synthetic aperture radar (SAR) has made a significant breakthrough in the three-dimensional resolution of traditional SAR. In the airborne array SAR 3D imaging technology, the baseline length is the main factor restricting the resolution. Airborne array flexible SAR can increase the baseline length to improve the resolution and interference performance by mounting antennae on the wing. The existing research lacks results obtained using flexible actual data processing and specific motion compensation methods. Thus, this paper proposes a motion error estimation and compensation method for an airborne array flexible SAR based on a multi-channel interferometric phase. Firstly, a flexible channel motion compensation model is established based on the multi-channel interference phase of airborne array flexible SAR. Then, based on the rigid multi-channel data, combined with the ground control points, the least square method, and the global optimal search algorithm, the accurate rigid baseline length and the central incidence angle are obtained. Finally, according to the multi-channel interference phase inversion of the flexible motion error and combined with the motion compensation model, the flexible data are compensated in the time domain. The actual results indicate that, compared with traditional motion compensation methods, our method can obtain accurate flexible compensation data. This study improves the interference performance of multi-channel data of airborne array flexible SAR and lays a solid foundation for the high-precision 3D reconstruction of airborne array flexible SAR.

Binocular Structured Light 3-D Reconstruction System for Low-Light Underwater Environments: Design, Modeling, and Laser-Based Calibration

The three dimensional (3D) perception of the low-light underwater environment has always been a major challenge which greatly limits the underwater operations. In this paper, an underwater active vision measurement system based on binocular structured light is designed to achieve high precision 3D reconstruction. Firstly, the fusion technology of binocular camera and laser addresses underwater optical attenuation and feature sparsity. Then, in order to avoid the huge inertia caused by the overall movement of the system, a laser scanner based on the mirror-galvanometer is used to accomplish static scanning of the scene. Subsequently, considering the influence of multiple media, underwater refraction models including monocular imaging model, binocular ranging model, and binocular polar curve constraint model are systematically proposed. What’s more, the conventional checkerboard-based passive visual calibration method is ineffective for low-light waters. Therefore, a simple calibration block is designed, and a new multi-objective laser-based calibration algorithm based on laser geometric constraints is proposed. Finally, the effectiveness of the system is verified by analyzing the 3D reconstruction results of underwater objects.

An efficient 3D extraction and reconstruction method for myelinated axons of mouse cerebral cortex based on mixed intelligence

Accurate reconstruction of the 3D morphology and spatial distribution of myelinated axons in mouse brains is very important for understanding the mechanism and dynamic behavior of long-distance information transmission between brain regions. However, it is difficult to segment and reconstruct myelinated axons automatically due to two reasons: the amount of it is huge and the morphology of it is different between brain regions. Traditional artificial labeling methods usually require a large amount of manpower to label each myelinated axon slice by slice, which greatly hinders the development of the mouse brain connectome. In order to solve this problem and improve the reconstruction efficiency, this paper proposes an annotation generation method that takes the myelinated axon as prior knowledge, which can greatly reduce the manual labeling time while reaching the level of manual labeling. This method consists of three steps. Firstly, the 3D axis equation of myelinated axons is established by sparse axon artificial center point labels on slices, and the region to be segmented is pre-extracted according to the 3D axis. Subsequently, the U-Net network was trained by a small number of artificially labeled myelinated axons and was used for precise segmentation of output by the last step, so as to obtain accurate axon 2D morphology. Finally, based on the segmentation results, the high-precision 3D reconstruction of axons was performed by imaris software, and the spatial distribution of myelinated axons in the mouse brain was reconstructed. In this paper, the effectiveness of this method was verified on the dataset of high-resolution X-ray microtomography of the mouse cortex. Experiments show that this method can achieve an average MIoU 81.57, and the efficiency can be improved by more than 1400x compared with the manual labeling method.

Single frame color fringe projection profilometry based on empirical mode decomposition imbalance correction algorithm for fast three-dimensional reconstruction

Phase-shifting profilometry (PSP) is considered to be the most accurate technique for phase retrieval with fringe projection profilometry (FPP) systems. However, PSP requires that multiple phase-shifted fringe patterns be acquired, usually sequentially, which limits PSP to static or quasistatic imaging. We introduce color FPP (CFPP), which provides three-dimensional (3D) imaging using a single acquisition, for the application of real-time profiling. A single frame acquisition provides all three phase-shifted fringe patterns needed for the PSP phase retrieval algorithm. However, some problems must be overcome in CFPP, namely, color coupling and imbalance. Following up on the study, we propose a fixed coefficient decoupling method based on point-to-point correction and then introduce the empirical mode decomposition algorithm to decompose a color fringe pattern into three high-frequency components and three low-frequency components for imbalance correction, which improves measurement accuracy and realizes a single frame measurement for complex and discontinuous objects. After correction by the proposed method, the shape of the tested object can be recovered by one a single frame color fringe pattern, which meets the need for dynamic measurement, and the accuracy is not limited by the movement speed of object. The simulations and experimental results show that the proposed method can significantly reduce the influence of color coupling and imbalance in CFPP, and thus realize dynamic and high-precision 3D reconstruction.

3D reconstruction method for tree seedlings based on point cloud self-registration

The 3D reconstruction of tree seedlings can help to assess phenotypic architectures, conceive virtual urban landscapes and design computer games. The existing multicamera photograph technology already has the capability to accurately reconstruct 3D models for small scene plants, such as corn and vegetable seedlings. However, the existing plant 3D reconstruction system has several shortcomings, such as its high cost, complicated operation procedure, and unsuitability for seedling trees. Therefore, this paper proposes an autonomous alignment method for seedling point clouds that can realize the low-cost and fast 3D reconstruction of batch seedlings. In this study, we designed a system based on a low-cost Kinect camera and a precision turntable to construct 3D seedling models. A special turntable was adopted to achieve self-registration for the seedling point clouds. It was efficient for us to obtain several 3D seedlings models with only one registration. The system could capture images automatically from different viewpoints and submit these images to a graphic workstation for processing. In our work, we set three fixed views, V2, V3 and V4, to evaluate the cumulative errors caused by multiview matching. It needn’t touch any parts of the seedings to create 3D models at different view by the proposed method. Herein, the large proportions of 0 < mean absolute distance, MD ≤ 0.6 cm and 0 < standard deviation, SD ≤ 0.4 cm, between the reference and the reconstructed point cloud showed that the 3D reconstruction method was accurate, stable and flexible. Additionally, we validated the phenotypic structure measurement, and the height H was highly accurate (R2 > 0.985) when using the 3D reconstruction models of seedlings. Experiments demonstrate that the proposed method has the potential to obtain high-precision 3D reconstruction models and phenotypic parameters for seedlings via low-cost equipment with high-efficiency processing algorithms.

EVALUATION OF STRUCTURAL FND FUNCTIONAL FEATURES OF REIMPLANTED AORTIC VALVE AFTER DAVID I PROCEDURE WITH THE USE OF HIGH-PRECISSION 3-DIMENSIONAL SEGMENTATION OF THE AORTIC ROOT

Aneurysm of the aortic root accompanied by aortic insufficiency is a common cardiovascular pathology. The optimal method of surgical treatment of this pathology is the reimplantation of the aortic valve into the aortic root synthetic prosthesis (David procedure). Currently, there is no comprehensive data on the influence of initial geometric parameters of the aortic root structures on the functioning of the reimplanted aortic valve after valve-preserving aortic root replacement. We have introduced a novel technique for creation and segmentation of a three-dimensional reconstruction of the aortic valve and root, which involves the sequential application of two computer-aided design environments: In Vesalius v 3.1.1 (Centro de Tecnologia da Informação Renato Archer (CTI)) and 3- Matic v. 13.0 (Materialise, Leuven, Belgium). The proposed method allows accurate measurement and evaluation of spatial parameters of the AV leaflets, fibrous ring, sinuses of Valsalva and sinotubular junction. The developed technique was used to evaluate the anatomical and functional features of the aortic valve in a 68-year-old patient who underwent David I procedure. Segmentation and measurement of the geometric parameters of the aortic root were performed before operation, as well as 18 months after surgical treatment. To assess the function of the reimplanted aortic valve during the long-term follow-up, a detailed echocardiographic assessment was used. Echocardiographic assessment included measuring volume and area of the regurgitation flow, the width of vena contracta, and the regurgitation fraction. The application of high-precision 3D reconstruction opens up additional opportunities in the field of surgical treatment of the aortic root and AV pathology: advantages in choosing an individual patient-oriented surgical tactics, parameters of the implanted surgical material and/or device, and in predicting long-term results of surgical treatment.

Single-shot high-precision 3D reconstruction with color fringe projection profilometry based BP neural network

In order to reduce the influence of nonlinear factors such as gamma nonlinearity and color crosstalk on the phase retrieval accuracy with color fringe projection profilometry (CFPP) in rapid measurement, a single-shot high-precision backpropagation (BP) neural network based CFPP is proposed. Firstly, neural networks are trained from samples through a training process. The neural network inputs are the RGB components of composite color fringe patterns. The numerator and denominator of the arctangent function are used as network outputs. The monochromatic fringe patterns are adopted. The numerator and denominator of the arctangent function are calculated by the 15-step phase-shift method, which serves as the network training target. Applying this neural network to the prediction of the standard phase can reduce the influence of nonlinear factors on the phase accuracy. Then, the absolute phase was unwrapped by using geometric constraints. Finally, the 3D reconstruction with high precision can be realized by using only a single-shot color fringe. The theoretical model of the proposed method and the process of neural network training are introduced. The results of 3D reconstruction are compared with other algorithms in the same condition. The experimental results demonstrate that the RMS of the proposed algorithm can reach 0.0107 mm, compared with 0.0121 mm of the traditional 3-step phase-shift method. The proposed method is applied to the online quality inspection of relay components, and the results verify the high accuracy and high efficiency of measurement.

Composite fringe projection deep learning profilometry for single-shot absolute 3D shape measurement.

Single-shot fringe projection profilometry (FPP) is essential for retrieving the absolute depth information of the objects in high-speed dynamic scenes. High-precision 3D reconstruction using only one single pattern has become the ultimate goal in FPP. The frequency-multiplexing (FM) method is a promising strategy for realizing single-shot absolute 3D measurement by compounding multi-frequency fringe information for phase unwrapping. In order to solve the problem of serious spectrum aliasing caused by multiplexing schemes that cannot be removed by traditional spectrum analysis algorithms, we apply deep learning to frequency multiplexing composite fringe projection and propose a composite fringe projection deep learning profilometry (CDLP). By combining physical model and data-driven approaches, we demonstrate that the model generated by training an improved deep convolutional neural network can directly perform high-precision and unambiguous phase retrieval on a single-shot spatial frequency multiplexing composite fringe image. Experiments on both static and dynamic scenes demonstrate that our method can retrieve robust and unambiguous phases information while avoiding spectrum aliasing and reconstruct high-quality absolute 3D surfaces of objects only by projecting a single composite fringe image.

High-precision 3D reconstruction method for topography measurement of complex mechanical parts

High-precision 3D reconstruction method for topography measurement of complex mechanical parts

High-Precision 3D Reconstruction for Small-to-Medium-Sized Objects Utilizing Line-Structured Light Scanning: A Review

Three-dimensional reconstruction technology has demonstrated broad application potential in the industrial, construction, medical, forestry, agricultural, and pastural sectors in the last few years. High-quality digital point cloud information exists to help researchers to understand objects and environments. However, current research mainly focuses on making adaptive adjustments to various scenarios and related issues in the application of this technology rather than looking for further improvements and enhancements based on technical principles. Meanwhile, a review of approaches, algorithms, and techniques for high-precision 3D reconstruction utilizing line-structured light scanning, which is analyzed from a deeper perspective of elementary details, is lacking. This paper takes the technological path as the logical sequence to provide a detailed summary of the latest development status of each key technology, which will serve potential users and new researchers in this field. The focus is placed on exploring studies reconstructing small-to-medium-sized objects, as opposed to performing large-scale reconstructions in the field.