High-precision 3D Reconstruction Research Articles

High precision 3D reconstruction and target location based on the fusion of visual features and point cloud registration

High precision 3D reconstruction and target location based on the fusion of visual features and point cloud registration

Quantitative Evaluation of Post-Tillage Soil Structure Based on Close-Range Photogrammetry

Quantitative Evaluation of Post-Tillage Soil Structure Based on Close-Range Photogrammetry

Visual reconstruction of milling workpiece surface topography based on phase shifting and complementary gray code

When the surface texture of a workpiece is extremely fine, cameras struggle to accurately capture depth information, making conventional machine vision methods insufficient for achieving micrometer-scale three-dimensional surface reconstructions. To overcome this limitation, the study focuses on high-precision 3D reconstruction of the surface morphology of milled workpieces. Given the smoothness of milled surfaces, their susceptibility to overexposure, and the difficulty in extracting depth information from two-dimensional image pixels, the paper proposes a novel method that combines phase-shifting with complementary Gray code to achieve micrometer-level surface reconstruction. The superiority of this method over traditional phase-shifting techniques is demonstrated through comparisons of overall morphology, two-dimensional power spectral density (2D PSD), and the average deviations of sampled surfaces. Results show that the proposed method reduces the average relative error in surface deviations by 25.89% compared to traditional techniques. Furthermore, cross-sectional analyses reveal that the reconstructed point cloud surface more closely mirrors the actual peak-to-valley characteristics of the real surface. Experimental results confirm that this method effectively captures the surface features of milled workpieces, indicating broad potential for applications in precision manufacturing.

Three-dimensional modeling of 500 kV transmission lines by airborne LiDAR

This study aims to enhance the accuracy of three-dimensional (3D) modeling in the planning, design, operation management, and risk assessment of 500 kV transmission lines. Addressing the limitations of traditional two-dimensional (2D) or low-precision 3D models, a new method based on airborne Light Detection and Ranging (LiDAR) technology is proposed. Initially, high-density, high-precision 3D point cloud data are collected using airborne LiDAR technology. This data is then processed using specialized software for noise removal, terrain classification extraction, and Geographic Information System (GIS) data integration. Subsequently, a Convolutional Neural Network (CNN) is employed to classify the 3D point cloud data to enhance the accuracy of complex structure recognition. Lastly, the point cloud data are converted into linear and planar entities using Computer-Aided Design (CAD) technology, achieving high-precision 3D reconstruction of the 500 kV transmission lines and their complex terrain environment. Experimental results show that the spatial resolution of the constructed model reaches 0.1 m per pixel, with overall geometric accuracy within ±5 cm. Compared to traditional aerial imagery and ground survey methods, the model based on airborne LiDAR technology achieves improvements of 30 % in detail richness and 35 % in terrain adaptability, with a 45 % increase in decision-making efficiency. These improvements significantly enhance the accuracy of line operations, safety assessments, and environmental impact analyses, providing solid data support for the modernization and intelligent development of the power industry.

Three-dimensional particulate volume fraction reconstruction in the fluid based on the Lambert–Beer physics information neural network

The measurement of particle volume fraction in flow fields is of great significance in scientific research and engineering applications. As one of the particle detection techniques, the light extinction method is widely used in measuring nano-particles volume fraction in flow fields due to its simplicity and non-contact nature. In particular, in complex reactive flow fields like combustion reactions, the volume fraction of soot particulate and other particles can be accurately measured and reconstructed via the light extinction method that based on the Beer–Lambert law. This is crucial for exploring combustion phenomena, understanding their internal mechanisms, and reducing pollutant emissions. However, due to the enormous computational burden, current algebra reconstruction techniques struggle to achieve high-precision three-dimensional (3D) reconstruction of particles. Therefore, this paper originally proposes a 3D reconstruction algorithm based on the Beer–Lambert law physical information neural networks (LB-PINNs). By incorporating physical information as constraints into the particle reconstruction process, it is possible to achieve high-precision 3D reconstruction of particles in complex flow field environments with low computational cost. Meanwhile, to address the trade-off issues of reconstruction accuracy and smooth noise resistance in previous reconstruction algorithms, i.e., Tikhonov regularization, this paper employs dynamically adjusted regularization parameters in the LB-PINN algorithm. This approach ensures smooth noise-resistant processing while maintaining reconstruction accuracy, significantly reducing computation time and resource consumption. According to the experimental results, LB-PINNs demonstrate superior performance compared to previous reconstruction algorithms when reconstructing the soot volume fraction in complex reacting flow fields, i.e., combustion flame scenarios.

Three-dimensional reconstruction method for layered structures based on a frequency modulated continuous wave terahertz radar.

The feasibility of employing a continuous-wave terahertz detection system for non-contact and non-destructive testing (NDT) in multi-layered bonding structures is assessed in this study. The paper introduces the detection principle of terahertz frequency modulated continuous wave (FMCW) radar and outlines the two-dimensional (2D) scanning platform, which integrates optical lenses, three linear actuators, a control platform, and data acquisition units. Experimental results on two types of insulation with prefabricated defects demonstrate the capability of terahertz waves for transparent inspection imaging. These results confirm the viability of terahertz FMCW detection technology as an advanced NDT tool for multi-layered bonding structures. However, the inherent limitations of terahertz wavelength and hardware systems pose challenges in discriminating reflection peaks on upper and lower surfaces. To address this issue, a local adaptive empirical wavelet coefficient modal decomposition (LAEWCMD) method is proposed to enhance the longitudinal discrimination ability of terahertz detection. The proposed method involves segmenting the 2D terahertz detection image into regions to differentiate between defective and non-defective areas. Continuous wavelet transforms (CWT) are then applied to the range signals of each region to derive continuous wavelet coefficients (CWCs). Subsequently, empirical mode decomposition (EMD) is performed on the CWCs to decompose them into intrinsic mode functions (IMFs) and residual signals. The 1st IMF is utilized for three-dimensional (3D) reconstruction, and the regions are fused to generate the final output. The effectiveness of the proposed method is validated on aircraft thermal protection structures (TPS), achieving high-precision 3D reconstruction. This offers a novel approach for the application of terahertz computed tomography imaging and NDT.

High-Precision 3D reconstruction and quantitative structure description: Linking microstructure to macroscopic heat transfer of aerogels

The intricate interplay between microstructure and macroscopic transfer processes in porous materials has long been recognized, yet achieving a definitive link between them remains a formidable challenge. Herein, we propose an effective approach to address this challenge by introducing a high-precision 3D reconstruction and structure descriptor database analysis method of nanoporous aerogels, which enables deeper understanding and accurate predictions of structure-directed properties. By employing focused ion beam-scanning electron microscope (FIB-SEM) technique supplemented with image preprocessing and image segmentation, we successfully built a 3D reconstruction model that exhibits high consistency with experimental results. Then, a structure descriptor database from the view of original porous structure, structure connectivity, and topological information is established, and 10 descriptors are preliminarily filtered relevance with thermal conductivity. Consequently, the quantitative relationship between descriptor and thermal conductivity is evaluated via a back propagation (BP) neural network, and three categories structure descriptors which have the most significant impact are determined as (i) the description of gaseous heat transfer pathways, (ii) the description of three-dimensional connectivity, and (iii) the description of interface complexity. With the profound understanding of structure descriptor, the present work offers a novel, structure-focused strategy that closely link the microstructure to macroscopic properties of porous materials.

Three-dimentional reconstruction of underwater side-scan sonar images based on shape-from-shading and monocular depth fusion

Modern marine research requires high-precision three-dimensional (3D) underwater data. Underwater environments experience severe visible light attenuation, which leads to inferior imaging compared with air. In contrast, sound waves are less affected underwater; hence side-scan sonar is used for underwater 3D reconstruction. Typically, the shape-from-shading algorithm (SfS) is widely used to reconstruct surface normal or heights from side-scan sonar images. However, this approach has challenges because of global information loss and noise. To address these issues, this study introduces a surface-normal fusion method. Specifically, we propose a frequency separation SfS algorithm using a discrete cosine transform, which provides a surface-normal map with less noise. We then fuse the surface-normal map with a novel depth estimation network to achieve high-precision 3D reconstruction of underwater side-scan sonar images. We conducted experiments on synthetic, NYU-depth-v2, and real side-scan sonar datasets to demonstrate the effectiveness of the proposed method.

Disparity Refinement for Stereo Matching of High-Resolution Remote Sensing Images Based on GIS Data

With the emergence of the Smart City concept, the rapid advancement of urban three-dimensional (3D) reconstruction becomes imperative. While current developments in the field of 3D reconstruction have enabled the generation of 3D products such as Digital Surface Models (DSM), challenges persist in accurately reconstructing shadows, handling occlusions, and addressing low-texture areas in very-high-resolution remote sensing images. These challenges often lead to difficulties in calculating satisfactory disparity maps using existing stereo matching methods, thereby reducing the accuracy of 3D reconstruction. This issue is particularly pronounced in urban scenes, which contain numerous super high-rise and densely distributed buildings, resulting in large disparity values and occluded regions in stereo image pairs, and further leading to a large number of mismatched points in the obtained disparity map. In response to these challenges, this paper proposes a method to refine the disparity in urban scenes based on open-source GIS data. First, we register the GIS data with the epipolar-rectified images since there always exists unignorable geolocation errors between them. Specifically, buildings with different heights present different offsets in GIS data registering; thus, we perform multi-modal matching for each building and merge them into the final building mask. Subsequently, a two-layer optimization process is applied to the initial disparity map based on the building mask, encompassing both global and local optimization. Finally, we perform a post-correction on the building facades to obtain the final refined disparity map that can be employed for high-precision 3D reconstruction. Experimental results on SuperView-1, GaoFen-7, and GeoEye satellite images show that the proposed method has the ability to correct the occluded and mismatched areas in the initial disparity map generated by both hand-crafted and deep-learning stereo matching methods. The DSM generated by the refined disparity reduces the average height error from 2.2 m to 1.6 m, which demonstrates superior performance compared with other disparity refinement methods. Furthermore, the proposed method is able to improve the integrity of the target structure and present steeper building facades and complete roofs, which are conducive to subsequent 3D model generation.

Accurate and semantic 3D reconstruction of maize leaves

The rapid advances in equipment and technology for crop three-dimensional (3D) data acquisition have enabled the high-throughput and high-quality acquisition of crop 3D data. However, the challenge now lies in comprehending unstructured crop 3D data and extracting phenotypes and knowledge. Extracting semantic feature points from plant leaves and achieving high-precision 3D reconstruction is a prevalent and formidable issue in this field. In this study, we present an accurate method for semantic 3D reconstruction of maize leaves. The algorithm utilizes 3D point cloud data of maize leaves as the initial input. Subsequently, the semantic 3D reconstruction of maize leaves is achieved by means of point cloud preprocessing, initial triangular meshing, hole filling, remeshing, mesh parameterization, two-dimensional (2D) edge detection, 2D semantic point interpolation, 3D semantic point selection, mesh generation, and optimization of feature-point position. A total of 56 leaves from five different maize cultivars were reconstructed utilizing this method. The results indicate that the average distance error index dCDMP between the reconstructed mesh vertices and the input point cloud is less than 0.3 cm2 for all five cultivars, and the mean of the maximum distance error index dHDMP for each cultivar is within the range of 0.5–0.8 cm, indicating it is high-accuracy. The visualization of leaves shows good consistency between the reconstructed mesh and corresponding point cloud. Additionally, the reconstructed 3D mesh of the leaves exhibits enhanced triangular facet consistency. Each vertex can be precisely identified as a leaf vein point, leaf edge point, leaf tip point, or leaf surface point, and includes semantic data such as the position on the leaf blade, which can be directly employed for phenotype estimations. High-precision reconstruction of 3D models of crop organs that include semantic information and cultivar resolution yields significant benefits. This technology can improve the resolution and accuracy of crop morphological and structural research, and it is also crucial for crop phenomics, functional-structural plant modelling, and fusion research between plant phenotyping and crop models.

Retracted: High-Precision 3D Reconstruction of Cooperative Markers under Motion Blur

Retracted: High-Precision 3D Reconstruction of Cooperative Markers under Motion Blur

Robot trajectory planning for autonomous 3D reconstruction of cockpit in aircraft final assembly testing

The trend towards automation and intelligence in aircraft final assembly testing has led to a new demand for autonomous perception of unknown cockpit operation scenes in robotic collaborative airborne system testing. To address this demand, a robotic automated 3D reconstruction cell which enables to autonomously plan the robot end-camera's trajectory is developed for image acquisition and 3D modeling of the cockpit operation scene. A continuous viewpoint path planning algorithm is proposed that incorporates both 3D reconstruction quality and robot path quality into optimization process. Smoothness metrics for viewpoint position paths and orientation paths are introduced together for the first time in 3D reconstruction. To ensure safe and effective movement, two spatial constraints, Domain of View Admissible Position (DVAP) and Domain of View Admissible Orientation (DVAO), are implemented to account for robot reachability and collision avoidance. By using diffeomorphism mapping, the orientation path is transformed into 3D, consistent with the position path. Both orientation and position paths can be optimized in a unified framework to maximize the gain of reconstruction quality and path smoothness within DVAP and DVAO. The reconstruction cell is capable of automatic data acquisition and fine scene modeling, using the generated robot C-space trajectory. Simulation and physical scene experiments have confirmed the effectiveness of the proposed method to achieve high-precision 3D reconstruction while optimizing robot motion quality.

A novel method for 3D reconstruction in tight and restricted space based on improved line-structured light

Three-dimensional (3D) reconstruction techniques have been widely employed in many fields. But precise endoscopic 3D measurement inside of objects, especially in tight and restricted spaces, is still a challenge since there is no easy access for measurement tools. Endoscope with 2D-view function has been widely used. However, the precise measurement technology of the inner surface of small hollow objects has not been widely researched. The components of the existing endoscope three-dimensional measuring probe are complex, expensive and not performed well in accuracy. Thus, a novel method and a miniaturized endoscopic 3D sensor is introduced. To be suit in tight and restricted space, the sensor probe prototype has a maximum 6 mm in thickness. The miniatured sensor prototype is mainly based on improved line-structured light, which performs better in robustness and measurement accuracy than coded projection. Especially, the components are inexpensive and easily available, which reduces the manufacturing cost thus making it easily to popularize in industry. Correspondingly, to ensure the sensor measurement accuracy, we also proposed a calibration method for a miniature line-structured light system based on specially designed planar targets. It should be noted that the 3D extension of the line-structured light system relies on the movement of the translation stage. Taking full account of the characteristics of the sensor prototype, a high-precision direction calibration method of the translation stage based on joint multi-calibration position noise estimation is proposed. Finally, the accuracy experiment is carried out on the gauge block to verify that the measurement method proposed can achieve high-precision 3D reconstruction in a restricted space, and that the 3D reconstruction error is less than 10µm.

Flexible and fast calibration method for uni-directional multi-line structured light system

Multi-line Shift Profilometry (MLSP) is widely used to measure the high-reflectivity object by projecting high-contrast binary stripe image. However, in the calibration of a MLSP system, the lens distortion of the projector makes the most existing calibration methods fail in obtaining high accurate result. Therefore, to avoid the influence of lens distortion, we propose a novel flexible and fast calibration method for MLSP, which requiring only uni-directional patterns. In this method, for each bending light plane, lens distortion is avoided by establishing a polynomial model between three-dimensional (3D) coordinates and camera sub-pixel coordinates. Because the position of calibration planar targets is placed freely without any constraint, this method is flexible and convenient to operate. Experimental results demonstrate that the proposed method achieves high-precision 3D reconstruction.

High-precision 3D reconstruction of multiple magnetic targets based on center weighting method

A high-precision three-dimensional reconstruction method of multiple magnetic targets is studied in this paper. The inversion space is divided into cubic grid cells, and various complex shapes of magnetic targets are simulated by the combination of grid cells. The forward calculation of the magnetic gradient tensor in the spatial domain is performed. Then, the correlation imaging method is used to calculate cross correlation between the observed magnetic gradient tensor and the theoretical magnetic gradient at each grid cell tensor due to a magnetic dipole. The spatial distribution and center position of magnetic targets are preliminarily estimated by the resultant correlation coefficients, and use them to construct the center weighting function as a constraint to the inversion equations. The solution is completed by using the conjugate gradient method to estimate the magnetic susceptibility of each grid cell, and the 3D outline of the magnetic target is depicted by using the visualization technique. The experimental results show that the proposed method can achieve the 3D reconstruction of multiple simple magnetic targets with high precision.

Motion Error Estimation and Compensation of Airborne Array Flexible SAR Based on Multi-Channel Interferometric Phase

Airborne array synthetic aperture radar (SAR) has made a significant breakthrough in the three-dimensional resolution of traditional SAR. In the airborne array SAR 3D imaging technology, the baseline length is the main factor restricting the resolution. Airborne array flexible SAR can increase the baseline length to improve the resolution and interference performance by mounting antennae on the wing. The existing research lacks results obtained using flexible actual data processing and specific motion compensation methods. Thus, this paper proposes a motion error estimation and compensation method for an airborne array flexible SAR based on a multi-channel interferometric phase. Firstly, a flexible channel motion compensation model is established based on the multi-channel interference phase of airborne array flexible SAR. Then, based on the rigid multi-channel data, combined with the ground control points, the least square method, and the global optimal search algorithm, the accurate rigid baseline length and the central incidence angle are obtained. Finally, according to the multi-channel interference phase inversion of the flexible motion error and combined with the motion compensation model, the flexible data are compensated in the time domain. The actual results indicate that, compared with traditional motion compensation methods, our method can obtain accurate flexible compensation data. This study improves the interference performance of multi-channel data of airborne array flexible SAR and lays a solid foundation for the high-precision 3D reconstruction of airborne array flexible SAR.

Binocular Structured Light 3-D Reconstruction System for Low-Light Underwater Environments: Design, Modeling, and Laser-Based Calibration

The three dimensional (3D) perception of the low-light underwater environment has always been a major challenge which greatly limits the underwater operations. In this paper, an underwater active vision measurement system based on binocular structured light is designed to achieve high precision 3D reconstruction. Firstly, the fusion technology of binocular camera and laser addresses underwater optical attenuation and feature sparsity. Then, in order to avoid the huge inertia caused by the overall movement of the system, a laser scanner based on the mirror-galvanometer is used to accomplish static scanning of the scene. Subsequently, considering the influence of multiple media, underwater refraction models including monocular imaging model, binocular ranging model, and binocular polar curve constraint model are systematically proposed. What’s more, the conventional checkerboard-based passive visual calibration method is ineffective for low-light waters. Therefore, a simple calibration block is designed, and a new multi-objective laser-based calibration algorithm based on laser geometric constraints is proposed. Finally, the effectiveness of the system is verified by analyzing the 3D reconstruction results of underwater objects.

An efficient 3D extraction and reconstruction method for myelinated axons of mouse cerebral cortex based on mixed intelligence

Accurate reconstruction of the 3D morphology and spatial distribution of myelinated axons in mouse brains is very important for understanding the mechanism and dynamic behavior of long-distance information transmission between brain regions. However, it is difficult to segment and reconstruct myelinated axons automatically due to two reasons: the amount of it is huge and the morphology of it is different between brain regions. Traditional artificial labeling methods usually require a large amount of manpower to label each myelinated axon slice by slice, which greatly hinders the development of the mouse brain connectome. In order to solve this problem and improve the reconstruction efficiency, this paper proposes an annotation generation method that takes the myelinated axon as prior knowledge, which can greatly reduce the manual labeling time while reaching the level of manual labeling. This method consists of three steps. Firstly, the 3D axis equation of myelinated axons is established by sparse axon artificial center point labels on slices, and the region to be segmented is pre-extracted according to the 3D axis. Subsequently, the U-Net network was trained by a small number of artificially labeled myelinated axons and was used for precise segmentation of output by the last step, so as to obtain accurate axon 2D morphology. Finally, based on the segmentation results, the high-precision 3D reconstruction of axons was performed by imaris software, and the spatial distribution of myelinated axons in the mouse brain was reconstructed. In this paper, the effectiveness of this method was verified on the dataset of high-resolution X-ray microtomography of the mouse cortex. Experiments show that this method can achieve an average MIoU 81.57, and the efficiency can be improved by more than 1400x compared with the manual labeling method.

Single frame color fringe projection profilometry based on empirical mode decomposition imbalance correction algorithm for fast three-dimensional reconstruction

Phase-shifting profilometry (PSP) is considered to be the most accurate technique for phase retrieval with fringe projection profilometry (FPP) systems. However, PSP requires that multiple phase-shifted fringe patterns be acquired, usually sequentially, which limits PSP to static or quasistatic imaging. We introduce color FPP (CFPP), which provides three-dimensional (3D) imaging using a single acquisition, for the application of real-time profiling. A single frame acquisition provides all three phase-shifted fringe patterns needed for the PSP phase retrieval algorithm. However, some problems must be overcome in CFPP, namely, color coupling and imbalance. Following up on the study, we propose a fixed coefficient decoupling method based on point-to-point correction and then introduce the empirical mode decomposition algorithm to decompose a color fringe pattern into three high-frequency components and three low-frequency components for imbalance correction, which improves measurement accuracy and realizes a single frame measurement for complex and discontinuous objects. After correction by the proposed method, the shape of the tested object can be recovered by one a single frame color fringe pattern, which meets the need for dynamic measurement, and the accuracy is not limited by the movement speed of object. The simulations and experimental results show that the proposed method can significantly reduce the influence of color coupling and imbalance in CFPP, and thus realize dynamic and high-precision 3D reconstruction.

3D reconstruction method for tree seedlings based on point cloud self-registration

The 3D reconstruction of tree seedlings can help to assess phenotypic architectures, conceive virtual urban landscapes and design computer games. The existing multicamera photograph technology already has the capability to accurately reconstruct 3D models for small scene plants, such as corn and vegetable seedlings. However, the existing plant 3D reconstruction system has several shortcomings, such as its high cost, complicated operation procedure, and unsuitability for seedling trees. Therefore, this paper proposes an autonomous alignment method for seedling point clouds that can realize the low-cost and fast 3D reconstruction of batch seedlings. In this study, we designed a system based on a low-cost Kinect camera and a precision turntable to construct 3D seedling models. A special turntable was adopted to achieve self-registration for the seedling point clouds. It was efficient for us to obtain several 3D seedlings models with only one registration. The system could capture images automatically from different viewpoints and submit these images to a graphic workstation for processing. In our work, we set three fixed views, V2, V3 and V4, to evaluate the cumulative errors caused by multiview matching. It needn’t touch any parts of the seedings to create 3D models at different view by the proposed method. Herein, the large proportions of 0 < mean absolute distance, MD ≤ 0.6 cm and 0 < standard deviation, SD ≤ 0.4 cm, between the reference and the reconstructed point cloud showed that the 3D reconstruction method was accurate, stable and flexible. Additionally, we validated the phenotypic structure measurement, and the height H was highly accurate (R2 > 0.985) when using the 3D reconstruction models of seedlings. Experiments demonstrate that the proposed method has the potential to obtain high-precision 3D reconstruction models and phenotypic parameters for seedlings via low-cost equipment with high-efficiency processing algorithms.