3D Surface Reconstruction Method Research Articles

Asteroid-NeRF: A deep-learning method for 3D surface reconstruction of asteroids

Context. Asteroids preserve important information about the origin and evolution of the Solar System. Three-dimensional (3D) surface models of asteroids are essential for exploration missions and scientific research. Regular methods for 3D surface reconstruction of asteroids, such as stereo-photogrammetry (SPG), usually struggle to reconstruct textureless areas and can only generate sparse surface models. Stereo-photoclinometry (SPC) can reconstruct pixel-wise topographic details but its performance depends on the availability of images obtained under different illumination conditions and suffers from uncertainties related to surface reflectance and albedo. Aims. This paper presents Asteroid-NeRF, a novel deep-learning method for 3D surface reconstruction of asteroids that is based on the state-of-the-art neural radiance field (NeRF) method. Methods. Asteroid-NeRF uses a signed distance field (SDF) to reconstruct a 3D surface model from multi-view posed images of an asteroid. In addition, Asteroid-NeRF incorporates appearance embedding to adapt to different illumination conditions and to maintain the geometric consistency of a reconstructed surface, allowing it to deal with the different solar angles and exposure conditions commonly seen in asteroid images. Moreover, Asteroid-NeRF incorporates multi-view photometric consistency to constrain the SDF, enabling optimised reconstruction. Results. Experimental evaluations using actual images of asteroids Itokawa and Bennu demonstrate the promising performance of Asteroid-NeRF, complementing traditional methods such as SPG and SPC. Furthermore, due to the global consistency and pixel-wise training of Asteroid-NeRF, it produces highly detailed surface reconstructions. Asteroid-NeRF offers a new and effective solution for high-resolution 3D surface reconstruction of asteroids that will aid future exploratory missions and scientific research on asteroids.

SSRNet: Scalable 3D Surface Reconstruction Network.

Learning-based surface reconstruction methods have received considerable attention in recent years due to their excellent expressiveness. However, existing learning-based methods lack scalability in processing large-scale point clouds. This paper proposes a novel scalable learning-based 3D surface reconstruction method based on octree, called SSRNet. SSRNet works in a scalable reconstruction pipeline, which divides oriented point clouds into different local parts and then processes them in parallel. Accommodating this scalable design pattern, SSRNet constructs local geometric features for octree vertices. Such features comprise the relation between the vertices and the implicit surface, ensuring geometric perception. Focusing on local geometric information also enables the network to avoid the overfitting problem and generalize well on different datasets. Finally, as a learning-based method, SSRNet can process large-scale point clouds in a short time. And to further solve the efficiency problem, we provide a lightweight and efficient version that is about five times faster while maintaining reconstruction performance. Experiments show that our methods achieve state-of-the-art performance with outstanding efficiency.

An Automatic and Robust Visual SLAM Method for Intra-Abdominal Environment Reconstruction

Three-dimensional (3D) surface reconstruction is used to solve the problem of the narrow field of view in laparoscopy. It can provide surgeons or computer-assisted surgery systems with real-time complete internal abdominal anatomy. However, rapid changes in image depth, less texture, and specular reflection pose a challenge for the reconstruction. It is difficult to stably complete the reconstruction process using feature-based simultaneous localization and mapping (SLAM) method. This paper proposes a robust laparoscopic 3D surface reconstruction method using SLAM, which can automatically select appropriate parameters for stereo matching and robustly find matching point pairs for laparoscope motion estimation. The changing trend of disparity maps is used to predict stereo matching parameters to improve the quality of the disparity map. Feature patch extraction and tracking are selected to replace feature point extraction and matching in motion estimation, which reduces its failure and interruption in feature-based SLAM. The proposed feature patch matching method is suitable for parallel computing, which can improve its computing speed. Evaluation results on public in vivo and ex vivo porcine abdominal video data show the efficiency and robustness of our 3D surface reconstruction approach.

UniRender: Reconstructing 3D Surfaces from Aerial Images with a Unified Rendering Scheme

While recent advances in the field of neural rendering have shown impressive 3D reconstruction performance, it is still a challenge to accurately capture the appearance and geometry of a scene by using neural rendering, especially for remote sensing scenes. This is because both rendering methods, i.e., surface rendering and volume rendering, have their own limitations. Furthermore, when neural rendering is applied to remote sensing scenes, the view sparsity and content complexity that characterize these scenes will severely hinder its performance. In this work, we aim to address these challenges and to make neural rendering techniques available for 3D reconstruction in remote sensing environments. To achieve this, we propose a novel 3D surface reconstruction method called UniRender. UniRender offers three improvements in locating an accurate 3D surface by using neural rendering: (1) unifying surface and volume rendering by employing their strengths while discarding their weaknesses, which enables accurate 3D surface position localization in a coarse-to-fine manner; (2) incorporating photometric consistency constraints during rendering, and utilizing the points reconstructed by structure from motion (SFM) or multi-view stereo (MVS), to constrain reconstructed surfaces, which significantly improves the accuracy of 3D reconstruction; (3) improving the sampling strategy by locating sampling points in the foreground regions where the surface needs to be reconstructed, thus obtaining better detail in the reconstruction results. Extensive experiments demonstrate that UniRender can reconstruct high-quality 3D surfaces in various remote sensing scenes.

A non-destructive approach to estimate buttress volume using 3D point cloud data

Buttressed trees provide mechanical support for themselves and offer essential ecological functions, such as nutrient acquisition, while being one of the largest sources of volume or biomass estimation variation in tropical forests. In this study, we collected 51 buttressed trees from (33) Democratic Republic of Congo, (12) Indonesia, and (6) Costa Rica, including (48) point clouds, and (3) destructive measurement. Specifically, we compared the performance of the Alpha Shape Algorithm (ASA) and the Slice Triangulation (ST) method on buttress volume estimation based on 30 point clouds with two species. Six point clouds from Costa Rica were used to validate the 3D surface reconstruction method. Meanwhile, we developed three allometric models based on 36 point clouds: a diameter above the buttress-based (DAB, 39 to 203 cm) model, a diameter computed from the non-convex area (Darea130) model, and the convex hull perimeter (Dconvex130) of the breast height model. The developed models were validated with independent data, including (6) point clouds and (3) destructive measurements, to highlight the broader contextualization and application of these methods. Volume estimated by the ASA and ST showed a high agreement with the reference volume acquired using the Smalian formula (relative RMSE of 0.07 and 0.11, respectively, regardless of species effect). ASA was also robust when modeling trees with more and shallower horizontal buttresses. Darea130 was the most accurate predictor to estimate buttress volume, with a lower Akaike information criterion (−66.25) than DAB (−59.55) and Dconvex130 (30.56); however, DAB and Darea130 (relative RMSE of 0.21 and 0.23, respectively) showed similar performance when validated with independent datasets. Our results indicate that the ASA approach performs better than both the ST and allometric models used in this study. Furthermore, the ASA method can help correct the bias in the present and past estimates of volume and biomass of large trees, which are foundational components for understanding biomass allocation and dynamics in tropical forests contemporary fields.

3D Surface Reconstruction Method for Microsamples Based on Polarization Modulation Microscopy

A novel microscopic Shape-From-Polarization (SFP) 3D shape reconstruction method is proposed, in which degree of polarization is calculated using Polarized Bidirectional Reflectance Distribution Function (PBRDF) to overcome azimuthal ambiguity for both specular and diffuse polarization microscopic scenes. Based on the calculated degree of polarization, a polarization modulation microscopy oriented variational model is established to estimate surface normal vector and rebuild 3D shape of the sample with “fuzzy guidance” based on atomic force microscopy (AFM). Experiments on different microscopic samples indicate the proposed method can restore the stereoscopic morphology distribution with precision in micrometer.

Research on 3D medical image surface reconstruction based on data mining and machine learning

Three-dimensional (3D) medical images are prone to overlap, and there are some problems, such as low detection efficiency and inconsistent with the actual situation. Therefore, a 3D medical image surface reconstruction method based on data mining and machine learning is proposed. The 3D medical images were classified according to different ways, the information frame of 3D medical images was established and the surface overlapping information model of 3D images was given. Based on this information framework, the nonlinear function of overlapping area information of 3D medical images was constructed. The weight of the nonlinear function was used to calculate the input and output results of overlapping area information. Combined with the input mode of 3D medical image information, the error between the information output and the expected output was set. The nonlinear function weight of the overlapping area information of 3D medical images was modified by using the learning rate and the use time of the overlapping area information, and the influence factors of the overlapping information detection were obtained by increasing the situation terms, so as to complete the detection of the surface reconstruction information of 3D medical images. The experimental results show that the information detection results of the proposed method fit well with the actual situation, and the information detection efficiency is high.

Modeling three-dimensional rough surface and simulation of temperature and flow field in laser transmission welding

• A numerical model for rough surface is established by applying the anisotropic 3D surface reconstruction method. • A 3D FEM model considering rough surface is developed to simulate and analyze the temperature field, molten pool geometry and flow fields in the LTW process. • The temperature of the contact surface between the transparent and absorbent layer is discontinuous, which is consistent with the incomplete contact under real welding conditions. The traditional Laser Transmission Welding (LTW) simulation method assumes that the welding contact interface is smooth. However, the actual welding interfaces exhibit a large number of rough surfaces. The mismatch may decrease the accuracy of the traditional finite element model (FEM). Thus, this work aims to establish a three-dimensional (3D) FEM model considering the rough surface. Numerical model for 3D rough surface of welding specimen is established. The distribution of molten pool area, temperature and flow field coupling are discussed. Then, a series of experiments were set up to obtain the weld width to verify the model. The results show that the temperature of the contact surface between the transparent part and the absorbent layer is discontinuous, which is consistent with the incomplete contact of the contact surface under real welding conditions. The error of weld width values between the simulated and experiment is smaller, which also confirm that the established model is more precise.

Wavelia Breast Imaging: The Optical Breast Contour Detection Subsystem

Wavelia is a low-power electromagnetic wave breast imaging device for breast cancer diagnosis, which consists of two subsystems, both performing non-invasive examinations: the Microwave Breast Imaging (MBI) subsystem and the Optical Breast Contour Detection (OBCD) subsystem. The Wavelia OBCD subsystem is a 3D scanning device using an infrared 3D stereoscopic camera, which performs an azimuthal scan to acquire 3D point clouds of the external surface of the breast. The OBCD subsystem aims at reconstructing fully the external envelope of the breast, with high precision, to provide the total volume of the breast and morphological data as a priori information to the MBI subsystem. This paper presents a new shape-based calibration procedure for turntable-based 3D scanning devices, a new 3D breast surface reconstruction method based on a linear stretching function, as well as the breast volume computation method that have been developed and integrated with the Wavelia OBCD subsystem, before its installation at the Clinical Research Facility of Galway (CRFG), in Ireland, for first-in-human clinical testing. Indicative results of the Wavelia OBCD subsystem both from scans of experimental breast phantoms and from patient scans are thoroughly presented and discussed in the paper.

A Real-Time 3D Measurement System for the Blast Furnace Burden Surface Using High-Temperature Industrial Endoscope.

Capturing the three-dimensional (3D) shape of the burden surface of a blast furnace (BF) in real-time with high accuracy is crucial for improving gas flow distribution, optimizing coke operation, and stabilizing BF operation. However, it is difficult to perform 3D shape measurement of the burden surface in real-time during the ironmaking process because of the high-temperature, high-dust, and lightless enclosed environment inside the BF. To solve this problem, a real-time 3D measurement system is developed in this study by combining an industrial endoscope with a virtual multi-head camera array 3D reconstruction method. First, images of the original burden surface are captured using a purpose-built industrial endoscope. Second, a novel micro-pixel luminance polarization method is proposed and applied to compensate for the heavy noise in the backlit images due to high dust levels and poor light in the enclosed environment. Third, to extract depth information, a multifeature-based depth key frame classifier is designed to filter out images with high levels of clarity and displacement. Finally, a 3D shape burden surface reconstruction method based on a virtual multi-head camera array is proposed for capturing the real-time 3D shape of the burden surface in an operational BF. The results of an industrial experiment illustrate that the proposed method can measure the 3D shape of the entire burden surface and provide reliable burden surface shape information for BF control.

High-speed surface reconstruction of a flying bird using structured light.

Birds fly effectively and maneuver nimbly by dynamically changing the shape of their wings during each wingbeat. These shape changes have yet to be quantified automatically at high temporal and spatial resolution. Therefore, we developed a custom 3D surface reconstruction method, which uses a high-speed camera to identify spatially encoded binary striped patterns that are projected on a flying bird. This non-invasive structured-light method allows automated 3D reconstruction of each stand-alone frame and can be extended to multiple views. We demonstrate this new technique by automatically reconstructing the dorsal surface of a parrotlet wing at 3200framess-1 during flapping flight. From this shape we analyze key parameters such as wing twist and angle of attack distribution. While our binary 'single-shot' algorithm is demonstrated by quantifying dynamic shape changes of a flying bird, it is generally applicable to moving animals, plants and deforming objects.

3D Surface Reconstruction of Noisy Point Clouds Using Growing Neural Gas: 3D Object/Scene Reconstruction

With the advent of low-cost 3D sensors and 3D printers, scene and object 3D surface reconstruction has become an important research topic in the last years. In this work, we propose an automatic (unsupervised) method for 3D surface reconstruction from raw unorganized point clouds acquired using low-cost 3D sensors. We have modified the growing neural gas network, which is a suitable model because of its flexibility, rapid adaptation and excellent quality of representation, to perform 3D surface reconstruction of different real-world objects and scenes. Some improvements have been made on the original algorithm considering colour and surface normal information of input data during the learning stage and creating complete triangular meshes instead of basic wire-frame representations. The proposed method is able to successfully create 3D faces online, whereas existing 3D reconstruction methods based on self-organizing maps required post-processing steps to close gaps and holes produced during the 3D reconstruction process. A set of quantitative and qualitative experiments were carried out to validate the proposed method. The method has been implemented and tested on real data, and has been found to be effective at reconstructing noisy point clouds obtained using low-cost 3D sensors.

Colored 3D surface reconstruction using Kinect sensor

A colored 3D surface reconstruction method which effectively fuses the information of both depth and color image using Microsoft Kinect is proposed and demonstrated by experiment. Kinect depth images are processed with the improved joint-bilateral filter based on region segmentation which efficiently combines the depth and color data to improve its quality. The registered depth data are integrated to achieve a surface reconstruction through the colored truncated signed distance fields presented in this paper. Finally, the improved ray casting for rendering full colored surface is implemented to estimate color texture of the reconstruction object. Capturing the depth and color images of a toy car, the improved joint-bilateral filter based on region segmentation is used to improve the quality of depth images and the peak signal-to-noise ratio (PSNR) is approximately 4.57 dB, which is better than 1.16 dB of the joint-bilateral filter. The colored construction results of toy car demonstrate the suitability and ability of the proposed method.

A method for brain 3D surface reconstruction from MR images

Due to the encephalic tissues are highly irregular, three-dimensional (3D) modeling of brain always leads to complicated computing. In this paper, we explore an efficient method for brain surface reconstruction from magnetic resonance (MR) images of head, which is helpful to surgery planning and tumor localization. A heuristic algorithm is proposed for surface triangle mesh generation with preserved features, and the diagonal length is regarded as the heuristic information to optimize the shape of triangle. The experimental results show that our approach not only reduces the computational complexity, but also completes 3D visualization with good quality.

Calibration-free quantitative surface topography reconstruction in scanning electron microscopy

This work presents a new approach to obtain reliable surface topography reconstructions from 2D Scanning Electron Microscopy (SEM) images. In this method a set of images taken at different tilt angles are compared by means of digital image correlation (DIC). It is argued that the strength of the method lies in the fact that precise knowledge about the nature of the rotation (vector and/or magnitude) is not needed. Therefore, the great advantage is that complex calibrations of the measuring equipment are avoided. The paper presents the necessary equations involved in the methods, including derivations and solutions. The method is illustrated with examples of 3D reconstructions followed by a discussion on the relevant experimental parameters.

Exploiting Visibility Information in Surface Reconstruction to Preserve Weakly Supported Surfaces

We present a novel method for 3D surface reconstruction from an input cloud of 3D points augmented with visibility information. We observe that it is possible to reconstruct surfaces that do not contain input points. Instead of modeling the surface from input points, we model free space from visibility information of the input points. The complement of the modeled free space is considered full space. The surface occurs at interface between the free and the full space. We show that under certain conditions a part of the full space surrounded by the free space must contain a real object also when the real object does not contain any input points; that is, an occluder reveals itself through occlusion. Our key contribution is the proposal of a new interface classifier that can also detect the occluder interface just from the visibility of input points. We use the interface classifier to modify the state-of-the-art surface reconstruction method so that it gains the ability to reconstruct weakly supported surfaces. We evaluate proposed method on datasets augmented with different levels of noise, undersampling, and amount of outliers. We show that the proposed method outperforms other methods in accuracy and ability to reconstruct weakly supported surfaces.

Robust surface reconstruction of teeth from raw pointsets

This paper discusses a 3D robust dental surface reconstruction method. The surface reconstruction method usually consists of the following stages: mesh generation, surface registration, and surface merging. A large amount of noise will be accumulated after these three processes especially for the dental model with many sharp features of high curvature. To obtain an accurate dental surface, this paper first strengthens the mesh generation and the surface registration using optimal parameters. Then, a 3D robust dental surface reconstruction is performed by iteratively executing the smallest univalue segment assimilating nucleus filtering method and the surface attraction method. Finally, virtual scanning and real scanning models are introduced to validate the accuracy of the presented method. The experiments show that our method can robustly reconstruct the 3D dental model.

3D Surface Reconstruction from Single 2D Image

The image-based 3D reconstruction technique is an important method of 3D data passive acquired in computer vision. Aimed at the limitation of traditional SFS based 3D surface reconstruction method, a combined image-based 3D surface reconstruction method is given in this paper. Firstly, the pretreatment method of gradation is applied to convert the input color image to 256 gray image; secondly, shape from shading method is applied to recover the 3D profile of object; finally, the B-spline technique is adopted to reconstruct the model of surface and improve the resolution of surface. The key techniques such as the principle and algorithm of B-spline interpolation reconstruction are analyzed, and the experimental results are also given. The single-image based 3D reconstruction method given in this paper will supply a helpful condition for the extensive used of SFS technique.

Image-based 3D surface reconstruction by combination of photometric, geometric, and real-aperture methods

An image-based 3D surface reconstruction method based on simultaneous evaluation of intensity and polarisation features (shape from photopolarimetric reflectance) and its combination with absolute depth data is introduced in this article. The proposed technique is based on the analysis of single or multiple intensity and polarisation images. To compute the surface gradients, we present a global optimisation method based on a variational framework and a local optimisation method based on solving a set of non-linear equations individually for each image pixel. These approaches are suitable for strongly non-Lambertian surfaces and those of diffuse reflectance behaviour and can also be adapted to surfaces of non-uniform albedo. We describe how independently measured absolute depth data is integrated into the shape from photopolarimetric reflectance framework in order to increase the accuracy of the 3D reconstruction result. In this context we concentrate on dense but noisy depth data obtained by depth from defocus and on sparse but accurate depth data obtained by stereo or structure from motion analysis. We show that depth from defocus information should preferentially be used for initialising the optimisation schemes for the surface gradients. For integration of sparse depth information, we suggest an optimisation scheme that simultaneously adapts the surface gradients to the measured intensity and polarisation data and to the surface slopes implied by depth differences between pairs of depth points. In principle, arbitrary sources of depth information are possible in the presented framework. Experiments on synthetic and on real-world data reveal that while depth from defocus is especially helpful for providing an initial estimate of the surface gradients and the albedo in the absence of a-priori knowledge, integration of stereo or structure from motion information significantly increases the 3D reconstruction accuracy. In our real-world experiments, we regard the scenarios of 3D reconstruction of raw forged iron surfaces in the domain of industrial quality inspection and the generation of a digital elevation model of a section of the lunar surface in the context of space-based planetary exploration.

Shape Estimation Using Polarization and Shading from Two Views

This paper presents a novel method for 3D surface reconstruction that uses polarization and shading information from two views. The method relies on polarization data acquired using a standard digital camera and a linear polarizer. Fresnel theory is used to process the raw images and to obtain initial estimates of surface normals, assuming that the reflection type is diffuse. Based on this idea, the paper presents two novel contributions to the problem of surface reconstruction. The first is a technique to enhance the surface normal estimates by incorporating shading information into the method. This is done using robust statistics to estimate how the measured pixel brightnesses depend on the surface orientation. This gives an estimate of the object material reflectance function, which is used to refine the estimates of the surface normals. The second contribution is to use the refined estimates to establish correspondence between two views of an object. To do this, a set of patches are extracted from each view and are aligned by minimizing an energy functional based on the surface normal estimates and local topographic properties. The optimum alignment parameters for different patch pairs are then used to establish stereo correspondence. This process results in an unambiguous field of surface normals, which can be integrated to recover the surface depth. Our technique is most suited to smooth, non-metallic surfaces. It complements existing stereo algorithms since it does not require salient surface features to obtain correspondences. An extensive set of experiments, yielding reconstructed objects and reflectance functions, are presented and compared to ground truth.