Implicit Surface Reconstruction Research Articles

Multi-view neural 3D reconstruction of micro- and nanostructures with atomic force microscopy

Atomic Force Microscopy (AFM) is a widely employed tool for micro- and nanoscale topographic imaging. However, conventional AFM scanning struggles to reconstruct complex 3D micro- and nanostructures precisely due to limitations such as incomplete sample topography capturing and tip-sample convolution artifacts. Here, we propose a multi-view neural-network-based framework with AFM, named MVN-AFM, which accurately reconstructs surface models of intricate micro- and nanostructures. Unlike previous 3D-AFM approaches, MVN-AFM does not depend on any specially shaped probes or costly modifications to the AFM system. To achieve this, MVN-AFM employs an iterative method to align multi-view data and eliminate AFM artifacts simultaneously. Furthermore, we apply the neural implicit surface reconstruction technique in nanotechnology and achieve improved results. Additional extensive experiments show that MVN-AFM effectively eliminates artifacts present in raw AFM images and reconstructs various micro- and nanostructures, including complex geometrical microstructures printed via two-photon lithography and nanoparticles such as poly(methyl methacrylate) (PMMA) nanospheres and zeolitic imidazolate framework-67 (ZIF-67) nanocrystals. This work presents a cost-effective tool for micro- and nanoscale 3D analysis.

Neural implicit surface reconstruction of freehand 3D ultrasound volume with geometric constraints

Three-dimensional (3D) freehand ultrasound (US) is a widely used imaging modality that allows non-invasive imaging of medical anatomy without radiation exposure. Surface reconstruction of US volume is vital to acquire the accurate anatomical structures needed for modeling, registration, and visualization. However, traditional methods cannot produce a high-quality surface due to image noise. Despite improvements in smoothness, continuity, and resolution from deep learning approaches, research on surface reconstruction in freehand 3D US is still limited. This study introduces FUNSR, a self-supervised neural implicit surface reconstruction method to learn signed distance functions (SDFs) from US volumes. In particular, FUNSR iteratively learns the SDFs by moving the 3D queries sampled around volumetric point clouds to approximate the surface, guided by two novel geometric constraints: sign consistency constraint and on-surface constraint with adversarial learning. Our approach has been thoroughly evaluated across four datasets to demonstrate its adaptability to various anatomical structures, including a hip phantom dataset, two vascular datasets and one publicly available prostate dataset. We also show that smooth and continuous representations greatly enhance the visual appearance of US data. Furthermore, we highlight the potential of our method to improve segmentation performance, and its robustness to noise distribution and motion perturbation.

PSDF: Prior-Driven Neural Implicit Surface Learning for Multi-view Reconstruction.

Surface reconstruction has traditionally relied on the Multi-View Stereo (MVS)-based pipeline, which often suffers from noisy and incomplete geometry. This is due to that although MVS has been proven to be an effective way to recover the geometry of the scenes, especially for locally detailed areas with rich textures, it struggles to deal with areas with low texture and large variations of illumination where the photometric consistency is unreliable. Recently, Neural Implicit Surface Reconstruction (NISR) combines surface rendering and volume rendering techniques and bypasses the MVS as an intermediate step, which has emerged as a promising alternative to overcome the limitations of traditional pipelines. While NISR has shown impressive results on simple scenes, it remains challenging to recover delicate geometry from uncontrolled real-world scenes which is caused by its underconstrained optimization. To this end, the framework PSDF is proposed which resorts to external geometric priors from a pretrained MVS network and internal geometric priors inherent in the NISR model to facilitate high-quality neural implicit surface learning. Specifically, the visibility-aware feature consistency loss and depth prior-assisted sampling based on external geometric priors are introduced. These proposals provide powerfully geometric consistency constraints and aid in locating surface intersection points, thereby significantly improving the accuracy and delicate reconstruction of NISR. Meanwhile, the internal prior-guided importance rendering is presented to enhance the fidelity of the reconstructed surface mesh by mitigating the biased rendering issue in NISR. Extensive experiments on Tanks and Temples datasets show that PSDF achieves state-of-the-art performance on complex uncontrolled scenes.

ANISE: Assembly-Based Neural Implicit Surface Reconstruction.

We present ANISE, a method that reconstructs a 3D shape from partial observations (images or sparse point clouds) using a part-aware neural implicit shape representation. The shape is formulated as an assembly of neural implicit functions, each representing a different part instance. In contrast to previous approaches, the prediction of this representation proceeds in a coarse-to-fine manner. Our model first reconstructs a structural arrangement of the shape in the form of geometric transformations of its part instances. Conditioned on them, the model predicts part latent codes encoding their surface geometry. Reconstructions can be obtained in two ways: (i) by directly decoding the part latent codes to part implicit functions, then combining them into the final shape; or (ii) by using part latents to retrieve similar part instances in a part database and assembling them in a single shape. We demonstrate that, when performing reconstruction by decoding part representations into implicit functions, our method achieves state-of-the-art part-aware reconstruction results from both images and sparse point clouds. When reconstructing shapes by assembling parts retrieved from a dataset, our approach significantly outperforms traditional shape retrieval methods even when significantly restricting the database size. We present our results in well-known sparse point cloud reconstruction and single-view reconstruction benchmarks.

Joint stereo 3D object detection and implicit surface reconstruction

We present a new learning-based framework S-3D-RCNN that can recover accurate object orientation in SO(3) and simultaneously predict implicit rigid shapes from stereo RGB images. For orientation estimation, in contrast to previous studies that map local appearance to observation angles, we propose a progressive approach by extracting meaningful Intermediate Geometrical Representations (IGRs). This approach features a deep model that transforms perceived intensities from one or two views to object part coordinates to achieve direct egocentric object orientation estimation in the camera coordinate system. To further achieve finer description inside 3D bounding boxes, we investigate the implicit shape estimation problem from stereo images. We model visible object surfaces by designing a point-based representation, augmenting IGRs to explicitly address the unseen surface hallucination problem. Extensive experiments validate the effectiveness of the proposed IGRs, and S-3D-RCNN achieves superior 3D scene understanding performance. We also designed new metrics on the KITTI benchmark for our evaluation of implicit shape estimation.

Three-Dimensional Surface Reconstruction from Point Clouds Using Euler’s Elastica Regularization

Euler’s elastica energy regularizer, initially employed in mathematical and physical systems, has recently garnered much attention in image processing and computer vision tasks. Due to the non-convexity, non-smoothness, and high order of its derivative, however, the term has yet to be effectively applied in 3D reconstruction. To this day, the industry is still searching for 3D reconstruction systems that are robust, accurate, efficient, and easy to use. While implicit surface reconstruction methods generally demonstrate superior robustness and flexibility, the traditional methods rely on initialization and can easily become trapped in local minima. Some low-order variational models are able to overcome these issues, but they still struggle with the reconstruction of object details. Euler’s elastica term, on the other hand, has been found to share the advantages of both the TV regularization term and the curvature regularization term. In this paper, we aim to address the problems of missing details and complex computation in implicit 3D reconstruction by efficiently using Euler’s elastica term. The main contributions of this article can be outlined in three aspects. Firstly, Euler’s elastica is introduced as a regularization term in 3D point cloud reconstruction. Secondly, a new dual algorithm is devised for the proposed model, significantly improving solution efficiency compared to the commonly used TV model. Lastly, numerical experiments conducted in 2D and 3D demonstrate the remarkable performance of Euler’s elastica in enhancing features of curved surfaces during point cloud reconstruction. The reconstructed point cloud surface adheres more closely to the initial point cloud surface when compared to the classical TV model. However, it is worth noting that Euler’s elastica exhibits a lesser capability in handling local extrema compared to the TV model.

Sat-Mesh: Learning Neural Implicit Surfaces for Multi-View Satellite Reconstruction

Automatic reconstruction of surfaces from satellite imagery is a hot topic in computer vision and photogrammetry. State-of-the-art reconstruction methods typically produce 2.5D elevation data. In contrast, we propose a one-stage method directly generating a 3D mesh model from multi-view satellite imagery. We introduce a novel Sat-Mesh approach for satellite implicit surface reconstruction: We represent the scene as a continuous signed distance function (SDF) and leverage a volume rendering framework to learn the SDF values. To address the challenges posed by lighting variations and inconsistent appearances in satellite imagery, we incorporate a latent vector in the network architecture to encode image appearances. Furthermore, we introduce a multi-view stereo constraint to enhance surface quality. This constraint minimizes the similarity between image patches to optimize the position and orientation of the SDF surface. Experimental results demonstrate that our method achieves superior visual quality and quantitative accuracy in generating mesh models. Moreover, our approach can learn seasonal variations in satellite imagery, resulting in texture mesh models with different and consistent seasonal appearances.

DiffSVR: Differentiable Neural Implicit Surface Rendering for Single-View Reconstruction with Highly Sparse Depth Prior

It is well-known that 3D shape and texture reconstruction from a single-view image is a very challenging and an ill-posed problem, especially without 3D supervision/ground truth. Existing neural implicit surface reconstruction approaches do easily get trapped in the local minima and cannot produce high-fidelity geometry and high-quality textures (and rendered images) under single-view setting, even with provided highly sparse depth prior. In this paper, we propose a new self-supervised learning method DiffSVR that represents a complicated surface as a new depth-aware occupancy function (DOF) and utilizes an end-to-end differentiable surface rendering paradigm to optimize the neural DOF field relying only on single-view image with highly sparse depth information. The developed surface-aware sampling, occupancy self-labeling, and differentiable surface rendering with inverse computation techniques can enhance both the neural implicit surface reconstruction and the neural renderer. The extensive experiments and comparisons on two real-world benchmark datasets (e.g., DTU and KITTI) demonstrate that our approach not only numerically outperforms the current state-of-the-art methods by a large margin, but also produces surface mesh model with qualitatively better geometric details and more accurate textures, as well as exhibits good performance on generalizability and flexibility. The code and data are available at https://github.com/akomarichev/DiffSVR.

Point normal orientation and surface reconstruction by incorporating isovalue constraints to Poisson equation

Oriented normals are common pre-requisites for many geometric algorithms based on point clouds, such as Poisson surface reconstruction. However, it is not trivial to obtain a consistent orientation. In this work, we bridge orientation and reconstruction in the implicit space and propose a novel approach to orient point cloud normals by incorporating isovalue constraints to the Poisson equation. In implicit surface reconstruction, the reconstructed shape is represented as an isosurface of an implicit function defined in the ambient space. Therefore, when such a surface is reconstructed from a set of sample points, the implicit function values at the points should be close to the isovalue corresponding to the surface. Based on this observation and the Poisson equation, we propose an optimization formulation that combines isovalue constraints with local consistency requirements for normals. We optimize normals and implicit functions simultaneously and solve for a globally consistent orientation. Thanks to the sparsity of the linear system, our method can work on an average laptop with reasonable computational time. Experiments show that our method can achieve high performance in non-uniform and noisy data and manage varying sampling densities, artifacts, multiple connected components, and nested surfaces. The source code is available at https://github.com/Submanifold/IsoConstraints.

Interactive Restoration of Three-Dimensional Implicit Surface with營rregular燩arts

Implicit surface generation based on the interpolation of surface points is one of the well-known modeling methods in the area of computer graphics. Several methods for the implicit surface reconstruction from surface points have been proposed on the basis of radial basis functions, a weighted sum of local functions, splines, wavelets, and combinations of them. However, if the surface points contain errors or are sparsely distributed, irregular components, such as curvature-shaped redundant bulges and unexpectedly generated high-frequency components, are commonly seen. This paper presents a framework for restoring irregular components generated on and around surfaces. Users are assumed to specify local masks that cover irregular components and parameters that determine the degree of restoration. The algorithm in this paper removes the defects based on the user-specific masks and parameters. Experiments have shown that the proposed methods can effectively remove redundant protrusions and jaggy noise.

Local offset point cloud transformer based implicit surface reconstruction

AbstractImplicit neural representations, such as MLP, can well recover the topology of watertight object. However, MLP fails to recover geometric details of watertight object and complicated topology due to dealing with point cloud in a point‐wise manner. In this paper, we propose a point cloud transformer called local offset point cloud transformer (LOPCT) as a feature fusion module. Before using MLP to learn the implicit function, the input point cloud is first fed into the local offset transformer, which adaptively learns the dependency of the local point cloud and obtains the enhanced features of each point. The feature‐enhanced point cloud is then fed into the MLP to recover the geometric details and sharp features of watertight object and complex topology. Extensive reconstruction experiments of watertight object and complex topology demonstrate that our method achieves comparable or better results than others in terms of recovering sharp features and geometric details. In addition, experiments on watertight objects demonstrate the robustness of our method in terms of average result.

Topology-controllable Implicit Surface Reconstruction Based on Persistent Homology

In geometric design, reconstructing an implicit surface with high-quality geometry and expected topology from point clouds has been a challenging problem. However, traditional topological invariants, such as Betti numbers, are not easily used in controlling topology in implicit surface reconstruction. Persistent homology provides a quantitative measurement, i.e. , persistence diagram (PD), and allows tracking of pairs of key points that generate and destroy topological holes. In this study, we propose a topology-controllable implicit surface reconstruction method from point clouds based on persistent homology. Specifically, given a point cloud with normals, the signed distance field is first constructed, and then a B-spline function represented distance function is generated by fitting the signed distance values through progressive iterative approximation. By designing a topological target function using the persistent pairs in PDs according to topological priors, the control coefficients of the B-spline function are optimized to extract an implicit surface, i.e. , an iso-surface of the B-spline function, with expected topology. Experiments show that the proposed method can reconstruct surfaces with higher topological quality than other reconstruction methods. Moreover, the proposed method can be used to edit the topology of an implicit surface. • The topology-controllable framework for implicit B-spline surface reconstruction improves geometric quality and ensures the desired topology of the extracted surface. • Principles of designing the topological target function for implicit surface reconstruction are proposed. • Reconstruction on sparse point clouds and the application of topology editing are explored.

Implicit Surface Reconstruction via RBF Interpolation: A Review

Background: Implicit surface is a kind of surface modeling tool, which is widely used in point cloud reconstruction, deformation and fusion due to its advantages of good smoothness and Boolean operation. The most typical method is the surface reconstruction with Radial Basis Functions (RBF) under normal constraints. RBF has become one of the main methods of point cloud fitting because it has a strong mathematical foundation, an advantage of computation simplicity, and the ability of processing nonuniform points. Objective: Techniques and patents of implicit surface reconstruction interpolation with RBF are surveyed. Theory, algorithm, and application are discussed to provide a comprehensive summary for implicit surface reconstruction in RBF and Hermite Radial Basis Functions (HRBF) interpolation. Methods: RBF implicit surface reconstruction interpolation can be divided into RBF interpolation under the constraints of points and HRBF interpolation under the constraints of points and corresponding normals. Results: A total of 125 articles were reviewed, in which more than 30% were related to RBF in the last decade. The continuity properties and application fields of the popular global supported radial basis functions and compactly supported radial basis functions are analyzed. Different methods of RBF and HRBF implicit surface reconstruction are evaluated, and the challenges of these methods are discussed. Conclusion: In future work, implicit surface reconstruction via RBF and HRBF should be further studied in fitting accuracy, computation speed, and other fundamental problems. In addition, it is a more challenging but valuable research direction to construct a new RBF with both compact support and improved fitting accuracy.

Mesh Processing for Snapping Feature Points and Polylines in Orebody Modeling

The 3D refinement modeling of the orebody provides an important guarantee for the estimation of the resources and reserves of an ore deposit. Implicit modeling techniques can effectively improve the efficiency of orebody modeling and facilitate the dynamic updating of the model. However, due to the problems of ambiguity and missing features during implicit surface interpolation and implicit surface reconstruction, the mesh models of orebodies obtained by means of implicit modeling techniques do not easily snap to the geological feature points and feature polylines obtained based on geological sampling data. In essence, all models are inaccurate, but geological sampling data are very useful and valuable, which should be accurately and effectively involved in the orebody modeling process. This would help to improve the reliability of resource estimation and mining design. The main contribution of this paper is the proposal of a method for accurately snapping orebody features after implicit modeling. This method enables the orebody model to snap accurately to the geological feature points and feature polylines and realizes the accurate clipping of the model boundary. We tested the method with real geological datasets. The results showed that the method is applicable and effective when the geological feature points and feature polylines are close to those of the orebody mesh model and the shape trend changes little, and the model can thus meet the practical application requirements of mines.

Implicit surface reconstruction based on a new interpolation/approximation radial basis function

Interpolation and approximation radial basis function (RBF) has been widely used in the implicit surface reconstruction. However, a simple interpolation or approximation method may either have unwanted oscillation around the scattered points, or have low accuracy at scattered points. In this paper, we construct a new representation of RBF, which unites interpolation and approximation. A further adaptive algorithm to determine interpolation and approximation is presented. With the new representation, we can develop quasi-interpolation method from an approximation method into a method combining interpolation and approximation. And we can determine interpolation or approximation at each of scattered points without solving large linear systems. Experiments show that the given method is precision-controlled and fast in implicit surface reconstruction.

The relaxed implicit randomized algebraic reconstruction technique for curve and surface reconstruction

Implicit curve and surface reconstruction has wide applications in computer graphics, which attracts great attention of many researchers. In this paper, we propose the relaxed implicit randomized algebraic reconstruction technique (RIR-ART) for curve and surface reconstruction. The RIR-ART method generates a sequence of curves and surfaces by adjusting the control coefficients with the relaxation parameters iteratively. It is proved that the sequence of curves and surfaces generated by the RIR-ART method with the suitable relaxation parameters converge to the least-norm results in terms of the mean square error. The RIR-ART method can eliminate the extra zero-level sets effectively during its iterative process, and converge faster than the implicit progressive-iterative approximation method (Hamza et al. 2020). Numerical examples present the efficiency and effectiveness of the proposed method.

Learning modified indicator functions for surface reconstruction

Surface reconstruction is a fundamental problem in 3D graphics. In this paper, we propose a learning-based approach for implicit surface reconstruction from raw point clouds without normals. Our method is inspired by Gauss Lemma in potential energy theory, which gives an explicit integral formula for the indicator functions. We design a novel deep neural network to perform surface integral and learn the modified indicator functions from un-oriented and noisy point clouds. We concatenate features with different scales for accurate point-wise contributions to the integral. Moreover, we propose a novel Surface Element Feature Extractor to learn local shape properties. Experiments show that our method generates smooth surfaces with high normal consistency from point clouds with different noise scales and achieves state-of-the-art reconstruction performance compared with current data-driven and non-data-driven approaches.

Repair of Geological Models Based on Multiple Material Marching Cubes

In this paper, we present a multi-domain implicit surface reconstruction algorithm for geological modeling based on the labeling of voxel points. The improved algorithm sets a label for each voxel point to represent the type of its geological domain and then obtains all the voxel points in the void areas. After that, the improved algorithm modifies the labels of the voxel points in the void areas and finally reconstructs the geological models through the Multiple Material Marching Cubes (M3C) algorithm. The improved algorithm solves the problems of some unexpected overlaps and voids in geological modeling by setting and modifying the labels of the voxel points. Our key contribution is proposing a labeling processing method to repair the overlap and void defects generated in the geological modeling and realizing the improved M3C algorithm. The experimental results of some geological models show the performance of the improved method. Compared with the original method, the improved method can repair the overlap and void defects in geological modeling to ensure the raw structural adjacency relationships of the geological bodies.

Orienting unorganized points and extracting isosurface for implicit surface reconstruction

Orienting unorganized points and extracting isosurface for implicit surface reconstruction

Progressive Discrete Domains for Implicit Surface Reconstruction

AbstractMany global implicit surface reconstruction algorithms formulate the problem as a volumetric energy minimization, trading data fitting for geometric regularization. As a result, the output surfaces may be located arbitrarily far away from the input samples. This is amplified when considering i) strong regularization terms, ii) sparsely distributed samples or iii) missing data. This breaks the strong assumption commonly used by popular octree‐based and triangulation‐based approaches that the output surface should be located near the input samples. As these approaches refine during a pre‐process, their cells near the input samples, the implicit solver deals with a domain discretization not fully adapted to the final isosurface. We relax this assumption and propose a progressive coarse‐to‐fine approach that jointly refines the implicit function and its representation domain, through iterating solver, optimization and refinement steps applied to a 3D Delaunay triangulation. There are several advantages to this approach: the discretized domain is adapted near the isosurface and optimized to improve both the solver conditioning and the quality of the output surface mesh contoured via marching tetrahedra.