Signed Distance Function Research Articles

Learning neural implicit surfaces with local probability standard variance

AbstractReconstructing geometric shapes from sparse multiview has always been a challenging task. With the development of neural implicit surfaces, geometry‐based volume rendering surface reconstruction methods have been proven to be able to reconstruct high‐quality surfaces. However, existing geometry‐based reconstruction methods completely associate volume density with signed distance function or unsigned distance function, resulting in the same volume density peak that can only be reconstructed near the object surface. When there are transparent surfaces in the scene, existing methods prioritize the reconstruction of opaque surfaces, neglecting the reconstruction of transparent surfaces, which is disadvantageous when reconstructing real scenes. To solve this problem, we introduce local probability standard variance, which calculates volume density together with signed distance function. In this way, it can reconstruct the volume density that matches the transparency characteristics of the object surface. The method can reconstruct the surface of transparent objects, and experiments on two transparent surface datasets show that the method performs better.

Development of convolutional neural network-based surrogate model for three-dimensional vacuum plume prediction via direct simulation Monte Carlo method

The vacuum plume phenomenon encountered during lunar exploration missions poses significant challenges, such as impingement forces, heat fluxes, and spacecraft contamination. Numerical simulation represents the predominant method for evaluating the impacts of vacuum plumes. However, the conventional direct simulation Monte Carlo (DSMC) method, despite being the standard, is notably time-consuming and impractical for real-time analysis. Addressing this limitation, our research explores deep learning, specifically convolutional neural networks (CNN), for the efficient prediction of vacuum plume dynamics. We introduce a novel CNN-based DSMC method (CNN-DSMC-3D), leveraging a dataset obtained from three-dimensional DSMC simulations. This approach translates the spacecraft's shape and boundary conditions into a signed distance function and an identifier matrix. The CNN-DSMC-3D method effectively predicts the vacuum plume field, aligning closely with DSMC results across various lunar surface conditions. Crucially, the CNN-DSMC-3D method achieves a speed increase in four to six orders of magnitude over the conventional DSMC method, demonstrating substantial potential for real-time aerospace applications and offering a paradigm shift in the simulation of lunar landing scenarios.

A multiphysics coupling framework for exascale simulation of fracture evolution in subsurface energy applications

Predicting the evolution of fractured media is challenging due to coupled thermal, hydrological, chemical and mechanical processes that occur over a broad range of spatial scales, from the microscopic pore scale to field scale. We present a software framework and scientific workflow that couples the pore scale flow and reactive transport simulator Chombo-Crunch with the field scale geomechanics solver in GEOS to simulate fracture evolution in subsurface fluid-rock systems. This new multiphysics coupling capability comprises several novel features. An HDF5 data schema for coupling fracture positions between the two codes is employed and leverages the coarse resolution of the GEOS mechanics solver which limits the size of data coupled, and is, thus, not taxed by data resulting from the high resolution pore scale Chombo-Crunch solver. The coupling framework requires tracking of both before and after coarse nodal positions in GEOS as well as the resolved embedded boundary in Chombo-Crunch. We accomplished this by developing an approach to geometry generation that tracks the fracture interface between the two different methodologies. The GEOS quadrilateral mesh is converted to triangles which are organized into bins and an accessible tree structure; the nodes are then mapped to the Chombo representation using a continuous signed distance function that determines locations inside, on and outside of the fracture boundary. The GEOS positions are retained in memory on the Chombo-Crunch side of the coupling. The time stepping cadence for coupled multiphysics processes of flow, transport, reactions and mechanics is stable and demonstrates temporal reach to experimental time scales. The approach is validated by demonstration of 9 days of simulated time of a core flood experiment with fracture aperture evolution due to invasion of carbonated brine in wellbore-cement and sandstone. We also demonstrate usage of exascale computing resources by simulating a high resolution version of the validation problem on OLCF Frontier.

Geom-DeepONet: A point-cloud-based deep operator network for field predictions on 3D parameterized geometries

Modern digital engineering design process commonly involves expensive repeated simulations on varying three-dimensional (3D) geometries. The efficient prediction capability of neural networks (NNs) makes them a suitable surrogate to provide design insights. Nevertheless, few available NNs can handle solution prediction on varying 3D shapes. We present a novel deep operator network (DeepONet) variant called Geom-DeepONet, which encodes parameterized 3D geometries and predicts full-field solutions on an arbitrary number of nodes. To the best of the authors’ knowledge, this is the first attempt in the literature and is our primary novelty. In addition to expressing shapes using mesh coordinates, the signed distance function for each node is evaluated and used to augment the inputs to the trunk network of the Geom-DeepONet, thereby capturing both explicit and implicit representations of the 3D shapes. The powerful geometric encoding capability of a sinusoidal representation network (SIREN) is also exploited by replacing the classical feedforward neural networks in the trunk with SIREN. Additional data fusion between the branch and trunk networks is introduced by an element-wise product. A numerical benchmark was conducted to compare Geom-DeepONet to PointNet and vanilla DeepONet, where results show that our architecture trains fast with a small memory footprint and yields the most accurate results among the three. Results show a much lower generalization error of our architecture on unseen dissimilar designs than vanilla DeepONet. Training the model on 2500 cuboid designs is done within 2 h on a single A100 GPU while generating the training dataset through finite element simulations took about 27 h. Once trained, the model can predict vector fields and a single prediction can be over 105 times faster than its corresponding implicit finite element simulation if the mesh is large. The ability of the proposed model to perform efficient and accurate field predictions on variable 3D geometries, especially those discretized by different nodes and elements, makes it a valuable tool for preliminary performance evaluation and design optimizations and is the most significant contribution of the current work.

Depth Supervised Neural Surface Reconstruction from Airborne Imagery

Abstract. While originally developed for novel view synthesis, Neural Radiance Fields (NeRFs) have recently emerged as an alternative to multi-view stereo (MVS). Triggered by a manifold of research activities, promising results have been gained especially for textureless, transparent, and reflecting surfaces, while such scenarios remain challenging for traditional MVS-based approaches. However, most of these investigations focus on close-range scenarios, with studies for airborne scenarios still missing. For this task, NeRFs face potential difficulties at areas of low image redundancy and weak data evidence, as often found in street canyons, facades or building shadows. Furthermore, training such networks is computationally expensive. Thus, the aim of our work is twofold: First, we investigate the applicability of NeRFs for aerial image blocks representing different characteristics like nadir-only, oblique and high-resolution imagery. Second, during these investigations we demonstrate the benefit of integrating depth priors from tie-point measures, which are provided during presupposed Bundle Block Adjustment. Our work is based on the state-of-the-art framework VolSDF, which models 3D scenes by signed distance functions (SDFs), since this is more applicable for surface reconstruction compared to the standard volumetric representation in vanilla NeRFs. For evaluation, the NeRF-based reconstructions are compared to results of a publicly available benchmark dataset for airborne images.

Meta-learning integrated with Signed Distance Functions to Enhance Scene Text Segmentation

Meta-learning integrated with Signed Distance Functions to Enhance Scene Text Segmentation

A brain subcortical segmentation tool based on anatomy attentional fusion network for developing macaques

Magnetic Resonance Imaging (MRI) plays a pivotal role in the accurate measurement of brain subcortical structures in macaques, which is crucial for unraveling the complexities of brain structure and function, thereby enhancing our understanding of neurodegenerative diseases and brain development. However, due to significant differences in brain size, structure, and imaging characteristics between humans and macaques, computational tools developed for human neuroimaging studies often encounter obstacles when applied to macaques. In this context, we propose an Anatomy Attentional Fusion Network (AAF-Net), which integrates multimodal MRI data with anatomical constraints in a multi-scale framework to address the challenges posed by the dynamic development, regional heterogeneity, and age-related size variations of the juvenile macaque brain, thus achieving precise subcortical segmentation. Specifically, we generate a Signed Distance Map (SDM) based on the initial rough segmentation of the subcortical region by a network as an anatomical constraint, providing comprehensive information on positions, structures, and morphology. Then we construct AAF-Net to fully fuse the SDM anatomical constraints and multimodal images for refined segmentation. To thoroughly evaluate the performance of our proposed tool, over 700 macaque MRIs from 19 datasets were used in this study. Specifically, we employed two manually labeled longitudinal macaque datasets to develop the tool and complete four-fold cross-validations. Furthermore, we incorporated various external datasets to demonstrate the proposed tool’s generalization capabilities and promise in brain development research. We have made this tool available as an open-source resource at https://github.com/TaoZhong11/Macaque_subcortical_segmentation for direct application.

Enhancing endoscopic scene reconstruction with color-aware inverse rendering through neural SDF and radiance fields.

Virtual surgical training is crucial for enhancing minimally invasive surgical skills. Traditional geometric reconstruction methods based on medical CT/MRI images often fall short in providing color information, which is typically generated through pseudo-coloring or artistic rendering. To simultaneously reconstruct both the geometric shape and appearance information of organs, we propose a novel organ model reconstruction network called Endoscope-NeSRF. This network jointly leverages neural radiance fields and Signed Distance Function (SDF) to reconstruct a textured geometric model of the organ of interest from multi-view photometric images acquired by an endoscope. The prior knowledge of the inverse correlation between the distance from the light source to the object and the radiance improves the real physical properties of the organ. The dilated mask further refines the appearance and geometry at the organ's edges. We also proposed a highlight adaptive optimization strategy to remove highlights caused by the light source during the acquisition process, thereby preventing the reconstruction results in areas previously affected by highlights from turning white. Finally, the real-time realistic rendering of the organ model is achieved by combining the inverse rendering and Bidirectional Reflectance Distribution Function (BRDF) rendering methods. Experimental results show that our method closely matches the Instant-NGP method in appearance reconstruction, outperforming other state-of-the-art methods, and stands as the superior method in terms of geometric reconstruction. Our method obtained a detailed geometric model and realistic appearance, providing a realistic visual sense for virtual surgical simulation, which is important for medical training.

An endoscopic chisel: intraoperative imaging carves 3D anatomical models.

Preoperative imaging plays a pivotal role in sinus surgery where CTs offer patient-specific insights of complex anatomy, enabling real-time intraoperative navigation to complement endoscopy imaging. However, surgery elicits anatomical changes not represented in the preoperative model, generating an inaccurate basis for navigation during surgery progression. We propose a first vision-based approach to update the preoperative 3D anatomical model leveraging intraoperative endoscopic video for navigated sinus surgery where relative camera poses are known. We rely on comparisons of intraoperative monocular depth estimates and preoperative depth renders to identify modified regions. The new depths are integrated in these regions through volumetric fusion in a truncated signed distance function representation to generate an intraoperative 3D model that reflects tissue manipulation RESULTS: We quantitatively evaluate our approach by sequentially updating models for a five-step surgical progression in an ex vivo specimen. We compute the error between correspondences from the updated model and ground-truth intraoperative CT in the region of anatomical modification. The resulting models show a decrease in error during surgical progression as opposed to increasing when no update is employed. Our findings suggest that preoperative 3D anatomical models can be updated using intraoperative endoscopy video in navigated sinus surgery. Future work will investigate improvements to monocular depth estimation as well as removing the need for external navigation systems. The resulting ability to continuously update the patient model may provide surgeons with a more precise understanding of the current anatomical state and paves the way toward a digital twin paradigm for sinus surgery.

M-NeuS: Volume rendering based surface reconstruction and material estimation

Although significant advances have been made in the field of multi-view 3D reconstruction using implicit neural field-based methods, existing reconstruction methods overlook the estimation of the material information (e.g. the base color, albedo, roughness, and metallic) during the learning process. In this paper, we propose a novel differentiable rendering framework, named as material NueS (M-NeuS), for simultaneously achieving precise surface reconstruction and competitive material estimation. For surface reconstruction, we perform multi-view geometry optimization by proposing an enhanced-low-to-high frequency encoding registration strategy (EFERS) and a second-order interpolated signed distance function (SI-SDF) for precise details and outline reconstruction. For material estimation, inspired by the NeuS, we first propose a volume-rendering-based material estimation strategy (VMES) to estimate the base color, albedo, roughness, and metallic accurately. And then, different from most material estimation methods that need ground-truth geometric priors, we use the geometry information reconstructed in the surface reconstruction stage and the directions of incidence from different viewpoints to model a neural light field, which can extract the lighting information from image observations. Next, the extracted lighting and the estimated base color, albedo, roughness, and metallic are optimized by the physics-based rendering equation. Extensive experiments demonstrate that our M-NeuS can not only reconstruct more precise geometry surface than existing state-of-the-art (SOTA) reconstruction methods but also can estimate competitive material information: the base color, albedo, roughness, and metallic.

A deep learning approach for hydrofoil optimization of tidal turbines

Hydrofoil optimization plays a crucial role in improving the hydrodynamic performance of tidal turbines. However, the high-dimensional representations in hydrofoil design space require significant Computational Fluid Dynamics (CFD) simulations, increasing computational load and time expenditure. This paper proposes a hydrofoil optimization framework based on deep learning to achieve an effective mapping between design parameters, pressure fields, and hydrodynamic characteristics. The designed hydrofoil represented using the Signed Distance Function (SDF) is input to Convolutional Neural Networks (CNN) for fast prediction of hydrodynamic coefficients and pressure field. Due to the effective integration of the prediction model and genetic optimization algorithm, this framework can generate a large number of smooth hydrofoils and rapidly predict their hydrodynamic performance to achieve efficient optimization of hydrofoils. The optimized hydrofoil shapes obtained based on the above framework exhibit a higher lift-to-drag ratio than common hydrofoils. Furthermore, the optimized hydrofoils are applied to design tidal energy turbine blades. The simulation results demonstrate that the hydrodynamic performance of tidal turbines can be effectively improved through hydrofoil design optimization. The proposed framework enhances the design efficiency of the tidal turbine rotor and provides turbine hydrofoils with higher power coefficients.

SRTRP-Net: A multi-task learning network for segmentation and prediction of stereotactic radiosurgery treatment response in brain metastases

Before the Stereotactic Radiosurgery (SRS) treatment, it is of great clinical significance to avoid secondary genetic damage and guide the personalized treatment plans for patients with brain metastases (BM) by predicting the response to SRS treatment of brain metastatic lesions. Thus, we developed a multi-task learning model termed SRTRP-Net to provide prior knowledge of BM ROI and predict the SRS treatment response of the lesion. In dual-encoder tumor segmentation Network (DTS-Net), two parallel encoders encode the original and mirrored multi-modal MRI images. The differences in the dual-encoder features between foreground and background are enhanced by the symmetrical visual difference block (SVDB). In the bottom layer of the encoder, a transformer is used to extract local contextual features in the spatial and depth dimensions of low-resolution images. Then, the decoder of DTS-Net provides the prior knowledge for predicting the response to SRS treatment by performing BM segmentation. SRS response prediction network (SRP-Net) directly utilizes shared multi-modal MRI features weighted by the signed distance map (SDM) of the masks. The bidirectional multi-dimensional feature fusion module (BMDF) fuses the shared features and the clinical text information features to obtain comprehensive tumor information for characterizing tumors and predicting SRS treatment response. Experiments based on internal and external clinical datasets have shown that SRTRP-Net achieves comparable or better results. We believe that SRTRP-Net can help clinicians accurately develop personalized first-time treatment regimens for BM patients and improve their survival.

Triple-task mutual consistency for semi-supervised 3D medical image segmentation

Semi-supervised deep learning algorithm is an effective means of medical image segmentation. Among these methods, multi-task learning with consistency regularization has achieved outstanding results. However, most of the existing methods usually simply embed the Signed Distance Map (SDM) task into the network, which underestimates the potential ability of SDM in edge awareness and leads to excessive dependence between tasks. In this work, we propose a novel triple-task mutual consistency (TTMC) framework to enhance shape and edge awareness capabilities, and overcome the task dependence problem underestimated in previous work. Specifically, we innovatively construct the Signed Attention Map (SAM), a novel fusion image with attention mechanism, and use it as an auxiliary task for segmentation to enhance the edge awareness ability. Then we implement a triple-task deep network, which jointly predicts the voxel-wise classification map, the Signed Distance Map and the Signed Attention Map. In our proposed framework, an optimized differentiable transformation layer associates SDM with voxel-wise classification map and SAM prediction, while task-level consistency regularization utilizes unlabeled data in an unsupervised manner. Evaluated on the public Left Atrium dataset and NIH Pancreas dataset, our proposed framework achieves significant performance gains by effectively utilizing unlabeled data, outperforming recent state-of-the-art semi-supervised segmentation methods. Code is available at https://github.com/Saocent/TTMC.

NeUDF: Learning Neural Unsigned Distance Fields With Volume Rendering.

Multi-view shape reconstruction has achieved impressive progresses thanks to the latest advances in the neural implicit rendering. However, existing methods based on signed distance function (SDF) are limited to closed surfaces, failing to reconstruct a wide range of real-world objects that contain open-surface structures. In this work, we introduce a new neural rendering framework, coded NeUDF, that can reconstruct surfaces with arbitrary topologies solely from multi-view supervision. To gain the flexibility of representing arbitrary surfaces, NeUDF leverages the unsigned distance function (UDF) as surface representation. While a naive extension of SDF-based neural renderer cannot scale to UDF, we formalize the rules of neural volume rendering for open surface reconstruction (e.g., self-consistent, unbiased, occlusion-aware), and derive a dedicated rendering weight function specially tailored for UDF. Furthermore, to cope with open surface rendering, where the in/out test is no longer valid, we present a dedicated normal regularization strategy to resolve the surface orientation ambiguity. We extensively evaluate our method over a number of challenging datasets, including two typical open surface datasets MGN (Bhatnagar et al., 2019) and Deep Fashion 3D (Zhu et al., 2020). Experimental results demonstrate that NeUDF can significantly outperform the state-of-the-art methods in the task of multi-view surface reconstruction, especially for the complex shapes with open boundaries.

Convolutional Neural Networks-Based For Predicting Aerodynamic Coefficient Of Airfoils At Ultra-Low Reynolds Number

Many applications, including airplane design, wind turbines, and heat transmission, use symmetric or asymmetric airfoils. Engineers employ these airfoil shapes to optimize performance and efficiency. Each airfoil has a unique set of aerodynamic coefficients that must be calculated to maximize the airfoil design. Engineers utilize numerous ways to calculate coefficients, such as lift and drag. One of the methods is the prediction method, which effectively reduces time and cost. This study's training dataset is obtained from particle-based numerical computation using the Lattice Boltzmann Method (LBM). Then, Convolutional Neural Networks (CNN) are used as a prediction method to get the aerodynamic coefficients of airfoils for lift and drag based on two different Reynolds numbers. In CNN, airfoil geometry representation is essential. The Signed Distance Function (SDF) was used to convert airfoil geometry into RGB pictures. On the other hand, the SDF method cannot explain different flow conditions; in this case, it is represented by the Reynolds number (Re). Therefore, we propose a Text-based Watermarking Method (TWM) to differentiate between Re = 500 and Re = 1000. Each airfoil representation was trained and tested to generate each prediction model using a modified LeNet-5. The computation results show that using CNN with TWM on SDF to define the Reynolds numbers could predict the lift and drag coefficients with varying angles of attack. Future research can focus on generalizations to different aerodynamic aspects and practical applications in complex scenarios.

A novel quasi-smooth tetrahedral numerical manifold method and its application in topology optimization based on parameterized level-set method

In this paper, a novel quasi-smooth tetrahedral numerical manifold method (NMM) and its two-dimensional (2D) counterpart are proposed. A new topology optimization method is established by combining the quasi-smooth manifold element (QSME) with the parameterized level set method (PLSM). The QSME introduces an innovative displacement function characterized by high accuracy and high-order continuity, effectively addressing the “linear dependence” (LD) issue inherent in traditional high-order NMM. To integrate QSME and PLSM, the corresponding optimization formulations and sensitivity analyses are provided. In order to fully utilize advantages of this novel quasi-smooth NMM and the PLSM, an element subdivision technique based on model recognition is proposed to accurately capture the physical boundaries. Additionally, a volume fraction update method based on element refinement is proposed. Taking advantage of the characteristics of the PLSM, a structure visualization method based on the sign distance function is developed to accurately describe curve boundary. This method allows for precise visualization of optimized structures. This study verifies high efficiency of the QSME-based PLSM for minimum compliance topology optimization problems in both 2D and 3D structures. Some representative structural optimization examples are tested to demonstrate effectiveness of the proposed method in both 2D and 3D problems, especially in complex design domain.

QuickCSGModeling: Quick CSG Operations Based on Fusing Signed Distance Fields for VR Modeling

The latest advancements in Virtual Reality (VR) enable the creation of 3D models within a holographic immersive simulation environment. In this article, we create QuickCSGModeling , a user-friendly mid-air interactive modeling system. We first prepare a dataset consisting of diverse components and precompute the discrete signed distance function (SDF) for each component. During the modeling phase, users can freely design complicated shapes with a pair of VR controllers. Based on the discrete SDF representation, any CSG-like operation (union, intersection, and subtraction) can be performed voxel-wisely. Also, we maintain a single dynamic SDF for the whole scene, whose zero-level set surface exactly encodes the most recent constructed shape. Both SDF fusion and surface extraction are implemented via GPU for a smooth user experience. A total of 34 volunteers were asked to create their favorite models using QuickCSGModeling. With a simple training, most of them can create a fascinating shape or even a descriptive scene quickly. We also discuss how to extend our system to create articulated models with hinges, where an adaptive cube subdivision has to be enforced to improve the reconstruction accuracy around the hinge part, followed by a Dual Contouring-based surface extraction. 1

High-fidelity 3D reconstruction of plants using Neural Radiance Fields

Accurate reconstruction of plant phenotypes plays a key role in optimizing sustainable farming practices in the field of Precision Agriculture (PA). Currently, optical sensor-based approaches dominate the field, but the need for high-fidelity 3D reconstruction of crops and plants in unstructured agricultural environments remains challenging. Recently, a promising development has emerged in the form of Neural Radiance Fields (NeRF), a novel method that utilizes neural density fields. This technology has shown impressive performance in various novel vision synthesis tasks, but has remained relatively unexplored in the agricultural context. In our study, we focus on two fundamental tasks within plant phenotyping: (1) the synthesis of 2D novel-view images and (2) the 3D reconstruction of crop and plant models. We explore the world of NeRF, in particular two state-of-the-art (SOTA) methods: Instant-NGP, which excels in generating high-quality images with impressive training and inference speed, and Instant-NSR, which improves the reconstructed geometry by incorporating the Signed Distance Function (SDF) during training. In particular, we present a novel plant phenotype dataset comprising real plant images from production environments. This dataset is a first-of-its-kind initiative aimed at comprehensively exploring the advantages and limitations of NeRF in agricultural contexts. Our experimental results show that NeRF demonstrates commendable performance in the synthesis of novel-view images and is able to achieve reconstruction results that are competitive with Reality Capture, a leading commercial software for 3D Multi-View Stereo (MVS)-based reconstruction. Moreover, our study also highlights certain drawbacks of NeRF, including relatively slow training speeds, performance limitations in cases of insufficient sampling, and challenges in obtaining geometry quality in complex setups. In conclusion, NeRF introduces a new paradigm in plant phenotyping, providing a powerful tool capable of generating multiple representations, such as multi-view images, point cloud and mesh, from a single process.

Enhanced conservative phase field method for moving contact line problems

AbstractThe traditional Allen–Cahn phase field model doesn't conserve mass and is mostly used in solidification microstructure formation. However, a recently modified Allen–Cahn phase field model has riveted the attention of the academic community. It was obtained by subtracting the curvature‐driven flow term from the advective Allen–Cahn phase field model, and thus improves the boundedness of the phase field. More recently, the model has been further refined with the recovered signed distance function to compute interface normal vectors. This paper develops a three dimensional phase field model, based on the abovementioned Allen–Cahn phase field model. The model was discretized using a finite difference method on a half‐staggered grid. More important, interfacial tension was expressed in a potential form. The model was tested against a number of cases and was applied to impacts in various conditions. Besides, the model was parallelized using the shared memory parallelism, OpenMP, to facilitate computation.

HISR: Hybrid Implicit Surface Representation for Photorealistic 3D Human Reconstruction

Neural reconstruction and rendering strategies have demonstrated state-of-the-art performances due, in part, to their ability to preserve high level shape details. Existing approaches, however, either represent objects as implicit surface functions or neural volumes and still struggle to recover shapes with heterogeneous materials, in particular human skin, hair or clothes. To this aim, we present a new hybrid implicit surface representation to model human shapes. This representation is composed of two surface layers that represent opaque and translucent regions on the clothed human body. We segment different regions automatically using visual cues and learn to reconstruct two signed distance functions (SDFs). We perform surface-based rendering on opaque regions (e.g., body, face, clothes) to preserve high-fidelity surface normals and volume rendering on translucent regions (e.g., hair). Experiments demonstrate that our approach obtains state-of-the-art results on 3D human reconstructions, and also shows competitive performances on other objects.