Signed Distance Field Research Articles

Signed distance field framework for unified DEM modeling of granular media with arbitrary particle shapes

Signed distance field framework for unified DEM modeling of granular media with arbitrary particle shapes

EndoBeams.jl: A Julia finite element package for beam-to-surface contact problems in cardiovascular mechanics

The increasing use of mini-invasive and endovascular surgical techniques is at the origin of the pressing need for computational models to support planning and training. Several implantable devices have a wire-like structure, which can be modelled using beam elements. Our objective is to create an efficient Finite Element (FE) modelling framework for such devices. For that, we developed the EndoBeams.jl package, written exclusively in Julia, for the numerical simulation of contact interactions between wire-like structures and rigid surfaces. The package is based on a 3D FE corotational formulation for frictional contact dynamics of beams. The rigid target surface is described implicitly using a signed distance field, predefined in a volumetric grid. Since the main objective behind this package is to find the best compromise between computational speed and code readability, the algorithm, originally in Matlab, was translated and optimised in Julia, a programming language designed to combine the performance of low-level languages with the productivity of high-level ones. To evaluate the robustness, a set of tests were conducted to compare the simulation results and computational time of EndoBeams.jl against literature data, the original Matlab code and the commercial software Abaqus. The tests proved the accuracy of the underlying beam-to-surface formulation and showed the drastic performance improvement of the Julia code with respect to the original one. EndoBeams.jl is also slightly faster than Abaqus. Finally, as a proof of concept in cardiovascular medicine, a further example is shown where the deployment of a braided stent is simulated within an idealised artery.

Real time contact surface prediction for grasping with complex geometries

The prediction of the contact areas for grasping different objects presents numerous problems with difficult and, most of the times, slow solutions. Common approaches simplify either the gripper or the shape of the object. This work integrates a fast collision detection system to the grasping process which approximates contact surfaces between arbitrarily complex grippers and grasped objects in realtime. Our collision detection method is based on the Voxelmap–Pointshell algorithm, which works with voxelized signed distance fields and point clouds of the scene. Concretely, the workspace of the gripper’s fingers is voxelized and a uniform surface point representation of the grasped object is generated. The intersection of both data structures during online grasping execution yields the approximate contact surfaces. In our setup, a 2-finger gripper with soft pads is used for grasping validation, performed through successful antipodal point detection under 1 ms. Simulated experimental results are provided for multiple shapes and resolutions.

Semi-supervised medical image segmentation via a tripled-uncertainty guided mean teacher model with contrastive learning.

Due to the difficulty in accessing a large amount of labeled data, semi-supervised learning is becoming an attractive solution in medical image segmentation. To make use of unlabeled data, current popular semi-supervised methods (e.g., temporal ensembling, mean teacher) mainly impose data-level and model-level consistency on unlabeled data. In this paper, we argue that in addition to these strategies, we could further utilize auxiliary tasks and consider task-level consistency to better excavate effective representations from unlabeled data for segmentation. Specifically, we introduce two auxiliary tasks, i.e., a foreground and background reconstruction task for capturing semantic information and a signed distance field (SDF) prediction task for imposing shape constraint, and explore the mutual promotion effect between the two auxiliary and the segmentation tasks based on mean teacher architecture. Moreover, to handle the potential bias of the teacher model caused by annotation scarcity, we develop a tripled-uncertainty guided framework to encourage the three tasks in the student model to learn more reliable knowledge from the teacher. When calculating uncertainty, we propose an uncertainty weighted integration (UWI) strategy for yielding the segmentation predictions of the teacher. In addition, following the advance of unsupervised learning in leveraging the unlabeled data, we also incorporate a contrastive learning based constraint to help the encoders extract more distinct representations to promote the medical image segmentation performance. Extensive experiments on the public 2017 ACDC dataset and the PROMISE12 dataset have demonstrated the effectiveness of our method.

Shape dithering for 3D printing

We present an efficient, purely geometric, algorithmic, and parameter free approach to improve surface quality and accuracy in voxel-controlled 3D printing by counteracting quantization artifacts. Such artifacts arise due to the discrete voxel sampling of the continuous shape used to control the 3D printer, and are characterized by low-frequency geometric patterns on surfaces of any orientation. They are visually disturbing, particularly on small prints or smooth surfaces, and adversely affect the fatigue behavior of printed parts. We use implicit shape dithering, displacing the part's signed distance field with a high-frequent signal whose amplitude is adapted to the (anisotropic) print resolution. We expand the reverse generalized Fourier slice theorem by shear transforms, which we leverage to optimize a 3D blue-noise mask to generate the anisotropic dither signal. As a point process it is efficient and does not adversely affect 3D halftoning. We evaluate our approach for efficiency, geometric accuracy and show its advantages over the state of the art.

Make it Dense: Self-Supervised Geometric Scan Completion of Sparse 3D LiDAR Scans in Large Outdoor Environments

Mapping systems that turn sensor data into a model of the environment are standard components in mobile robotics. Outdoor robots are often equipped with 3D LiDAR sensors to obtain accurate range measurements at a high frame rate. The price for a robotic LiDAR sensor scales roughly linearly with the number of beams and thus the vertical resolution of the scanner. In general, the cheaper the sensors, the sparser the point cloud. In this letter, we address the problem of building dense models from sparse range data. Instead of requiring the vehicle to move slowly through the environment or to traverse the scene multiple times to cover the space densely, we investigate geometric scan completion through a learning-based approach. We revisit the traditional volumetric fusion pipeline based on truncated signed distance fields (TSDF) and propose a neural network to aid the 3D reconstruction on a frame-to-frame basis by completing each scan towards a dense TSDF volume. We propose a geometric scan completion network that is trained in a self-supervised fashion without labels. Our experiments illustrate that such frame-wise completion leads to maps that are on-par or even better compared to maps generated using a higher resolution LiDAR sensor. We additionally show that our system can be used to improve the performance of SLAM systems.

POp: Parameter Optimization of Differentiable Vector Patterns

AbstractProcedural materials are extensively used in computer graphics, since they provide editable, resolution‐independent representation of textures. However, tuning the parameters of procedural generators to achieve a desired result remains time‐consuming for users. Recently, inverse procedural material algorithms have been developed, exploiting differentiable rendering methods to find the parameters of a procedural model that match a target image. These approaches focus on raster textures. We propose pOp, a practical method for estimating the parameters of vector patterns, that are formed by collections of vector shapes arranged by an arbitrary procedural program. In our approach, patterns are defined as arbitrary programs, that control the translation, rotation and scale or vector graphics elements. We support elements typical of vector graphics, namely points, lines, circle, rounded rectangles, and quadratic Bèzier drawings, in multiple colors. We optimize the program parameters by automatically differentiating the signed distance field of the drawing, which we found to be significantly more reliable than using differentiable rendering of the final image. We demonstrate our method on a variety of cases, representing the variations found in structured vector patterns.

Efficient cloth simulation based on the material point method

AbstractIn this article, we propose a novel cloth simulation method based on the material point method (abbr. MPM). The response to the cloth strain is characterized by three separated models: the continuous in‐plane hyperelasticity, the discrete bending elastoplasticity, and the continuous frictional contact elastoplasticity. We make use of the moving least squares MPM (abbr. MLS‐MPM) transfer scheme and a discrete bending model to reduce computation overhead. We further extend the hyperelastic bending model to be elastoplastic, which can be further used for the simulation of a wide range of thin shell materials, such as thin metal sheets. A fast and approximate signed distance field generation method for humanoid animation is also proposed. Experimental results demonstrate that our method can efficiently and robustly simulate high‐resolution challenging scenes with hundreds of thousands of triangles.

Efficient online motion planning method for the robotic arm to pick-up moving objects smoothly with temporal constraints

This paper proposes a new efficient online motion planning method for the manipulator to grasp moving objects smoothly. The algorithm framework consists of front-end pathfinding and back-end nonlinear trajectory optimization. The sample-based pathfinding algorithm can predict the intersection location and find a safe initial path in the dynamic environment. The gradient-based continuous-time trajectory optimization method converts the Euclidean Signed Distance Field information into the joint space, uses the B-spline curve to represent the joint trajectories, optimizes the initial path using convex hull characteristics of B-spline, and generates a smooth and kinematics feasible trajectory in joint space. Finally, we use an adjustment method of iterative trajectory fragment length to guarantee kinodynamic feasibility. Simulation comparisons and real-world experiments verify our planning framework’s high performance.

Advances in Neural Rendering

AbstractSynthesizing photo‐realistic images and videos is at the heart of computer graphics and has been the focus of decades of research. Traditionally, synthetic images of a scene are generated using rendering algorithms such as rasterization or ray tracing, which take specifically defined representations of geometry and material properties as input. Collectively, these inputs define the actual scene and what is rendered, and are referred to as the scene representation (where a scene consists of one or more objects). Example scene representations are triangle meshes with accompanied textures (e.g., created by an artist), point clouds (e.g., from a depth sensor), volumetric grids (e.g., from a CT scan), or implicit surface functions (e.g., truncated signed distance fields). The reconstruction of such a scene representation from observations using differentiable rendering losses is known as inverse graphics or inverse rendering. Neural rendering is closely related, and combines ideas from classical computer graphics and machine learning to create algorithms for synthesizing images from real‐world observations. Neural rendering is a leap forward towards the goal of synthesizing photo‐realistic image and video content. In recent years, we have seen immense progress in this field through hundreds of publications that show different ways to inject learnable components into the rendering pipeline. This state‐of‐the‐art report on advances in neural rendering focuses on methods that combine classical rendering principles with learned 3D scene representations, often now referred to as neural scene representations. A key advantage of these methods is that they are 3D‐consistent by design, enabling applications such as novel viewpoint synthesis of a captured scene. In addition to methods that handle static scenes, we cover neural scene representations for modeling non‐rigidly deforming objects and scene editing and composition. While most of these approaches are scene‐specific, we also discuss techniques that generalize across object classes and can be used for generative tasks. In addition to reviewing these state‐of‐the‐art methods, we provide an overview of fundamental concepts and definitions used in the current literature. We conclude with a discussion on open challenges and social implications.

Gradient Flow Evolution for 3D Fusion From a Single Depth Sensor

We present a novel real-time framework for non-rigid 3D reconstruction that is robust to noise, camera poses, and large deformation from a single depth camera. KinectFusion has achieved high-quality 3D object reconstructions in real-time by implicitly representing an object’s surface with a signed distance field (SDF) representation from a single depth camera. Many studies for incremental reconstruction have been presented since then, with the surface estimation improving over time. Previous works primarily focused on improving conventional SDF matching and deformation schemes. In contrast to these works, the proposed framework tackles the problem of temporal inconsistency caused by SDF approximation and fusion to manipulate SDFs and reconstruct a target more accurately over time. In our reconstruction pipeline, we introduce a refinement evolution method, where an erroneous SDF from a depth sensor is recovered more accurately in a few iterations by propagating erroneous SDF values from the surface. Reliable gradients of refined SDFs enable more accurate non-rigid tracking of a target object. Furthermore, we propose a level-set evolution for SDF fusion, enabling SDFs to be manipulated stably in the reconstruction pipeline over time. The proposed methods are fully parallelizable and can be executed in real-time. Qualitative and quantitative evaluations show that incorporating the refinement and fusion methods into the reconstruction pipeline improves 3D reconstruction accuracy and temporal reliability by avoiding cumulative errors over time. Evaluation results show that our pipeline results in more accurate reconstruction that is robust to noise and large motions, as well as outperforms previous state-of-the-art reconstruction methods.

Deep Learning for Real-Time Aerodynamic Evaluations of Arbitrary Vehicle Shapes

The aerodynamic optimization process of cars requires multiple iterations between aerodynamicists and stylists. Response Surface Modeling and Reduced-Order Modeling are commonly used to eliminate the overhead due to Computational Fluid Dynamics, leading to faster iterations. However, a primary drawback of these models is that they can work only on the parametrized geometric features they were trained with. This study evaluates if deep learning models can predict the drag coefficient for an arbitrary input geometry without explicit parameterization. We use two similar data sets based on the publicly available DrivAer geometry for training. We use a modified U-Net architecture that uses Signed Distance Fields to represent the input geometries. Our models outperform the existing models by at least 11% in prediction accuracy for the drag coefficient. We achieved this improvement by combining multiple data sets that were created using different parameterizations, which is not possible with the methods currently used. We have also shown that it is possible to predict velocity fields and drag coefficient concurrently and that simple data augmentation methods can improve the results. In addition, we have performed an occlusion sensitivity study on our models to understand what information is used to make predictions. From the occlusion sensitivity study, we showed that the models were able to identify the geometric features and have discovered that the model has learned to exploit the global information present in the SDF. In contrast to the currently operational method, where design changes are restricted to the initially defined parameters, this study brings surrogate models one step closer to the long-term goal of having a model that can be used for approximate aerodynamic evaluation of unseen, arbitrary vehicle shapes, thereby providing complete design freedom to the vehicle stylists.

Clothing Visual Representation for 3D Human Reconstruction

To address the problem of fuzzy representation of clothing visual information in three-dimensional human reconstruction, a multi-stage based optimization method is proposed. Firstly, semantic features, shading features and high-frequency features are extracted from input images in preprocess. Secondly, the signed distance field is constructed based on the local depth features to implicitly represent the shape of clothed human. Then, to optimize the signed distance field and generate the rough three-dimensional human model, the dress hierarchy module is constructed, and the clothing semantic context is perceived by defining the dress hierarchy loss function. Finally, combining the shading features and high-frequency features to capture the details of wrinkles in the UV space, and an accurate three-dimensional human model is generated. The experimental results on BUFF dataset show that the normal <italic>L</italic>₂ error and the chamfer distance error are 0.13 and 1.30 respectively, which reduce by 13% and 2%. The proposed method can improve the accuracy of three-dimensional human reconstruction, which generates three-dimensional human model with clothing style, dress hierarchy and wrinkles.

Autonomous robotic additive manufacturing through distributed model‐free deep reinforcement learning in computational design environments

The objective of autonomous robotic additive manufacturing for construction in the architectural scale is currently being investigated in parts both within the research communities of computational design and robotic fabrication (CDRF) and deep reinforcement learning (DRL) in robotics. The presented study summarizes the relevant state of the art in both research areas and lays out how their respective accomplishments can be combined to achieve higher degrees of autonomy in robotic construction within the Architecture, Engineering and Construction (AEC) industry. A distributed control and communication infrastructure for agent training and task execution is presented, that leverages the potentials of combining tools, standards and algorithms of both fields. It is geared towards industrial CDRF applications. Using this framework, a robotic agent is trained to autonomously plan and build structures using two model-free DRL algorithms (TD3, SAC) in two case studies: robotic block stacking and sensor-adaptive 3D printing. The first case study serves to demonstrate the general applicability of computational design environments for DRL training and the comparative learning success of the utilized algorithms. Case study two highlights the benefit of our setup in terms of tool path planning, geometric state reconstruction, the incorporation of fabrication constraints and action evaluation as part of the training and execution process through parametric modeling routines. The study benefits from highly efficient geometry compression based on convolutional autoencoders (CAE) and signed distance fields (SDF), real-time physics simulation in CAD, industry-grade hardware control and distinct action complementation through geometric scripting. Most of the developed code is provided open source.

Voxel‐wise UV parameterization and view‐dependent texture synthesis for immersive rendering of truncated signed distance field scene model

In this paper, we introduced a novel voxel-wise UV parameterization and view-dependent texture synthesis for the immersive rendering of a truncated signed distance field (TSDF) scene model. The proposed UV parameterization delegates a precomputed UV map to each voxel using the UV map lookup table and consequently, enabling efficient and high-quality texture mapping without a complex process. By leveraging the convenient UV parameterization, our view-dependent texture synthesis method extracts a set of local texture maps for each voxel from the multiview color images and separates them into a single view-independent diffuse map and a set of weight coefficients for an orthogonal specular map basis. Furthermore, the view-dependent specular maps for an arbitrary view are estimated by combining the specular weights of each source view using the location of the arbitrary and source viewpoints to generate the view-dependent textures for arbitrary views. The experimental results demonstrate that the proposed method effectively synthesizes texture for an arbitrary view, thereby enabling the visualization of view-dependent effects, such as specularity and mirror reflection.

Prior-Guided Multi-View 3D Head Reconstruction

Recovery of a 3D head model including the complete face and hair regions is still a challenging problem in computer vision and graphics. In this paper, we consider this problem using only a few multi-view portrait images as input. Previous multi-view stereo methods that have been based, either on optimization strategies or deep learning techniques, suffer from low-frequency geometric structures such as unclear head structures and inaccurate reconstruction in hair regions. To tackle this problem, we propose a prior-guided implicit neural rendering network. Specifically, we model the head geometry with a learnable signed distance field (SDF) and optimize it via an implicit differentiable renderer with the guidance of some human head priors, including the facial prior knowledge, head semantic segmentation information and 2D hair orientation maps. The utilization of these priors can improve the reconstruction accuracy and robustness, leading to a high-quality integrated 3D head model. Extensive ablation studies and comparisons with state-of-the-art methods demonstrate that our method can generate high-fidelity 3D head geometries with the guidance of these priors.

Interactive Editing of Voxel-Based Signed Distance Fields

Signed distance functions computed in discrete form from given RGB-D data as regular voxel grids can represent manifold shapes as the zero crossing of a trivariate function; the corresponding meshes can be derived by the Marching Cubes algorithm. However, 3D models automatically reconstructed in this way often contain irrelevant objects or artifacts, such as holes or noise, due to erroneous scan data and error-prone reconstruction processes. This paper presents an approach for interactive editing of signed distance functions, derived from RGB-D data in the form of regular voxel grids, that enables the manual refinement and enhancement of reconstructed 3D geometry. To this end, we combine concepts known from constructive solid geometry, where complex models are created from simple base shapes, with the voxel-based representation of geometry reconstructed from real-world scans. Our approach can be implemented entirely on GPU to enable real-time interaction. Further, we present how to implement high-level operators, such as copy, move, and unification.

SDF-RVD: Restricted Voronoi Diagram on Signed Distance Field

Signed distance fields (SDFs), as a powerful surface representation, draw more and more attention in recent years, especially with the innovation of geometric deep learning techniques. In fact, many kinds of popular surface representations, whether explicit forms (point cloud, mesh, or S(u,v)=x(u,v),y(u,v),z(u,v)) or implicit forms (f(x,y,z)=0), can be easily transformed into the SDF representation. It is an interesting research direction to accomplish some important operations directly on the SDF representation. Considering that restricted Voronoi diagrams (RVDs), as a fundamental geometric tool, are central to sampling, meshing and many other geometry processing tasks, we focus on how to define and compute RVDs based SDF, named SDF-RVD. Given a set of samples {xi}i=1m lying on the surface, the key idea is to define RVD based on analyzing how the 3D Voronoi diagram w.r.t. {xi}i=1m passes through the implicit surface, without rigorously decomposing the watertight surface into a set of curved surface patches. We further observe that SDF-RVD works well on an arbitrary surface representation as long as it is equipped with the projection operation. Even for the mesh input, the timing cost spent in computing SDF-RVD is nearly independent of the original mesh resolution, which distinguishes itself from the traditional RVD based meshing approaches. We further show that SDF-RVD is able to naturally work with centroidal Voronoi tessellation (CVT) and report a well triangulated mesh as the output. Finally, we link SDF-RVD plus CVT up with those continuous SDF based neural networks (e.g., DeepSDF) for directly transforming a point cloud into a high-quality triangle mesh, which does not include a step of extracting a poor-quality mesh using marching cubes.

영상센서를 이용한 실외 무인항공기의 충돌회피에 관한 연구

This study investigates real-time path planning for UAVs (Unmanned Aerial Vehicles) to avoid obstacles in an outdoor environment in a three-dimensional space. First, we implement an algorithm to convert from Octomap to a grid map to implement a global grid map. Secondly, the ESDF (Euclidean Signed-Distance Field) algorithm is integrated into the path planning obtained through the JPS (Jump Point Search) algorithm, and the B-Spline algorithm is applied to modify the path to consider the dynamic constraint of UAVs. Finally, path planning algorithms are verified in real-time through simulation and flight testing. The simulation is based on the Gazebo environment using the open-source autopilot PX4 with the ROS. A depth camera performance is compared for an outdoor flight test to select a depth camera mounted on the drone. Moreover, the practical obstacle-avoidance performance of the proposed real-time route planning in an outdoor space is confirmed through a flight test.

Raymarching Distance Fields with CUDA

Raymarching is a technique for rendering implicit surfaces using signed distance fields. It has been known and used since the 1980s for rendering fractals and CSG (constructive solid geometry) surfaces, but has rarely been used for commercial rendering applications such as film and 3D games. Raymarching was first used for photorealistic rendering in the mid 2000s by demoscene developers and hobbyist graphics programmers, receiving little to no attention from the academic community and professional graphics engineers. In the present work, we explain why the use of Simple and Fast Multimedia Library (SFML) by nearly all existing approaches leads to a number of inefficiencies, and hence set out to develop a CUDA oriented approach instead. We next show that the usual data handling pipeline leads to further unnecessary data flow overheads and therefore propose a novel pipeline structure that eliminates much of redundancy in the manner in which data are processed and passed. We proceed to introduce a series of data structures which were designed with the specific aim of exploiting the pipeline’s strengths in terms of efficiency while achieving a high degree of photorealism, as well as the accompanying models and optimizations that ultimately result in an engine which is capable of photorealistic and real-time rendering on complex scenes and arbitrary objects. Lastly, the effectiveness of our framework is demonstrated in a series of experiments which compare our engine both in terms of visual fidelity and computational efficiency with the leading commercial and open source solutions, namely Unreal Engine and Blender.