Volumetric Primitives Research Articles

Robust Shape Fitting for 3D Scene Abstraction.

Humans perceive and construct the world as an arrangement of simple parametric models. In particular, we can often describe man-made environments using volumetric primitives such as cuboids or cylinders. Inferring these primitives is important for attaining high-level, abstract scene descriptions. Previous approaches for primitive-based abstraction estimate shape parameters directly and are only able to reproduce simple objects. In contrast, we propose a robust estimator for primitive fitting, which meaningfully abstracts complex real-world environments using cuboids. A RANSAC estimator guided by a neural network fits these primitives to a depth map. We condition the network on previously detected parts of the scene, parsing it one-by-one. To obtain cuboids from single RGB images, we additionally optimise a depth estimation CNN end-to-end. Naively minimising point-to-primitive distances leads to large or spurious cuboids occluding parts of the scene. We thus propose an improved occlusion-aware distance metric correctly handling opaque scenes. Furthermore, we present a neural network based cuboid solver which provides more parsimonious scene abstractions while also reducing inference time. The proposed algorithm does not require labour-intensive labels, such as cuboid annotations, for training. Results on the NYU Depth v2 dataset demonstrate that the proposed algorithm successfully abstracts cluttered real-world 3D scene layouts.

FR-CSG: Fast and Reliable Modeling for Constructive Solid Geometry.

Reconstructing CSG trees from CAD models is a critical subject in reverse engineering. While there have been notable advancements in CSG reconstruction, challenges persist in capturing geometric details and achieving efficiency. Additionally, since non-axis-aligned volumetric primitives cannot maintain coplanar characteristics due to discretization errors, existing Boolean operations often lead to zero-volume surfaces and suffer from topological errors during the CSG modeling process. To address these issues, we propose a novel workflow to achieve fast CSG reconstruction and reliable forward modeling. First, we employ feature removal and model subdivision techniques to decompose models into sub-components. This significantly expedites the reconstruction by simplifying the complexity of the models. Then, we introduce a more reasonable method for primitive generation and filtering, and utilize a size-related optimization approach to reconstruct CSG trees. By re-adding features as additional nodes in the CSG trees, our method not only preserves intricate details but also ensures the conciseness, semantic integrity, and editability of the resulting CSG tree. Finally, we develop a coplanar primitive discretization method that represents primitives as large planes and extracts the original triangles after intersection. We extend the classification of triangles and incorporate a coplanar-aware Boolean tree assessment technique, allowing us to achieve manifold and watertight modeling results without zero-volume surfaces, even in extreme degenerate cases. We demonstrate the superiority of our method over state-of-the-art approaches. Moreover, the reconstructed CSG trees generated by our method contain extensive semantic information, enabling diverse model editing tasks.

Reconstructing Superquadrics from Intensity and Color Images.

The task of reconstructing 3D scenes based on visual data represents a longstanding problem in computer vision. Common reconstruction approaches rely on the use of multiple volumetric primitives to describe complex objects. Superquadrics (a class of volumetric primitives) have shown great promise due to their ability to describe various shapes with only a few parameters. Recent research has shown that deep learning methods can be used to accurately reconstruct random superquadrics from both 3D point cloud data and simple depth images. In this paper, we extended these reconstruction methods to intensity and color images. Specifically, we used a dedicated convolutional neural network (CNN) model to reconstruct a single superquadric from the given input image. We analyzed the results in a qualitative and quantitative manner, by visualizing reconstructed superquadrics as well as observing error and accuracy distributions of predictions. We showed that a CNN model designed around a simple ResNet backbone can be used to accurately reconstruct superquadrics from images containing one object, but only if one of the spatial parameters is fixed or if it can be determined from other image characteristics, e.g., shadows. Furthermore, we experimented with images of increasing complexity, for example, by adding textures, and observed that the results degraded only slightly. In addition, we show that our model outperforms the current state-of-the-art method on the studied task. Our final result is a highly accurate superquadric reconstruction model, which can also reconstruct superquadrics from real images of simple objects, without additional training.

Mixture of volumetric primitives for efficient neural rendering

Real-time rendering and animation of humans is a core function in games, movies, and telepresence applications. Existing methods have a number of drawbacks we aim to address with our work. Triangle meshes have difficulty modeling thin structures like hair, volumetric representations like Neural Volumes are too low-resolution given a reasonable memory budget, and high-resolution implicit representations like Neural Radiance Fields are too slow for use in real-time applications. We present Mixture of Volumetric Primitives (MVP), a representation for rendering dynamic 3D content that combines the completeness of volumetric representations with the efficiency of primitive-based rendering, e.g., point-based or mesh-based methods. Our approach achieves this by leveraging spatially shared computation with a deconvolutional architecture and by minimizing computation in empty regions of space with volumetric primitives that can move to cover only occupied regions. Our parameterization supports the integration of correspondence and tracking constraints, while being robust to areas where classical tracking fails, such as around thin or translucent structures and areas with large topological variability. MVP is a hybrid that generalizes both volumetric and primitive-based representations. Through a series of extensive experiments we demonstrate that it inherits the strengths of each, while avoiding many of their limitations. We also compare our approach to several state-of-the-art methods and demonstrate that MVP produces superior results in terms of quality and runtime performance.

Mixture of volumetric primitives for efficient neural rendering

Real-time rendering and animation of humans is a core function in games, movies, and telepresence applications. Existing methods have a number of drawbacks we aim to address with our work. Triangle meshes have difficulty modeling thin structures like hair, volumetric representations like Neural Volumes are too low-resolution given a reasonable memory budget, and high-resolution implicit representations like Neural Radiance Fields are too slow for use in real-time applications. We present Mixture of Volumetric Primitives (MVP), a representation for rendering dynamic 3D content that combines the completeness of volumetric representations with the efficiency of primitive-based rendering, e.g., point-based or mesh-based methods. Our approach achieves this by leveraging spatially shared computation with a convolutional architecture and by minimizing computation in empty regions of space with volumetric primitives that can move to cover only occupied regions. Our parameterization supports the integration of correspondence and tracking constraints, while being robust to areas where classical tracking fails, such as around thin or translucent structures and areas with large topological variability. MVP is a hybrid that generalizes both volumetric and primitive-based representations. Through a series of extensive experiments we demonstrate that it inherits the strengths of each, while avoiding many of their limitations. We also compare our approach to several state-of-the-art methods and demonstrate that MVP produces superior results in terms of quality and runtime performance.

Virtual Fixture Generation for Task Planning With Complex Geometries

Abstract Many robotic processes require the system to maintain a tool’s orientation and distance from a surface. To do so, researchers often use virtual fixtures (VFs) to either guide the robot along a path or forbid it from leaving the workspace. Previous efforts relied on volumetric primitives (planes, cylinders, etc.) or raw sensor data to define VFs. However, those approaches only work for a small subset of real-world objects. Extending this approach is complicated not only by VF generation but also by generalizing user traversal of the VF to command a robot trajectory remotely. In this study, we present the concept of task VFs, which convert layers of point cloud-based Guidance VF into a bidirectional graph structure and pair it with a Forbidden Region VF. These VFs are hardware-agnostic and can be generated from virtually any source data, including from parametric objects (superellipsoids, supertoroids, etc.), meshes (including from computer-aided design (CAD)), and real-time sensor data for open-world scenarios. We address surface convexity and concavity since these and distance to the task surface determine the size and resolution of VF layers. This article then presents the manipulator-to-task transform tool for task VF visualization and to limit human–robot interaction ambiguities. Testing confirmed generation success and users performed spatially discrete experiments to evaluate task VF usability complex geometries, which showed their interpretability. The manipulator-to-task transform tool applies many robotic applications, including collision avoidance, process design, training, task definition, etc. for virtually any geometry.

Learning to Predict Superquadric Parameters From Depth Images With Explicit and Implicit Supervision

Reconstruction of 3D space from visual data has always been a significant challenge in the field of computer vision. A popular approach to address this problem can be found in the form of bottom-up reconstruction techniques which try to model complex 3D scenes through a constellation of volumetric primitives. Such techniques are inspired by the current understanding of the human visual system and are, therefore, strongly related to the way humans process visual information, as suggested by recent visual neuroscience literature. While advances have been made in recent years in the area of 3D reconstruction, the problem remains challenging due to the many possible ways of representing 3D data, the ambiguity of determining the shape and general position in 3D space and the difficulty to train efficient models for the prediction of volumetric primitives. In this article, we address these challenges and present a novel solution for recovering volumetric primitives from depth images. Specifically, we focus on the recovery of superquadrics, a special type of parametric models able to describe a wide array of 3D shapes using only a few parameters. We present a new learning objective that relies on the superquadric (inside-outside) function and develop two learning strategies for training convolutional neural networks (CNN) capable of predicting superquadric parameters. The first uses explicit supervision and penalizes the difference between the predicted and reference superquadric parameters. The second strategy uses implicit supervision and penalizes differences between the input depth images and depth images rendered from the predicted parameters. CNN predictors for superquadric parameters are trained with both strategies and evaluated on a large dataset of synthetic and real-world depth images. Experimental results show that both strategies compare favourably to the existing state-of-the-art and result in high quality 3D reconstructions of the modelled scenes at a much shorter processing time.

Modeling rocky scenery using implicit blocks

We present a novel geologically based method to generate vertical walls of rocky cliffs, crags or promontories. Our method procedurally generates a distribution of fractures in the bedrock to create a set of tiling blocks defined as implicit volumetric primitives. Blocks are in turn implicitly replicated over the vertical parts of the terrain and combined together to obtain a consistent volumetric representation of the fractured bedrock patterns using generalized union and blending operators. Our framework provides multiple levels of control: In addition to automatically generated blocks, the geometry of specific ones can be prescribed by the user using implicit primitives or construction trees, the shape of the blocks can be controlled by several parameters, and the placement rules may adapt according to the underlying geological strata and geometry of the terrain.

A B-spline based framework for volumetric object modeling

With the recent development of Iso-geometric Analysis (IGA) (Cottrell et al., 2009) and advanced manufacturing technologies employing heterogeneous materials, such as additive manufacturing (AM) of functionally graded material, there is a growing emerging need for a full volumetric representation of 3D objects, that prescribes the interior of the object in addition to its boundaries. In this paper, we propose a volumetric representation (V-rep) for geometric modeling that is based on trimmed B-spline trivariates and introduce its supporting volumetric modeling framework. The framework includes various volumetric model (V-model) construction methods from basic non-singular volumetric primitives to high level constructors, as well as Boolean operations’ support for V-models. A V-model is decomposed into and defined by a complex of volumetric cells (V-cells), each of which can also represent a variety of additional varying fields over it, and hence over the entire V-model. With these capabilities, the proposed framework is able of supporting volumetric IGA needs as well as represent and manage heterogeneous materials for AM. Further, this framework is also a seamless extension to existing boundary representations (B-reps) common in all contemporary geometric modeling systems, and allows a simple migration of existing B-rep data, tools and algorithms. Examples of volumetric models constructed using the proposed framework are presented.

Indirect unstructured hex-dominant mesh generation using tetrahedra recombination

Corner-point gridding is widely used in reservoir and basin modeling but generally yields approximations in the representation of geological interfaces. This paper introduces an indirect method to generate a hex-dominant mesh conformal to 3D geological surfaces and well paths suitable for finite-element and control-volume finite-element simulations. By indirect, we mean that the method first generates an unstructured tetrahedral mesh whose tetrahedra are then merged into primitives (hexahedra, prisms, and pyramids). More specifically, we focus on determining the optimal set of primitives that can be recombined from a given tetrahedral mesh. First, we detect in the tetrahedral mesh all the feasible volumetric primitives using a pattern-matching algorithm (Meshkat and Talmor Int. J. Numer. Meth. Eng. 49(1-2), 17–30 2000) that we re-visit and extend with configurations that account for degenerated tetrahedra (slivers). Then, we observe that selecting the optimal set of primitives among the feasible ones can be formalized as a maximum weighted independent set problem (Bomze et al. 1999), known to be $\mathcal {N}\mathcal {P}$ -Complete. We propose several heuristic optimizations to find a reasonable set of primitives in a practical time. All the tetrahedra of each selected primitive are then merged to build the final unstructured hex-dominant mesh. This method is demonstrated on 3D geological models including a faulted and folded model and a discrete fracture network.

Scene Understanding by Reasoning Stability and Safety

This paper presents a new perspective for 3D scene understanding by reasoning object stability and safety using intuitive mechanics. Our approach utilizes a simple observation that, by human design, objects in static scenes should be stable in the gravity field and be safe with respect to various physical disturbances such as human activities. This assumption is applicable to all scene categories and poses useful constraints for the plausible interpretations (parses) in scene understanding. Given a 3D point cloud captured for a static scene by depth cameras, our method consists of three steps: (i) recovering solid 3D volumetric primitives from voxels; (ii) reasoning stability by grouping the unstable primitives to physically stable objects by optimizing the stability and the scene prior; and (iii) reasoning safety by evaluating the physical risks for objects under physical disturbances, such as human activity, wind or earthquakes. We adopt a novel intuitive physics model and represent the energy landscape of each primitive and object in the scene by a disconnectivity graph (DG). We construct a contact graph with nodes being 3D volumetric primitives and edges representing the supporting relations. Then we adopt a Swendson---Wang Cuts algorithm to partition the contact graph into groups, each of which is a stable object. In order to detect unsafe objects in a static scene, our method further infers hidden and situated causes (disturbances) in the scene, and then introduces intuitive physical mechanics to predict possible effects (e.g., falls) as consequences of the disturbances. In experiments, we demonstrate that the algorithm achieves a substantially better performance for (i) object segmentation, (ii) 3D volumetric recovery, and (iii) scene understanding with respect to other state-of-the-art methods. We also compare the safety prediction from the intuitive mechanics model with human judgement.

Reconstructing the World’s Museums

Virtual exploration tools for large indoor environments (e.g. museums) have so far been limited to either blueprint-style 2D maps that lack photo-realistic views of scenes, or ground-level image-to-image transitions, which are immersive but ill-suited for navigation. On the other hand, photorealistic aerial maps would be a useful navigational guide for large indoor environments, but it is impossible to directly acquire photographs covering a large indoor environment from aerial viewpoints. This paper presents a 3D reconstruction and visualization system for automatically producing clean and well-regularized texture-mapped 3D models for large indoor scenes, from ground-level photographs and 3D laser points. The key component is a new algorithm called “inverse constructive solid geometry (CSG)” for reconstructing a scene with a CSG representation consisting of volumetric primitives, which imposes powerful regularization constraints. We also propose several novel techniques to adjust the 3D model to make it suitable for rendering the 3D maps from aerial viewpoints. The visualization system enables users to easily browse a large-scale indoor environment from a bird’s-eye view, locate specific room interiors, fly into a place of interest, view immersive ground-level panorama views, and zoom out again, all with seamless 3D transitions. We demonstrate our system on various museums, including the Metropolitan Museum of Art in New York City—one of the largest art galleries in the world.

Human Motion Tracking with a Kinematic Parameterization of Extremal Contours

This paper addresses the problem of human motion tracking from multiple image sequences. The human body is described by five articulated mechanical chains and human body-parts are described by volumetric primitives with curved surfaces. If such a surface is observed with a camera, an extremal contour appears in the image whenever the surface turns smoothly away from the viewer. We describe a method that recovers human motion through a kinematic parameterization of these extremal contours. The method exploits the fact that the observed image motion of these contours is a function of both the rigid displacement of the surface and of the relative position and orientation between the viewer and the curved surface. First, we describe a parameterization of an extremal-contour point velocity for the case of developable surfaces. Second, we use the zero-reference kinematic representation and we derive an explicit formula that links extremal contour velocities to the angular velocities associated with the kinematic model. Third, we show how the chamfer-distance may be used to measure the discrepancy between predicted extremal contours and observed image contours; moreover we show how the chamfer distance can be used as a differentiable multi-valued function and how the tracker based on this distance can be cast into a continuous non-linear optimization framework. Fourth, we describe implementation issues associated with a practical human-body tracker that may use an arbitrary number of cameras. One great methodological and practical advantage of our method is that it relies neither on model-to-image, nor on image-to-image point matches. In practice we model people with 5 kinematic chains, 19 volumetric primitives, and 54 degrees of freedom; We observe silhouettes in images gathered with several synchronized and calibrated cameras. The tracker has been successfully applied to several complex motions gathered at 30 frames/second.

An automatic classification and robust segmentation procedure of spatial objects

This paper proposes a statistical procedure for the automatic volumetric primitives classification and segmentation of 3D objects surveyed with high density laser scanning range measurements. The procedure is carried out in three main phases: first, a Taylor’s expansion nonparametric model is applied to study the differential local properties of the surface so to classify and identify homogeneous point clusters. Classification is based on the study of the surface Gaussian and mean curvature, computed for each point from estimated differential parameters of the Taylor’s formula extended to second order terms. The geometrical primitives are classified into the following basic types: elliptic, hyperbolic, parabolic and planar. The last phase corresponds to a parametric regression applied to perform a robust segmentation of the various primitives. A Simultaneous AutoRegressive model is applied to define the trend surface for each geometric feature, and a Forward Search procedure puts in evidence outliers or clusters of non stationary data.

Matching Graphs with Fuzzy Attributes in Machine Vision

G.A. Bilodeau and R. Bergevin Laboratoire de vision et systemes numeriques, Pavillon Adrien-Pouliot Universite Laval, Sainte-Foy (QC), Canada, G1K 7P4 bilodeau@gel.ulaval.ca, bergevin@gel.ulaval.ca Abstract In object recognition and image querying applications, complex graphs often have to be compared to verify the similarity between two models. Since there is always uncertainty while models are constructed, the nodes and the edges require fuzzy attributes to properly describe the scene or the object. This paper addresses the problem of matching graphs with fuzzy attributes (GFAs) obtained by hypothesizing volumetric primitives from 2D parts. The GFAs of interests have nodes with many fuzzy attributes that correspond to volumetric hypotheses, and edges that describe the spatial relationship between the hypothesized volumetric primitives. A model for representing 2D parts by volumetric primitives is presented. Then, a method using structural indexing adapted to GFAs is proposed. This inexact matching method has been designed for matching GFAs in large databases.

Geon-driven shape models for holistic semantics

Cognitive theories of 3D shape decomposition have posited simple volumetric primitives (Geon Structural Description or GSD). In this work, we adapt these Geon models for 3D holistic shape design and present a methodology for evaluating holistic aspects of 3D shape, such as aesthetics, via user-interactive optimisation. The conceptual design space is defined through a set of constraints on the parameters of the decomposition. The preliminary shapes are evaluated by combining a computable measure of their function with a subjective evaluation by the user (for aesthetics etc). An important aspect in these early stages of design is the ability of the user to redefine the design specification itself; this is achieved by redefining either the parameter ranges, or the geonic-decomposition itself. Results using an example of the common household faucet are demonstrated on a 3D shape design tool OASIS implemented on top of a standard CAD package.

603 NURBS 立体に基く三次元形状モデリング

603 NURBS 立体に基く三次元形状モデリング

Virtual Cities From Digital Imagery

Increasing interest is being expressed in the use of virtual cities for many fields of application, ranging from planning and simulation to business, advertising and the games industry. Currently, the extraction of buildings from imagery is costly. However, recent developments in digital photogrammetry aim to produce more rapid and efficient methods of modelling three dimensional objects in urban environments. This paper presents a semi-automatic system for the extraction of buildings from digital aerial imagery with the aid of volumetric primitives. The user is supported by automated tools, for example matching tools, in order to reach the necessary efficiency. With an extraction time of about 20 s per building primitive, this method is competitive in terms of the performance of classical photogrammetric procedures, and also offers additional features for analysis, texture mapping and Internet applications.

Shape constancy from novel views.

Prior experiments on shape constancy from novel views are inconclusive: Some show that shapes of objects can be recognized reliably from novel views, whereas others show just the opposite. Our analysis of prior results suggests that shape constancy from novel views is reliable when the object has properties that constrain its shape: The object has volumetric primitives, it has surfaces, it is symmetrical, it is composed of geons, its contours are planar, and its images provide useful topological information about its three-dimensional structure. To test the role of some of these constraints, we performed a set of experiments. Solid shapes (polyhedra) were shown on a computer monitor by means of kinetic depth effect. Experiment 1 showed that shape constancy can be reliably achieved when a polyhedron is represented by its contours (most of the constraints are present), but not when it is represented by vertices or by a polygonal line connecting the vertices in a random order (all the constraints are absent). Experiments 2 and 3 tested the role of individual constraints. Results of these experiments show that shape constancy from novel views is reliable when the object has planar contours and when the shapes of the contours together with topological information about the relations among the contours constrain the possible interpretations of the shape. Symmetry of the object and the topological stability of its image also contribute to shape constancy.

Recovery of 3D volume from 2-tone images of novel objects

In 2-tone images (e.g., Dallenbach's cow), only two levels of brightness are used to convey image structure – dark object regions and shadows are turned to black and light regions are turned white. Despite a lack of shading, hue and texture information, many 2-tone images of familiar objects and scenes are accurately interpreted, even by naive observers. Objects frequently appear fully volumetric and are distinct from their shadows. If perceptual interpretation of 2-tone images is accomplished via bottom-up processes on the basis of geometrical structure projected to the image (e.g., volumetric parts, contour and junction information) novel objects should appear volumetric as readily as their familiar counterparts. We demonstrate that accurate volumetric representations are rarely extracted from 2-tone images of novel objects, even when these objects are constructed from volumetric primitives such as generalized cones (Marr, D., Nishihara, H.K., 1978. Proceedings of the Royal Society London 200, 269–294.; Biederman, I. 1985. Computer Vision, Graphics, and Image Processing 32, 29–73), or from the rearranged components of a familiar object which is itself recognizable as a 2-tone image. Even familiar volumes such as canonical bricks and cylinders require scenes with redundant structure (e.g., rows of cylinders) or explicit lighting (a lamp in the image) for recovery of global volumetric shape. We conclude that 2-tone image perception is not mediated by bottom-up extraction of geometrical features such as junctions or volumetric parts, but may rely on previously stored representations in memory and a model of the illumination of the scene. The success of this top-down strategy implies it is available for general object recognition in natural scenes.