Abstract
Reconstruction of 3D space from visual data has always been a significant challenge in the field of computer vision. A popular approach to address this problem can be found in the form of bottom-up reconstruction techniques which try to model complex 3D scenes through a constellation of volumetric primitives. Such techniques are inspired by the current understanding of the human visual system and are, therefore, strongly related to the way humans process visual information, as suggested by recent visual neuroscience literature. While advances have been made in recent years in the area of 3D reconstruction, the problem remains challenging due to the many possible ways of representing 3D data, the ambiguity of determining the shape and general position in 3D space and the difficulty to train efficient models for the prediction of volumetric primitives. In this article, we address these challenges and present a novel solution for recovering volumetric primitives from depth images. Specifically, we focus on the recovery of superquadrics, a special type of parametric models able to describe a wide array of 3D shapes using only a few parameters. We present a new learning objective that relies on the superquadric (inside-outside) function and develop two learning strategies for training convolutional neural networks (CNN) capable of predicting superquadric parameters. The first uses explicit supervision and penalizes the difference between the predicted and reference superquadric parameters. The second strategy uses implicit supervision and penalizes differences between the input depth images and depth images rendered from the predicted parameters. CNN predictors for superquadric parameters are trained with both strategies and evaluated on a large dataset of synthetic and real-world depth images. Experimental results show that both strategies compare favourably to the existing state-of-the-art and result in high quality 3D reconstructions of the modelled scenes at a much shorter processing time.
Highlights
3D reconstruction represents one of the central problems of computer vision
We introduce a novel geometry-aware learning objective based on the superquadric function that allows us to train convolutional neural networks (CNN) predictors capable of predicting the parameters of a single superquadric in general position from input depth images
To estimate the parameters λ of a superquadric model from an input depth image X, we present in this work two CNN learning strategies, such that the learned models fCNN produce parameter estimates as close to the reference values as possible, i.e.: λ = fCNN (X ; θ ), (5)
Summary
3D reconstruction represents one of the central problems of computer vision. It aims to interpret the shape, appearance, as well as the relative position of objects in the environment and to derive a unique (typically parameterized) description. In the context of artificial systems, a reconstructed scene can be used to inform an autonomous agent of its surroundings and to enable complex interactions, such as collision avoidance, maneuvering [1], [2] or grasping [3], [4]. While different approaches have been proposed in the literature for this task, bottom-up reconstruction of 3D scenes with volumetric primitives is appealing. Such methods use a fixed vocabulary of possible elementary
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
