A survey on the current state of the art on deep learning 3D reconstruction

Abstract

Efficient 3D reconstruction from different kind of inputs is a long standing effort of computer vision. Recent advancements in the field of machine learning, specifically deep learning, have started an interest in studying how well these techniques apply to the 3D reconstruction problem. Current efforts employ two main research directions: techniques applied to a single object, trying to reconstruct the surface as closely as possible from different kind of inputs and techniques applied to scenes made from multiple objects, which deal with topological representations, color, illumination and resource consumption. With a plethora of applications in computer graphics and computer vision, the results given by the deep learning techniques start to gain a serious position in the field of 3D reconstruction, yet no survey exist on the recent advancements on these new techniques. This survey summarizes the recent trend and applications of the deep-learning 3D reconstruction methods. We focus on learnable reconstruction methods from inputs like single image, multi-image and point cloud, using different representations, such as voxels, meshes and signed distance fields. Starting by presenting the datasets used, we follow by showing the main deep learning methods using different representations, presenting advantages and disadvantages. The second half of this survey focuses on scene reconstructions and open problems. Finally we conclude with a discussion of the importance of the 3D reconstruction and its possible applications in different fields such as automotive and mixed reality.