3D mesh reconstruction of indoor scenes from a single image in-the-wild

Abstract

3D reconstruction from a single RGB image is making some progress especially with the advent of deep learning in recent years. In this paper, we focus on 3D reconstruction of indoor scenes taking as input a single image, without point clouds, multi-view images, depth or masks. Our mesh reconstruction is based on an encoder-decoder framework of mesh deformation. Features extracted by the encoder from the input image will be used as supervision for mesh deformation. We propose an inter-point interaction attention in the decoder to exploit the vertexes’ influence on each other. We also use a smooth loss to generate smoother surface for objects. We have evaluated the proposed framework on the Pix3D dataset, and state-of-the-art performance has been achieved with visually appealing 3D geometry.