Part-Wise AtlasNet for 3D point cloud reconstruction from a single image

Abstract

Learning to generate three dimensional (3D) point clouds from a single image remains a challenging task. Numerous approaches with encoder–decoder architectures have been proposed. However, these methods are hard to realize structured reconstructions and usually lack constraints on the local structures of 3D objects. AtlasNet as a representative model of 3D reconstruction consists of many branches, and each branch is a neural network used to reconstruct one local patch of a 3D object. However, the neural networks in AtlasNet and the patches of 3D objects are not in one-to-one correspondence before training. This case is not conducive to adding some reconstruction constraints to the local structures of 3D objects. Based on the architecture of AtlasNet, we propose Part-Wise AtlasNet in which each neural network is only responsible for reconstructing one specific part of a 3D object. This kind of restriction facilitates imposition of several local constraints on the final reconstruction loss, hence better recovering 3D objects with finer local structures. Both the qualitative results and quantitative analysis show that the variants of the proposed method with the local reconstruction losses generate structured point clouds with a higher visual quality and achieve better performance than other methods in 3D point cloud generation from a single image.