Dynamic Domain Adaptation for Single-view 3D Reconstruction

Abstract

Learning 3D object reconstruction from a single RGB image is a fundamental and extremely challenging problem for robots. As acquiring labeled 3D shape representations for real-world data is time-consuming and expensive, synthetic image-shape pairs are widely used for 3D reconstruction. However, the models trained on synthetic data set did not perform equally well on real-world images. The existing method used the domain adaptation to fill the domain gap between different data sets. Unlike the approach simply considered global distribution for domain adaptation, this paper presents a dynamic domain adaptation (DDA) network to extract domain-invariant image features for 3D reconstruction. The relative importance between global and local distributions are considered to reduce the discrepancy between synthetic and real-world data. In addition, graph convolution network (GCN) based mesh generation methods have achieved impressive results than voxel-based and point cloud-based methods. However, the global context in a graph is not effectively used due to the limited receptive field of GCN. In this paper, a multi-scale processing method for graph convolution network (GCN) is proposed to further improve the performance of GCN-based 3D reconstruction. The experiment results conducted on both synthetic and real-world data set have demonstrated the effectiveness of the proposed methods.