Single-Shot 3D Mesh Estimation via Adversarial Domain Adaptation

Abstract

We present an adversarial deep domain adaptation (ADA) approach for training deep neural networks that estimate 3D pose and shape of a human from a single image. Existing datasets of in-the-wild images of humans have limited availability of 3D ground truth. We propose a novel deep architecture for 3D pose estimation and leverage the variations in pose, body shape and background in the synthetic datasets to train our network. Using ADA we adapt our network to real human images by designing a pipeline for joint 3D pose and shape estimation. Thus, we propose an ADA-based single-shot, straightforward, (no reprojection, no iterative refinement), end-to-end training approach via joint optimization on real and synthetic images. Through joint training on real and synthetic data, our network extracts features that are robust to domain shift. These features are then used to estimate the 3D mesh parameters in a single shot with no supervision on real samples. We compute the regression loss on synthetic samples with ground-truth mesh parameters. Knowledge is transferred from synthetic to real data through ADA without direct key point-based supervision.