Feature Bridging Networks for 3D Human Body Shape Estimation from a Single Depth Map

Abstract

This paper describes a novel deep neural network architecture to reconstruct an accurate human body shape from a single depth map. The proposed method utilizes a statistical parametric body shape model, which represents a wide variety of body shape with low-dimensional body shape parameters. We formulate the body shape reconstruction as a regression problem of the body shape parameters. One of the biggest challenges of the single-image shape reconstruction lies in a gap between input and output modalities. This is because an input depth map only contains a surface of a human body, while the output is a full 3D body shape model. To bridge this gap, we utilize dedicated two deep neural networks ShapeEncoder and DepthMapEncoder, which respectively process the 3D body model and the depth map. These two networks are bridged with a learned latent body feature space to enable accurate single-image body shape estimation. Furthermore, the proposed method also uses body joint positions estimated from the depth map to further improve the performance. The proposed approach is evaluated on real depth maps taken from 30 subjects and achieves significant performance improvements over the existing methods. Contribution-Multiple deep neural networks form a novel feature bridging architecture to achieve significant performance improvements.