Detailed 3D Human Body Reconstruction From a Single Image Based on Mesh Deformation

Abstract

In the research fields of 3D animation, virtual reality, etc., it is a very important task to reconstruct a 3D human body model with the surface textures. However, due to factors such as diversities of human body poses, uncertainty of depth, and a wide range of viewpoints, the reconstruction of the 3D human body from a single image is an enormous challenge. To solve the aforementioned challenge, this paper proposes a novel architecture to reconstruct the 3D human mesh with great surface details. The key point of the proposed architecture is to combine the topology of the SMPL (Skinned Multi-Person Linear) template mesh and the flexible mesh deformation. Firstly, the typical 2D CNN architecture, ResNet-50, extracts the perceptual features from the input image. The image features are fused into the 3D space via perceptual feature transformation. Our approach does not rely on the parameter space, although the mesh topology of the SMPL is preserved. The approach fully exploits the ability of the SMPL to represent various human body shapes, while avoiding the problem of the spatial loss caused by the parametric method. Secondly, to recover more details of the human body, a graph-based convolutional network is adopted to deform the 3D human body mesh. The network allows starting with fewer vertices and distributing vertex features to the representative positions in a coarse-to-fine manner, which fully utilizes the mesh topology. Experiments demonstrate that our approach can generate 3D human body mesh with great details qualitatively. Compared with the state-of-the-art methods, our approach also achieves better accuracy quantitatively.