Non-Parametric Anthropometric Graph Convolutional Network for Virtual Mannequin Reconstruction

Yueqi Zhong,Zhicai Yu,Azmat Hussain,Haoyang Xie

doi:10.1109/access.2019.2962833

Yueqi Zhong, Zhicai Yu + Show 2 more

Open Access

https://doi.org/10.1109/access.2019.2962833

Copy DOI

Journal: IEEE Access	Publication Date: Jan 1, 2020
Citations: 6	License type: CC BY 4.0

Affiliation: Donghua University, Bahauddin Zakariya University

Abstract

In this paper, we present a novel non-parametric method for precisely reconstructing a three dimensional (3D) virtual mannequin from anthropometric measurements and mask image(s) based on Graph Convolution Network (GCN). The proposed method avoids heavy dependence on a particular parametric body model such as SMPL or SCPAE and can predict mesh vertices directly, which is significantly more comfortable using a GCN than a typical Convolutional Neural Network (CNN). To further improve the accuracy of the reconstruction and make the reconstruction more controllable, we incorporate the anthropometric measurements into the developed GCN. Our non-parametric reconstruction results distinctly outperform the previous graph convolution method, both visually and in terms of anthropometric accuracy. We also demonstrate that the proposed network possesses the capability to reconstruct a plausible 3D mannequin from a single-view mask. The proposed method can be effortless extended to a parametric method by appending a Multilayer Perception (MLP) to regress the parametric space of the Principal Component Analysis (PCA) model to achieve 3D reconstruction as well. Extensive experimental results demonstrate that our anthropometric GCN itself is very useful in improving the reconstruction accuracy, and the proposed method is effective and robust for 3D mannequin reconstruction.

Highlights

Three dimensional (3D) virtual mannequin plays an essential role in many applications such as virtual try-on, made-tomeasure, sports science, movie industry, personalized entertainment, etc. [1]
Since this paper focuses on reconstructing a mannequin in a standard A-pose, we ignore the pose space introduced by SPCAE and utilize the shape scape only, i.e., Principal Component Analysis (PCA) space
We demonstrate that the proposed anthropometric Graph Convolution Network (GCN) can reconstruct a complete 3D mannequin from a single-view mask, in which we ignore the ResNet18 for the lateral mask and retrain our network

Summary

Introduction

Three dimensional (3D) virtual mannequin plays an essential role in many applications such as virtual try-on, made-tomeasure, sports science, movie industry, personalized entertainment (e.g., augmented reality games and virtual reality), etc. [1]. The overwhelming success of Convolutional Neural Network (CNN) has achieved for 3D human body estimation from image(s). There have been several parametric approaches [7]–[14] that can reconstruct 3D human mesh from image(s) by various CNNs accompany with some parametric body models such as SCAPE [15] and SMPL [16]. 1) Most image-based 3D human reconstruction methods only utilize CNNs as feature extractors to regress the parametric space corresponding to a particular parametric body model rather than a real 3D shape, i.e., the 3D body is not reconstructed explicitly and directly. 2) the existing approaches can estimate the complete shape and pose from image(s), the regression results are challenging to be used in some applications that have requirements on accuracy, such as made-to-measure, since they lack constraints SMPL cannot model facial expressions [17], [18]. 2) the existing approaches can estimate the complete shape and pose from image(s), the regression results are challenging to be used in some applications that have requirements on accuracy, such as made-to-measure, since they lack constraints

Objectives

Results

Conclusion