Neural hand reconstruction using an RGB Image

Abstract

Background This study presents a neural hand reconstruction method for monocular 3D hand pose and shape estimation. Methods Alternate to directly representing hand with 3D data, a novel UV position map is used to represent a hand pose and shape with 2D data that maps 3D hand surface points to 2D image space. Furthermore, an encoder-decoder neural network is proposed to infer such UV position map from a single image. To train this network with inadequate ground truth training pairs, we propose a novel MANOReg module that employs MANO model as a prior shape to constrain high-dimensional space of the UV p sition map. Results The quantitative and qualitative experiments demonstrate the effectiveness of our UV position map representation and MANOReg module.