Template-Based Hand Shape Recovery from a Single Depth Image

Abstract

We present a novel template-based shape recovery pipeline to reconstruct a dense non-rigid hand shape from a single depth image. Our proposed pipeline consists of multiple stages: preprocessing stage, rigid registration stage, and non-rigid registration stage. In the preprocessing stage, a hand point cloud is extracted from a depth image captured by a consumer depth camera. Then the hand template is roughly aligned with the sampled point cloud in the rigid registration stage. Finally, the rigidly aligned template is gradually wrapped to the input point cloud with iterative optimization in the non-rigid registration stage. We formulate the non-rigid surface fit as an optimization problem with a dedicated objective function. A confidence weight regularizer is introduced to facilitate high-quality deformation by maximizing the number of reliable correspondences while suppressing unreliable correspondences. Besides, a varying weigh strategy is employed to adjust the smooth weight of the hand joint regions to a smaller value compared to other hand regions, which allows local non-smooth deformation, thus makes deformations of the hand joint regions more plausible. Moreover, multiple hand joint locations constraints are integrated into our non-rigid registration pipeline to effectively reduce solution space and improve the deformation of the occluded hand regions. Extensive experiments show that our system capable of producing plausible deformations and recovering accurate hand shapes.