Networks Effectively Utilizing 2D Spatial Information for Accurate 3D Hand Pose Estimation

Abstract

In this work, we propose a new method for accurate 3D hand pose estimation from a single depth map using convolutional neural networks (CNN). Our method effectively makes use of 2D spatial information to improve the performance by two means. Firstly, we formulate 3D hand pose estimation as a two-task (2D joints detection and depth regression) problem so that we can directly utilize the ability of hourglass module on processing multi-scale information for estimating 2D joint coordinates. Secondly, 2D spatial information is used to help depth regression by introducing the spatial attention mechanism to our method. The experimental results demonstrate that our method achieves the state-of-the-art performance on ICVL hand posture dataset and a comparable performance with the state-of-the-arts on NYU dataset.