TriHorn-Net: A model for accurate depth-based 3D hand pose estimation

Abstract

Listen

Translate article

3D hand pose estimation methods have made significant progress recently. However, estimation accuracy is often far from sufficient for specific real-world applications, and thus there is significant room for improvement. This paper proposes TriHorn-Net, a novel model that uses specific innovations to improve hand pose estimation accuracy on depth images. The first innovation is decomposition of the 3D hand pose estimation into the estimation of 2D joint locations in the depth image space (UV), and the estimation of their corresponding depths aided by two complementary attention maps. This decomposition prevents depth estimation, which is a more difficult task, from interfering with the UV estimations at both the prediction and feature levels. The second innovation is PixDropout, which is, to the best of our knowledge, the first appearance-based data augmentation method for hand depth images. Experimental results demonstrate that the proposed model outperforms the state-of-the-art methods on three public benchmark datasets, achieving 5.73 mm, 7.13 mm, and 7.68 mm on the ICVL, MSRA, and NYU datasets, respectively. The proposed model achieves this performance with a relatively fast average inference speed of 9.25 ms per frame on an NVIDIA 1080Ti GPU, which, as discussed in the paper, places the proposed model among the fastest-performing 3D hand pose estimation models. Our implementation is available at https://github.com/mrezaei92/TriHorn-Net.