Abstract
3D hand pose estimation from a single depth image plays an important role in computer vision and human-computer interaction. Although recent hand pose estimation methods using convolution neural network (CNN) have shown notable improvements in accuracy, most of them have a limitation that they rely on a complex network structure without fully exploiting the articulated structure of the hand. A hand, which is an articulated object, is composed of six local parts: the palm and five independent fingers. Each finger consists of sequential-joints that provide constrained motion, referred to as a kinematic chain. In this paper, we propose a hierarchically-structured convolutional recurrent neural network (HCRNN) with six branches that estimate the 3D position of the palm and five fingers independently. The palm position is predicted via fully-connected layers. Each sequential-joint, i.e. finger position, is obtained using a recurrent neural network (RNN) to capture the spatial dependencies between adjacent joints. Then the output features of the palm and finger branches are concatenated to estimate the global hand position. HCRNN directly takes the depth map as an input without a time-consuming data conversion, such as 3D voxels and point clouds. Experimental results on public datasets demonstrate that the proposed HCRNN not only outperforms most 2D CNN-based methods using the depth image as their inputs but also achieves competitive results with state-of-the-art 3D CNN-based methods with a highly efficient running speed of 285 fps on a single GPU.
Highlights
Accurate 3D hand pose estimation has received considerable attention regarding a wide range of applications, such as virtual/augmented reality and human-computer interaction [1]
Based on the aforementioned results, it can be seen that our proposed hierarchically-structured convolutional recurrent neural network (HCRNN) achieves competitive performance compared with state-of-the-art methods and is very efficient, having a high frame rate, which shows the applicability to real-time applications
To design a practical architecture for 3D hand pose estimation, we considered the articulated structure of the hand and proposed an efficient regression network, namely termed HCRNN
Summary
Accurate 3D hand pose estimation has received considerable attention regarding a wide range of applications, such as virtual/augmented reality and human-computer interaction [1]. In these conventional CNN-based methods, there are two major approaches to improve estimation accuracy. To utilize 3D spatial information, Ge et al and Moon et al converted a depth image into a volumetric representation, such as 3D voxels, and applied a 3D CNN for 3D hand pose estimation [3]–[5]. 3D representation of inputs based on a 3D point cloud has been proposed [6]–[8]. These methods are effective for capturing the geometric properties of depth images [3], [5], they suffer from heavy parameters and complex data-conversion processes, resulting in high time complexity. We adopt a 2D depth image itself as input without a
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have