Lightweight 3D hand pose estimation by cascading CNNs with reinforcement learning

Abstract

This paper proposes a novel strategy for lightweight 3D hand pose estimation. The strategy decomposes the estimation process into feature extraction and feature exploitation, where feature extraction performs dimension reduction on the original input and outputs feature vectors. Feature exploitation is further analyzed and considered as a path optimization problem, and reinforcement learning (RL) is proved to be capable of tackling the problem accurately. A framework cascading convolutional neural networks (CNNs) and RL is next introduced to validate the effectiveness of the proposed strategy, where two different backbones are used to extract features, and RL is extended into continuous space to enhance accuracy. Ablation studies and experiments are carried out on NYU and ICVL datasets using the proposed strategy with continuous RL. The results show that the accuracy of continuous RL exceeds discrete RL, and the rapidity and accuracy leads the backbones. Comparative studies show the strategy achieves leading rapidity and accuracy in single-view depth-based methods.