BaSICNet: Lightweight 3-D Hand Pose Estimation Network Based on Biomechanical Structure Information for Dexterous Manipulator Teleoperation

Abstract

The use of hand pose for dexterous manipulator teleoperation is an attractive method to the control of the multi-fingered manipulators. Furthermore, the advancement of the deep learning and depth sensors has encouraged the development of the 3D hand pose estimation. However, developing a 3D hand pose estimation method with an accurate and real-time performance is still a difficult task in computer vision. In this paper, a lightweight depth-based network named biomechanical structure information cascade network (BaSICNet) is proposed by considering the global and local structure of human hands to improve the performance through a cascade network and a bone-constraint loss function. In addition, the BaSICNet is applied to a five-fingered dexterous manipulator platform to achieve visual hand-based teleoperation. Extensive evaluations on two public datasets show that the BaSICNet can produce accurate and fast 3D hand poses (9.15mm and 7.59mm mean errors on NYU and MSRA datasets with 114.7 fps), and can achieve superior 3D hand pose estimation balance of accuracy and speed when compared with state-of-the-art (SOAT) methods. Experiment results on the dexterous manipulator platform also show that the BaSICNet can be applied well for teleoperation.