Gesture-World Environment Technology for Mobile Manipulation – Remote Control System of a Robot with Hand Pose Estimation –

Abstract

The purpose of this paper is to propose a remotecontrolled robot system capable of accurate highspeed performance of the same operation strictly conforming to human operator movement without sensors or special control means. We specifically intend to implement high-precision high-speed 3D hand pose estimation enabling a remote-controlled robot to be operated using two cameras installed loosely orthogonally using one ordinary PC. The two cameras have their own database. Once sequential hand images are shot at high speed, the system starts selecting one database with bigger size of hand region in each recorded image. Coarse screening then proceeds based on proportional hand image information roughly corresponding to wrist rotation or thumb or finger extension. Finally, a detailed search is done for similarity among selected candidates. Experiments show that mean and standard deviation scores of errors in estimated angles at the proximal interphalangeal (PIP) index are 0.45 ± 14.57 and at the carpometacarpal (CM) thumb 4.7 ± 10.82, respectively, indicating it as a high-precision 3D hand pose estimation. Remote control of a robot with the proposed vision system shows high performance as well.