Face tracking and hand gesture recognition for human-robot interaction

Abstract

The interaction between man and machines has become an important topic for the robotic community as it can generalise the use of robots. For active H/R interaction scheme, the robot needs to detect human faces in its vicinity and then interpret canonical gestures of the tracked person assuming this interlocutor has been beforehand identified. In this context, we depict functions suitable to detect and recognise faces in video stream and then focus on face or hand tracking functions. An efficient colour segmentation based on a watershed on the skin-like coloured pixels is proposed. A new measurement model is proposed to take into account both shape and colour cues in the particle filter to track face or hand silhouettes in video stream. An extension of the basic condensation algorithm is proposed to achieve recognition of the current hand posture and automatic switching between multiple templates in the tracking loop. Results of tracking and recognition using are illustrated in the paper and show the process robustness in cluttered environments and in various light conditions. The limits of the method and future works are also discussed.