Kinect based calling gesture recognition for taking order service of elderly care robot

Abstract

This paper proposes a Kinect-based calling gesture recognition scenario for taking order service of an elderly care robot. The proposed scenarios are designed mainly for helping non expert users like elderly to call service robot for their service request. In order to facilitate elderly service, natural calling gestures are designed to interact with the robot. Our challenge here is how to make the natural calling gesture recognition work in a cluttered and randomly moving objects. In this approach, there are two modes of our calling gesture recognition: Skeleton based gesture recognition and Octree based gesture recognition. Individual people is segmented out from 3D point cloud acquired by Microsoft Kinect, skeleton is generated for each segment, face detection is applied to identify whether the segment is human or not, specific natural calling gestures are designed based on skeleton joints. For the case that user is sitting on a chair or sofa, correct skeleton cannot be generated, Octree based gesture recognition procedure is used to recognize the gesture, in which human segments with head and hand are identified by face detection as well as specific geometrical constrains and skin color evidence. The proposed method has been implemented and tested on “HomeMate”, a service robot developed for elderly care. The performance and results are given.