Hand detection and segmentation using multimodal information from Kinect

Abstract

Nowadays, hand gestures are becoming one of the most natural and intuitive ways of communication between human and computer. To this end, a complex process including hand gesture acquisition, hand detection, gesture representation and recognition must be carried out. This paper presents a method that detects hand and segments hand regions from images captured by a Kinect sensor. As Kinect sensor provides not only RGB images as conventional camera, but also depth and skeleton, in our work, we incorporate multi-modal data from Kinect to deal with hand detection and segmentation. Specifically, we use skeleton to approximately determine hand palm. Then a skin based detector will be applied to discard non-skin pixels from the region of interest. Using depth data helps to limit the human body regions and remove false positive regions from the previous steps. Finally, morphological operations will be applied to fill holes in the hand region. The main advantage of this method is very easy to implement and it performs in real-time on an ordinary computer. We evaluate the proposed method on a dataset of hand gestures captured from different viewpoints. Experiment shows that it provides reasonable accuracy at very high frame rate. It also produces comparable performance in comparison with deep learning based methods.