A New Approach to Hand Tracking and Gesture Recognition by a New Feature Type and HMM

Abstract

In this paper, we introduce a hand gesture recognition system to recognize real time gestures in Vietnamese sign language system. In our system, there are three modules: real time hand tracking, training gesture and gesture recognition using pseudo two dimension hidden Markov models (P2-DHMMs). In the hand tracking module, we introduce a new robust algorithm to obtain hand region, called Tower method, and use skin color for hand gesture tracking and recognition. Next, a gesture recognition system is developed, which can reliably recognize single hand gesture on a standard camera. Furthermore, we propose a new feature type in gesture recognition to improve the accuracy of overall system. In the experiments, we have tested our system to vocabulary of 29 gestures in Vietnamese sign language system (VSL), and show the effectiveness of the system and Tower method.