Robust methods for hand gesture spotting and recognition using Hidden Markov Models and Conditional Random Fields

Abstract

This paper proposes an automatic method that handles hand gesture spotting and recognition simultaneously. To spot meaningful gestures of numbers (0–9) accurately, a stochastic method for designing a non-gesture model with Hidden Markov Models (HMMs) versus Conditional Random Fields (CRFs) is proposed without training data. The non-gesture model provides a confidence measure that is used as an adaptive threshold to find the start and the end point of meaningful gestures, which are embedded in the input video stream. To reduce the states number of the non-gesture model with HMMs, similar probability distributions states are merged based on relative entropy measure. Additionally, the weights of self-transition feature functions are increased for short gesture to further improve the accuracy of gesture spotting and recognition with CRFs. Experimental results show that; the proposed method can successfully spot and recognize meaningful gestures with 93.31% and 90.49% reliability for HMMs and CRFs respectively. In addition, the model inference by HMMs are faster and the saving time is 66.42% using relative entropy. The reliability of CRFs method is improved from 86.12% to 90.49% using short gesture detector.