A Framework for Real-Time Hand Gesture Recognition in Uncontrolled Environments with Partition Matrix Model Based on Hidden Conditional Random Fields

Abstract

The main obstructions of making hand gesture recognition methods robust in real-world applications are the challenges from the uncontrolled environments, including: gesturing hand out of the scene, pause during gestures, complex background, skin-coloured regions moving in background, performers wearing short sleeve and face overlapping with hand. Therefore, a framework for real-time hand gesture recognition in uncontrolled environments is proposed in this paper. A novel tracking scheme is proposed to track multiple hand candidates in unconstrained background, and a weighting model for gesture classification based on Hidden Conditional Random Fields which takes trajectories of multiple hand candidates under different frame rates into consideration is also introduced. The framework achieved invariance under change of scale, speed and location of the hand gestures. The Experimental results of the proposed framework on Palm Graffiti Digits database and Warwick Hand Gesture database show that it can perform well in uncontrolled environments.