Gestuelle: A System to Recognize Dynamic Hand Gestures using Hidden Markov Model to control Windows Applications

Abstract

Human Computer Interaction has always been a challenging adventure for researchers. Communication between computers and humans, just as humans interact with one another has been the prime objective of HCI research. Many efforts have gone into Speech and Gesture Recognition to develop an approach that would allow users to interact with their system by using their voice or simple intuitive gestures as against sitting in front of the computer and using a mouse or keyboard. Natural interaction must be fast, convenient, effective and reliable. This paper introduces an application, “Gestuelle” that makes use of simple gestures to operate on common windows applications such as Windows Media Player, Live Photo Gallery, Power Point, Notepad etc. The idea is to develop a system that can recognize dynamic hand gestures by means of a simple web camera to control the computer even from a distance, without having to use a keyboard and mouse all the time. Gestuelle provides a cheap and easily portable solution to the everyday user as against using an expensive Microsoft Kinect or high resolution cameras or sensors to accomplish the same task. The system makes use of the Hidden Markov Model (HMM), works in real time and is designed to work in static backgrounds. The system makes use of LRB topology of HMM in conjunction with the Baum Welch Algorithm for training and the Forward and Viterbi Algorithms for testing and evaluating the input observation sequences and generating the best possible state sequence for pattern recognition.