Real-Time Hand Detection and Gesture Tracking with GMM and Model Adaptation

Abstract

Hand gestures are an efficient manner for human computer interaction (HCI). They can also be used for the development of a non-intrusive biometrics system. In this paper, we address the issues of hand detection and gesture tracking using a single camera. A simple yet effective approach is proposed for applications with complex backgrounds and minimal constraints on the subject. A hand detection approach is presented using a Bayesian classifier based on Gaussian Mixture Models (GMM) for identifying pixels of skin color. A connected component based region-growing algorithm is included for forming areas of skin pixels into areas of likely hand candidates. Given the detected hand region, we further detect the hand features using a deformable model for hand gesture estimation. We propose a novel method, a 3D physics-based dynamic mesh adaptation approach, to estimate and track hand shape and finger directions. The physics-based hand model adaptation algorithm allows us to model hand shape and orientation at the same time, thereby improving the robustness and speed for hand gesture tracking and regeneration.KeywordsGaussian Mixture ModelGesture RecognitionHand GestureSkin ToneHand ShapeThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.