Real-time viewpoint-invariant hand localization with cluttered backgrounds

Abstract

Over the past few years there has been a growing interest in visual interfaces based on gestures. Using gestures as a mean to communicate with a computer can be helpful in applications such as gaming platforms, domotic environments, augmented reality or sign language interpretation to name a few. However, a serious bottleneck for such interfaces is the current lack of accurate hand localization systems, which are necessary for tracking (re-)initialization and hand pose understanding. In fact, human hand is an articulated object with a very large degree of appearance variability which is difficult to deal with. For instance, recent attempts to solve this problem using machine learning approaches have shown poor generalization capabilities over different viewpoints and finger spatial configurations.In this article we present a model based approach to articulated hand detection which splits this variability problem by separately searching for simple finger models in the input image. A generic finger silhouette is localized in the edge map of the input image by combining curve and graph matching techniques. Cluttered backgrounds and thick textured images, which usually make it hard to compare edge information with silhouette models (e.g., using chamfer distance or voting based methods) are dealt with in our approach by simultaneously using connected curves and topological information. Finally, detected fingers are clustered using geometric constraints. Our system is able to localize in real time a hand with variable finger configurations in images with complex backgrounds, different lighting conditions and different positions of the hand with respect to the camera. Experiments with real images and videos and a simple visual interface are presented to validate the proposed method.