Vision-Based Approach for Real-Time Hand Detection and Gesture Recognition

Abstract

The need for interaction between humans and computer devices has advanced so much in recent years. Human-computer interaction (HCI) is the direct manipulation of graphic objects, in particular 3D objects, using several predefined gestures. A gesture can be defined as a physical movement of the hands, face, eyes, and body of the human and is an essential component of the language generation process. In HCI, the use of hand gestures provides an intuitive, attractive, and natural alternative for the interaction between the user and the computer. In this chapter, we present and discuss several vision-based approaches for real-time hand detection which represents the main challenge in real-time HCI applications. Then, we propose a new vision-based approach based on skin color detection for real-time hand detection and gesture recognition. We try to reduce as much as possible the constraints and limitations of the existing approaches. Our method for hand segmentation detects the user’s hand(s), even if the user’s face or other people are viewed by the camera, and can know if the user is doing a gesture using one hand or two hands. Also, our method is robust to the scene’s colors and illumination conditions, as it corrects the video’s colors and brightness before performing the hand segmentation. We also study the performance of hand detection using three color spaces. We show that the RGB mask was the best one. Our system can recognize several hand gestures and allows the user to manipulate a 2D image in real time.