Real-time Hand Gesture Research Articles

A Neural Network based Real Time Hand Gesture Recognition System

Gesture is habitually used in every day life style. It is so natural way to communicate. Hand gesture recognition method is widely used in the application area of Controlling mouse and/or keyboard functionality, mechanical system, 3D World, Manipulate virtual objects, Navigate in a Virtual Environment, Human/Robot Manipulation and Instruction Communicate at a distance. This paper introduces a real time hand gesture recognition system. This system consists of three stages: image acquisition, feature extraction, and recognition. In the first stage input image of hand gestures are acquiesced by digital camera in approximate frame rate. In second stage a rotation, translation, scaling and orientation invariant feature extraction method has been introduce to extract the feature of the input image based on moment feature extraction method. Finally, a neural network is used to recognize the hand gestures. The performance of the system tested on real data. Based on the experimental results, we noted that this system shows satisfactory performance in hand gesture recognition.

Novel Haar features for real-time hand gesture recognition using SVM

Due to the effect of lighting and complex background, most visual hand gesture recognition systems work only under restricted environments. Here, we propose a robust system which consists of three modules: digital zoom, adaptive skin detection, and hand gesture recognition. The first module detects user face and zooms in so that the face and upper torus take the central part of the image. The second module utilizes the detected user facial color information to detect the other skin color regions like hands. The last module is the most important part for doing both static and dynamic hand gesture recognition. The region of interest next to the detected user face is for fist/waving hand gesture recognition. To classify the dynamic hand gestures under complex background, motion history image and four groups of novel Haar-like features are investigated to classify the dynamic up, down, left, and right hand gestures. A simple efficient algorithm using Support Vector Machine is developed. These defined hand gestures are intuitive and easy for user to control most home appliances. Five users doing 50 dynamic hand gestures at near, medium, and far distances, respectively, were tested under complex environments. Experimental results showed that the accuracy was 95.37 % on average and the processing speed was 3.93 ms per frame. An application integrated with the developed hand gesture recognition was also given to demonstrate the feasibility of proposed system.

Locally regularized sliced inverse regression based 3D hand gesture recognition on a dance robot

Gesture recognition plays an important role in human machine interactions (HMIs) for multimedia entertainment. In this paper, we present a dimension reduction based approach for dynamic real-time hand gesture recognition. The hand gestures are recorded as acceleration signals by using a handheld with a 3-axis accelerometer sensor installed, and represented by discrete cosine transform (DCT) coefficients. To recognize different hand gestures, we develop a new dimension reduction method, locally regularized sliced inverse regression (LR-SIR), to find an effective low dimensional subspace, in which different hand gestures are well separable, following which recognition can be performed by using simple and efficient classifiers, e.g., nearest mean, k-nearest-neighbor rule and support vector machine. LR-SIR is built upon the well-known sliced inverse regression (SIR), but overcomes its limitation that it ignores the local geometry of the data distribution. Besides, LR-SIR can be effectively and efficiently solved by eigen-decomposition. Finally, we apply the LR-SIR based gesture recognition to control our recently developed dance robot for multimedia entertainment. Thorough empirical studies on ‘digits’-gesture recognition suggest the effectiveness of the new gesture recognition scheme for HMI.

Target-shooting exergame with a hand gesture control

Exertion games (exergames) pose interesting challenges in terms of user interaction techniques. Players are commonly unable to use traditional input devices such as mouse and keyboard, given the body movement requirements of this type of videogames. In this work we propose a hand gesture interface to direct actions in a target-shooting exertion game that is played while exercising on an ergo-bike. A vision-based hand gesture interface for interacting with objects in a 3D videogame is designed and implemented. The system is capable to issue game commands to any computer game that normally responds to mouse and keyboard without modifying the underlying source code of the game. The vision system combines Bag-of-features and Support Vector Machine (SVM) to achieve user-independent and real-time hand gesture recognition. In particular, a Finite State Machine (FSM) is used to build the grammar that generates gesture commands for the game. We carried out a user study to gather feedback from participants, and our preliminary results show the high level of interest from users use this multimedia system that implements a natural way of interaction. Albeit some concerns in terms of comfort, users had a positive experience using our exertion game and they expressed their positive intention to use a system like this in their daily lives.

Real Time Multiple Hand Gesture Recognition System for Human Computer Interaction

With the increasing use of computing devices in day to day life, the need of user friendly interfaces has lead towards the evolution of different types of interfaces for human computer interaction. Real time vision based hand gesture recognition affords users the ability to interact with computers in more natural and intuitive ways. Direct use of hands as an input device is an attractive method which can communicate much more information by itself in comparison to mice, joysticks etc allowing a greater number of recognition system that can be used in a variety of human computer interaction applications. The gesture recognition system consist of three main modules like hand segmentation, hand tracking and gesture recognition from hand features. The designed system further integrated with different applications like image browser, virtual game etc. possibilities for human computer interaction. Computer Vision based systems has the potential to provide more natural, non-contact solutions. The present research work focuses on to design and develops a practical framework for real time hand gesture

Handy: A real-time three color glove-based gesture recognizer with learning vector quantization

This paper presents Handy, a real-time hand gesture recognizer based on a three color glove. The recognizer is formed by three modules. The first module, fed by the frame acquired by a webcam, identifies the hand image in the scene. The second module, a feature extractor, represents the image by a nine-dimensional feature vector. The third module, the classifier, is performed by means of learning vector quantization. The recognizer, tested on a dataset of 907 hand gestures, has shown very high recognition rate.

Flexible Neural Trees for Online Hand Gesture Recognition using surface Electromyography

Normal hand gesture recognition methods using surface Electromyography (sEMG) signals require designers to use digital signal processing hardware or ensemble methods as tools to solve real time hand gesture classification. Some methods could also result in complicated computational models, complex circuit connection and lower online recognition ra

A Vision based Geometrical Method to find Fingers Positions in Real Time Hand Gesture Recognition

A novel method to calculate the bended finger’s angle has presented here which could be used to control the electro-mechanical robotic hand. It is assumed that the robotic hand has the human hand like joints and same number of degree of freedom as human hand. In many applications an equipment like human hand is needed, to do the same kind of operation like human do. These days it is easy to make the electro-mechanical robotic hand which has five fingers and same joint but it is not easily controllable as the human hand for accurate work. In our method the hand gesture will be interpreted for controlling the robotic hand. The angles for all the fingers will be calculated and that could be further passed to the robotic hand for controlling its finger. User would perform gesture according to the action as he wants to be done by robotic hand. Here finger positions are detected using geometric modeling of hand in the extracted image Region of interest cropping from the image made the algorithm faster.

Real-time Hand Gestures System for Mobile Robots Control

Autonomous mobile robot navigation in an indoor environment using vision sensor is receiving a considerable attention in current robotics research activities. In this paper, a robot controlled by real-time hand gesture is proposed. This system includes an image pre-processing and feature of extraction state that consists of bounding box and Center-Of-Mass based computation. Through the feature of extraction state, the object's Center-Of-Mass and bounding box attributes are extracted to be applied for gesture sign control. This system could be used in gesture recognition for robot control applications. The result shows the developed mobile robots could be controlled successfully through hand gestures that facilitate the process of human-robot interaction.

Real-Time Static Hand Gesture Recognition for American Sign Language (ASL) in Complex Background

Hand gestures are powerful means of communication among humans and sign language is the most natural and expressive way of communication for dump and deaf people. In this work, real-time hand gesture system is proposed. Experimental setup of the system uses fixed position low-cost web camera with 10 mega pixel resolution mounted on the top of monitor of computer which captures snapshot using Red Green Blue [RGB] color space from fixed distance. This work is divided into four stages such as image preprocessing, region extraction, feature extraction, feature matching. First stage converts captured RGB image into binary image using gray threshold method with noise removed using median filter [medfilt2] and Guassian filter, followed by morphological operations. Second stage extracts hand region using blob and crop is applied for getting region of interest and then “Sobel” edge detection is applied on extracted region. Third stage produces feature vector as centroid and area of edge, which will be compared with feature vectors of a training dataset of gestures using Euclidian distance in the fourth stage. Least Euclidian distance gives recognition of perfect matching gesture for display of ASL alphabet, meaningful words using file handling. This paper includes experiments for 26 static hand gestures related to A-Z alphabets. Training dataset consists of 100 samples of each ASL symbol in different lightning conditions, different sizes and shapes of hand. This gesture recognition system can reliably recognize single-hand gestures in real time and can achieve a 90.19% recognition rate in complex background with a “minimum-possible constraints” approach.

Foreword of Special Issue on “Nomadic Services and Applications”

Foreword of Special Issue on “Nomadic Services and Applications”

Real-time Hand Gesture Recognition by Shape Context Based Matching and Cost Matrix

How to recognize the shape gesture for new human-computer interface without controller required and bring entertainment, games industries and information appliances in new ways. In this paper, we would illustrate a real-time hand gesture recognition system by using shape context matching and cost matrix. The shape context is taken as a basis description for shape matching. It can be regarded as a global characterization descriptor to represent the distribution of points in a set with scale and rotation invariance. In this paper, we developed a perceptual interface for human-computer-interaction based on real-time hand gesture recognition. User could interact with computer program by performing body gesture instead of physical contact. The image of hand gesture was captured from CCD. The hand gesture image was transformed into proper instruction according to the shape information respectively. The instruction was transferred to an appropriate program to execute. The experience of our preliminary results shown the precision rates was up to 70% ~ 90%.

A Human-Machine Interaction Technique: Hand Gesture Recognition Based on Hidden Markov Models with Trajectory of Hand Motion

We have developed an efficient mechanism for real-time hand gesture recognition based on the trajectory of hand motion and the hidden Markov models classifier. In our system, we divide our gestures into single or both hands, one hand have been defined four basic types of directive gesture such as moving upward, downward, leftward, rightward. Then, two hands have twenty-four kinds of combination gesture. However, we apply the most natural and simple way to define eight kinds gestures in our developed human-machine interaction control system so that the users can easily operate the robot. Experimental results reveal that the face tracking rate is more than 97% in general situations and over 94% when the face suffers from temporal occlusion. The efficiency of system execution is very satisfactory, and we are encouraged to commercialize the robot in the near future.

Vision-Based Hand Tracking and Gesture Recognition for Augmented Assembly System

A real time hand tracking and gesture recognition approach which can deal with dynamic backgrounds is presented. This approach is based on computer vision. It segments hand from dynamic backgrounds using the color-based and appearance-based methods. Then, it locates the hand according to marker on the hand. Last, it recognizes the gesture based on geometry constraint of hand. In comparison with the traditional approaches, it provides a good real time performance, is easy to realize, does not require a stationary camera and is not sensitive with intensity different because its gesture recognition does not depend on the templates. Moreover, an augmented assembly system using the presented approach is described. The experiment result of the augmented assembly system demonstrated the effectiveness and robustness of our approach.

Real time vision-based hand gesture estimation for a human-computer interface

In this paper we discuss a technique for real-time hand gesture estimation by user interfaces. First, an image processing algorithm for hand gesture estimation is formulated assuming the hand to be a non-rigid body. A method is then proposed for real-time hand gesture estimation using pipelined parallel processing in real-time. With the proposed method, erroneous finger extraction is eliminated using global hand features to ensure stable gesture recognition. The influence exerted by occlusions on hand gesture estimation is examined and it is shown that this influence can be eliminated by an appropriate arrangement of multiple cameras. The efficiency of the proposed method is verified by building a user interface environment incorporating a prototype setup. In so doing, it is made clear that an acceptable interface environment may be implemented even without complete solution of problems related to occlusions or hand orientation. © 1997 Scripta Technica, Inc. Syst Comp Jpn, 28(7): 76–88, 1997

Real time vision‐based hand gesture estimation for a human‐computer interface

In this paper we discuss a technique for real-time hand gesture estimation by user interfaces. First, an image processing algorithm for hand gesture estimation is formulated assuming the hand to be a non-rigid body. A method is then proposed for real-time hand gesture estimation using pipelined parallel processing in real-time. With the proposed method, erroneous finger extraction is eliminated using global hand features to ensure stable gesture recognition. The influence exerted by occlusions on hand gesture estimation is examined and it is shown that this influence can be eliminated by an appropriate arrangement of multiple cameras. The efficiency of the proposed method is verified by building a user interface environment incorporating a prototype setup. In so doing, it is made clear that an acceptable interface environment may be implemented even without complete solution of problems related to occlusions or hand orientation. © 1997 Scripta Technica, Inc. Syst Comp Jpn, 28(7): 76–88, 1997