Hand Gesture Recognition System Research Articles

Hand Cricket Game Application Using Computer Vision

Hand cricket is an attempt to experience the childhood entrainment of playing cricket using our hands. This paper presents an application built to play Hand cricket using a computer vision-based real-time 3D hand gesture recognition system. To play the game the user will give inputs using hand gestures. These hand gestures will be identified using a real-time computer vision system that runs based on a CNN model. Since in this game both players must play simultaneously the computer also comes up with a number from 1 to 6 when a hand gesture is being read and recognized. The core engine of building this app is an image classifier that classifies a picture of a hand into 1,2,3,4,5,6 and none. This paper shows an interactive simple game application developed, that tries to make the game feel more natural and as if playing with a real person.

Deep Learning-Based Hand Gesture Recognition System and Design of a Human–Machine Interface

Deep Learning-Based Hand Gesture Recognition System and Design of a Human–Machine Interface

Realtime Recognition of Dynamic Hand Gestures in Practical Applications

Dynamic hand gesture acting as a semaphoric gesture is a practical and intuitive mid-air gesture interface. Nowadays benefiting from the development of deep convolutional networks, the gesture recognition has already achieved a high accuracy, however, when performing a dynamic hand gesture such as gestures of direction commands, some unintentional actions are easily misrecognized due to the similarity of the hand poses. This hinders the application of dynamic hand gestures and cannot be solved by just improving the accuracy of the applied algorithm on public datasets, thus it is necessary to study such problems from the perspective of human-computer interaction. In this article, two methods are proposed to avoid misrecognition by introducing activation delay and using asymmetric gesture design. First the temporal process of a dynamic hand gesture is decomposed and redefined, then a realtime dynamic hand gesture recognition system is built through a two-dimensional convolutional neural network. In order to investigate the influence of activation delay and asymmetric gesture design on system performance, a user study is conducted and experimental results show that the two proposed methods can effectively avoid misrecognition. The two methods proposed in this article can provide valuable guidance for researchers when designing realtime recognition system in practical applications.

Hand Gesture Recognition System based on 60 GHz FMCW Radar and Deep Neural Network

The proposed study provides a novel technique for recognizing hand gestures that use a combination of Deep Convolutional Neural Networks (DCNN) and 60 GHz Frequency Modulated Continuous Wave (FMCW) radar. The motion of a Human's hand is detected using the FMCW radar, and the various gestures are classified using the DCNN. Motion detection and frequency analysis are two techniques that the suggested system combines. The basis of the capability of motion detection in FMCW radars' is to recognize the Doppler shift in the received signal brought on by the target's motion. To properly identify the hand motions, the presented technique combines these two techniques. The system is analyzed using a collection of hand gesture photos, and the outcomes are analyzed with those of other hand gesture recognition systems which are already in use. A dataset of five different hand gestures is used to examine the proposed system. According to the experimental data, the suggested system can recognize gestures with an accuracy of 96.5%, showing its potential as a productive gesture recognition system. Additionally, the suggested system has a processing time of 100 ms and can run in real time. The outcomes also demonstrate the proposed system's resistance to noise and its ability to recognize gestures in a variety of configurations. For gesture detection applications in virtual reality and augmented reality systems, this research offers a promising approach.

Analysis and Evaluation of Feature Selection and Feature Extraction Methods

Hand gestures are widely used in human-to-human and human-to-machine communication. Therefore, hand gesture recognition is a topic of great interest. Hand gesture recognition is closely related to pattern recognition, where overfitting can occur when there are many predictors relative to the size of the training set. Therefore, it is necessary to reduce the dimensionality of the feature vectors through feature selection techniques. In addition, the need for portability in hand gesture recognition systems limits the use of deep learning algorithms. In this sense, a study of feature selection and extraction methods is proposed for the use of traditional machine learning algorithms. The feature selection methods analyzed are: maximum relevance and minimum redundancy (MRMR), Sequential, neighbor component analysis without parameters (NCAsp), neighbor component analysis with parameters (NCAp), Relief-F, and decision tree (DT). We also analyze the behavior of feature selection methods using classification and recognition accuracy and processing time. Feature selection methods were fed through seventeen feature extraction functions, which return a score proportional to its importance. The functions are then ranked according to their scores and fed to machine learning algorithms such as Artificial Neural Networks (ANN), Support Vector Machine (SVM), K-Nearest Neighbor (KNN), and Decision Tree (DT). This work demonstrates that all feature selection methods evaluated on ANN provide better accuracy. In addition, the combination and number of feature extraction functions influence the accuracy and processing time.

Design of a miniaturized wearable EIT system for imaging and hand gesture recognition

Abstract Hand gesture recognition using data from electrical impedance tomography (EIT) systems offers many promising applications, for example in the field of human-computer interaction. Due to its real-time capability and the use of harmless currents for humans, it can be used in medicine, robotics, or virtual environments. As already shown in similar works, for example by Zhang et al. finding a good compromise between accuracy, precision, framerate and the size of the system is a challenge [1], [2]. This work presents a truly wearable compact EIT system on a single 24.9 mm × 22.5 mm circuit board consisting of standard components without application-specific integrated circuit. A neural network (NN) classifies two gestures. It is able to distinguish two different gestures with an accuracy of 78,33 % and a precision of 76,56 %. This work has a strong focus on the size of the system and provides a starting point for further research in compact wearable gesture recognition systems. It shows the challenge of the compromise between size and quality of the signals.

An Intelligent Heuristic Manta-Ray Foraging Optimization and Adaptive Extreme Learning Machine for Hand Gesture Image Recognition

The development of hand gesture recognition systems has gained more attention in recent days, due to its support of modern human-computer interfaces. Moreover, sign language recognition is mainly developed for enabling communication between deaf and dumb people. In conventional works, various image processing techniques like segmentation, optimization, and classification are deployed for hand gesture recognition. Still, it limits the major problems of inefficient handling of large dimensional datasets and requires more time consumption, increased false positives, error rate, and misclassification outputs. Hence, this research work intends to develop an efficient hand gesture image recognition system by using advanced image processing techniques. During image segmentation, skin color detection and morphological operations are performed for accurately segmenting the hand gesture portion. Then, the Heuristic Manta-ray Foraging Optimization (HMFO) technique is employed for optimally selecting the features by computing the best fitness value. Moreover, the reduced dimensionality of features helps to increase the accuracy of classification with a reduced error rate. Finally, an Adaptive Extreme Learning Machine (AELM) based classification technique is employed for predicting the recognition output. During results validation, various evaluation measures have been used to compare the proposed model's performance with other classification approaches.

A Review of Hand Gesture Recognition Systems Based on Noninvasive Wearable Sensors

Hand gesture, one of the essential ways for a human to convey information and express intuitive intention, has a significant degree of differentiation, substantial flexibility, and high robustness of information transmission to make hand gesture recognition (HGR) one of the research hotspots in the fields of human–human and human–computer or human–machine interactions. Noninvasive, on‐body sensors can monitor, track, and recognize hand gestures for various applications such as sign language recognition, rehabilitation, myoelectric control for prosthetic hands and human–machine interface (HMI), and many other applications. This article systematically reviews recent achievements from noninvasive upper‐limb sensing techniques for HGR, multimodal sensing fusion to gain additional user information, and wearable gesture recognition algorithms to obtain more reliable and robust performance. Research challenges, progress, and emerging opportunities for sensor‐based HGR systems are also analyzed to provide perspectives for future research and progress.

Computer vision-based hand gesture recognition for human-robot interaction: a review

As robots have become more pervasive in our daily life, natural human-robot interaction (HRI) has had a positive impact on the development of robotics. Thus, there has been growing interest in the development of vision-based hand gesture recognition for HRI to bridge human-robot barriers. The aim is for interaction with robots to be as natural as that between individuals. Accordingly, incorporating hand gestures in HRI is a significant research area. Hand gestures can provide natural, intuitive, and creative methods for communicating with robots. This paper provides an analysis of hand gesture recognition using both monocular cameras and RGB-D cameras for this purpose. Specifically, the main process of visual gesture recognition includes data acquisition, hand gesture detection and segmentation, feature extraction and gesture classification, which are discussed in this paper. Experimental evaluations are also reviewed. Furthermore, algorithms of hand gesture recognition for human-robot interaction are examined in this study. In addition, the advances required for improvement in the present hand gesture recognition systems, which can be applied for effective and efficient human-robot interaction, are discussed.

Snapture—a Novel Neural Architecture for Combined Static and Dynamic Hand Gesture Recognition

As robots are expected to get more involved in people’s everyday lives, frameworks that enable intuitive user interfaces are in demand. Hand gesture recognition systems provide a natural way of communication and, thus, are an integral part of seamless human-robot interaction (HRI). Recent years have witnessed an immense evolution of computational models powered by deep learning. However, state-of-the-art models fall short of expanding across different gesture domains, such as emblems and co-speech. In this paper, we propose a novel hybrid hand gesture recognition system. Our Snapture architecture enables learning both static and dynamic gestures: by capturing a so-called snapshot of the gesture performance at its peak, we integrate the hand pose and the dynamic movement. Moreover, we present a method for analyzing the motion profile of a gesture to uncover its dynamic characteristics, which allows regulating a static channel based on the amount of motion. Our evaluation demonstrates the superiority of our approach on two gesture benchmarks compared to a state-of-the-art CNNLSTM baseline. Our analysis on a gesture class basis unveils the potential of our Snapture architecture for performance improvements using RGB data. Thanks to its modular implementation, our framework allows the integration of other multimodal data, like facial expressions and head tracking, which are essential cues in HRI scenarios, into one architecture. Thus, our work contributes both to integrative gesture recognition research and machine learning applications for non-verbal communication with robots.

Flex Sensor and Machine Learning-Based Mute Communication System

Individuals facing speech impairments encounter significant communication challenges. To address this, we have developed a groundbreaking system that seamlessly translates custom hand gestures into written text and spoken language. Our system combines flex sensors and state-of-the-art machine learning techniques, empowering individuals with speech impairments to express themselves effectively and improve their overall communication experience. By connecting flex sensors to an Arduino board and utilizing MATLAB for robust data collection, analysis, and processing, we achieve a highly accurate gesture recognition system. Through meticulous steps of data labeling, rigorous training of a machine learning model, and thorough accuracy evaluations, our system recognizes a wide range of gestures, including numbers 0-9, letters A-Z, and 10 Kannada language gestures conveying meaningful phrases. The hand gesture recognition system achieved notable performance with an accuracy of 83.5%, precision of 88.6%, recall of 95.1%, and F1 Score of 91.7%. Key Words: speech impairments, hand gestures, gesture recognition, flex sensors,

Textural feature descriptors for a static and dynamic hand gesture recognition system

Textural feature descriptors for a static and dynamic hand gesture recognition system

An Exploration into Human–Computer Interaction: Hand Gesture Recognition Management in a Challenging Environment

Scientists are developing hand gesture recognition systems to improve authentic, efficient, and effortless human–computer interactions without additional gadgets, particularly for the speech-impaired community, which relies on hand gestures as their only mode of communication. Unfortunately, the speech-impaired community has been underrepresented in the majority of human–computer interaction research, such as natural language processing and other automation fields, which makes it more difficult for them to interact with systems and people through these advanced systems. This system’s algorithm is in two phases. The first step is the Region of Interest Segmentation, based on the color space segmentation technique, with a pre-set color range that will remove pixels (hand) of the region of interest from the background (pixels not in the desired area of interest). The system’s second phase is inputting the segmented images into a Convolutional Neural Network (CNN) model for image categorization. For image training, we utilized the Python Keras package. The system proved the need for image segmentation in hand gesture recognition. The performance of the optimal model is 58 percent which is about 10 percent higher than the accuracy obtained without image segmentation.

A Deep Q-Network based hand gesture recognition system for control of robotic platforms

Hand gesture recognition (HGR) based on electromyography signals (EMGs) and inertial measurement unit signals (IMUs) has been investigated for human-machine applications in the last few years. The information obtained from the HGR systems has the potential to be helpful to control machines such as video games, vehicles, and even robots. Therefore, the key idea of the HGR system is to identify the moment in which a hand gesture was performed and it’s class. Several human-machine state-of-the-art approaches use supervised machine learning (ML) techniques for the HGR system. However, the use of reinforcement learning (RL) approaches to build HGR systems for human-machine interfaces is still an open problem. This work presents a reinforcement learning (RL) approach to classify EMG-IMU signals obtained using a Myo Armband sensor. For this, we create an agent based on the Deep Q-learning algorithm (DQN) to learn a policy from online experiences to classify EMG-IMU signals. The HGR proposed system accuracy reaches up to 97.45 pm 1.02% and 88.05 pm 3.10% for classification and recognition respectively, with an average inference time per window observation of 20 ms. and we also demonstrate that our method outperforms other approaches in the literature. Then, we test the HGR system to control two different robotic platforms. The first is a three-degrees-of-freedom (DOF) tandem helicopter test bench, and the second is a virtual six-degree-of-freedom (DOF) UR5 robot. We employ the designed hand gesture recognition (HGR) system and the inertial measurement unit (IMU) integrated into the Myo sensor to command and control the motion of both platforms. The movement of the helicopter test bench and the UR5 robot is controlled under a PID controller scheme. Experimental results show the effectiveness of using the proposed HGR system based on DQN for controlling both platforms with a fast and accurate response.

Static and dynamic hand gesture recognition system with deep convolutional levy flight whale optimization

Static and dynamic hand gesture recognition system with deep convolutional levy flight whale optimization

Empowering Hand Rehabilitation with AI-Powered Gesture Recognition: A Study of an sEMG-Based System.

Stroke is one of the most prevalent health issues that people face today, causing long-term complications such as paresis, hemiparesis, and aphasia. These conditions significantly impact a patient's physical abilities and cause financial and social hardships. In order to address these challenges, this paper presents a groundbreaking solution-a wearable rehabilitation glove. This motorized glove is designed to provide comfortable and effective rehabilitation for patients with paresis. Its unique soft materials and compact size make it easy to use in clinical settings and at home. The glove can train each finger individually and all fingers together, using assistive force generated by advanced linear integrated actuators controlled by sEMG signals. The glove is also durable and long-lasting, with 4-5 h of battery life. The wearable motorized glove is worn on the affected hand to provide assistive force during rehabilitation training. The key to this glove's effectiveness is its ability to perform the classified hand gestures acquired from the non-affected hand by integrating four sEMG sensors and a deep learning algorithm (the 1D-CNN algorithm and the InceptionTime algorithm). The InceptionTime algorithm classified ten hand gestures' sEMG signals with an accuracy of 91.60% and 90.09% in the training and verification sets, respectively. The overall accuracy was 90.89%. It showed potential as a tool for developing effective hand gesture recognition systems. The classified hand gestures can be used as a control command for the motorized wearable glove placed on the affected hand, allowing it to mimic the movements of the non-affected hand. This innovative technology performs rehabilitation exercises based on the theory of mirror therapy and task-oriented therapy. Overall, this wearable rehabilitation glove represents a significant step forward in stroke rehabilitation, offering a practical and effective solution to help patients recover from stroke's physical, financial, and social impact.

Novel Time-Distance Parameters Based Hand Gesture Recognition System Using Multi-UWB Radars

Hand gesture recognition (HGR) has a vital role to develop an intelligent interface for human–computer interaction. This letter proposed an automated system for HGR using three ultrawideband radars. In this system, the received radar reflection is measured for each hand gesture action which is recorded with respect to time duration and distance of hand from radar during hand motion. A time-distance matrix is formed for each gesture action, and reflection density is obtained for time-distance analysis. Furthermore, mean density function-based parameters are suggested, namely, reflection density per unit time, reflection density per unit distance, and mixed moments of time and distance with higher order functions. Features are computed in entire time-distance region and further limited to local time-distance ranges to extract the gesture distinguishable information. Computed features are ranked and given to the classification model. The achieved high accuracy justifies the primacy of features and the developed system.

A comparison of EMG-based hand gesture recognition systems based on supervised and reinforcement learning

Hand gesture recognition (HGR) based on electromyography signals (EMGs) has been one of the most relevant research topics in the human–machine interfaces field in recent years. The HGR systems are aimed at identifying the moment in which a hand gesture was performed as well as the gesture category. To date, several HGR state-of-the-art methods are based mainly on supervised machine learning (ML) techniques. However, the use of reinforcement learning (RL) approaches to classify EMGs has not yet been thoroughly evaluated. Moreover, the behavior of HGR systems based on ML and RL methods on large datasets for user-general HGR systems is still an open research problem. In the present work, we compare a supervised learning with a reinforcement learning HGR system, which are composed of the following stages: pre-processing, feature extraction, classification, and post-processing. We compared the performance of using both a supervised and a reinforcement learning method to classify and recognize EMGs for six different hand gestures. We performed experiments by using training, validation, and test sets on the EMG-EPN-612 public dataset, and the results were evaluated for user-general HGR models. The final accuracy results on the test set demonstrate that the best model was obtained for the supervised learning method, reaching up to 90.49%±9.7% and 86.83%±11.30% for classification and recognition accuracy respectively. The results obtained in this work demonstrated that supervised learning methods outperform reinforcement learning methods for user-general HGR systems based on EMGs for the EMG-EPN-612 dataset distribution.

Deep learning based spatio‐temporal hand gesture recognition system in complex environment

AbstractGesture recognition nowadays has grabbed the attention of researchers as they represent human behaviour in multiple practical ways. Amongst a variety of gestures available, hand gestures play an essential role in the field of human‐computer interaction when recognised efficiently in complex and dynamic environments. In this paper, we propose a dynamic hand gesture recognition system to recognise hand gestures appearing in different indoor and outdoor environments. Hand detection and tracking uses a two‐level system resulting in the formation of gesture trajectory in challenging conditions in which existing detection and tracking algorithms could not do so. A set of 45 features is provided as input to the various classification techniques. The redundancy problem has been reduced by selecting a set of optimum features using the analysis of variance method, which ranks the list of features. An incremental feature selection technique calculates recognition accuracy by selecting features according to rankings. This system provides an accuracy of 96.32% when used with machine learning and 97.5% when used with deep learning techniques. Recognition accuracy is calculated for various environments, including an extra hand, multiple persons in the video frame, and outdoor environment. All machine‐learning classifiers are combined using classifier combination to calculate the accuracy according to the majority‐voting rule. Based on the experimental results, it has been observed that deep learning provides better results compared to machine learning.

Recognition of Hand Gestures Based on EMG Signals with Deep and Double-Deep Q-Networks.

In recent years, hand gesture recognition (HGR) technologies that use electromyography (EMG) signals have been of considerable interest in developing human-machine interfaces. Most state-of-the-art HGR approaches are based mainly on supervised machine learning (ML). However, the use of reinforcement learning (RL) techniques to classify EMGs is still a new and open research topic. Methods based on RL have some advantages such as promising classification performance and online learning from the user's experience. In this work, we propose a user-specific HGR system based on an RL-based agent that learns to characterize EMG signals from five different hand gestures using Deep Q-network (DQN) and Double-Deep Q-Network (Double-DQN) algorithms. Both methods use a feed-forward artificial neural network (ANN) for the representation of the agent policy. We also performed additional tests by adding a long-short-term memory (LSTM) layer to the ANN to analyze and compare its performance. We performed experiments using training, validation, and test sets from our public dataset, EMG-EPN-612. The final accuracy results demonstrate that the best model was DQN without LSTM, obtaining classification and recognition accuracies of up to 90.37%±10.7% and 82.52%±10.9%, respectively. The results obtained in this work demonstrate that RL methods such as DQN and Double-DQN can obtain promising results for classification and recognition problems based on EMG signals.