Dynamic Hand Gesture Recognition System Research Articles

A Review on Sign Language Recognition and Learning System

This paper presents innovative approaches to various aspects of sign language recognition and learning, catering to the diverse needs within this field. It introduces a system utilizing 2D image sampling and concatenation, trained with convolutional neural networks, achieving accurate and robust sign recognition even with low-cost cameras. Another system, SignQuiz, offers a cost-effective web-based solution for learning finger-spelled signs in Indian Sign Language (ISL), outperforming traditional printed mediums. Additionally, a dynamic hand gesture recognition system employing deep learning architectures demonstrates superior performance over existing methods, addressing challenges such as hand segmentation and sequence modeling. Furthermore, an efficient deep convolutional neural network approach for hand gesture recognition, including transfer learning, achieves high recognition rates across various datasets. Finally, a robust model for static sign recognition in ISL, utilizing CNNs and evaluating on a diverse dataset, attains exceptional accuracy and outperforms previous works. These contributions collectively advance the field of sign language recognition and learning, offering solutions that are accurate, cost-effective, and efficient, thereby facilitating better communication and interaction for the hearing-impaired community. Key Words: CNN, Sign language recognition, Hand gesture recognition, Machine learning, Transfer Learning, SVM.

Realtime Recognition of Dynamic Hand Gestures in Practical Applications

Dynamic hand gesture acting as a semaphoric gesture is a practical and intuitive mid-air gesture interface. Nowadays benefiting from the development of deep convolutional networks, the gesture recognition has already achieved a high accuracy, however, when performing a dynamic hand gesture such as gestures of direction commands, some unintentional actions are easily misrecognized due to the similarity of the hand poses. This hinders the application of dynamic hand gestures and cannot be solved by just improving the accuracy of the applied algorithm on public datasets, thus it is necessary to study such problems from the perspective of human-computer interaction. In this article, two methods are proposed to avoid misrecognition by introducing activation delay and using asymmetric gesture design. First the temporal process of a dynamic hand gesture is decomposed and redefined, then a realtime dynamic hand gesture recognition system is built through a two-dimensional convolutional neural network. In order to investigate the influence of activation delay and asymmetric gesture design on system performance, a user study is conducted and experimental results show that the two proposed methods can effectively avoid misrecognition. The two methods proposed in this article can provide valuable guidance for researchers when designing realtime recognition system in practical applications.

Textural feature descriptors for a static and dynamic hand gesture recognition system

Textural feature descriptors for a static and dynamic hand gesture recognition system

Static and dynamic hand gesture recognition system with deep convolutional levy flight whale optimization

Static and dynamic hand gesture recognition system with deep convolutional levy flight whale optimization

Deep learning based spatio‐temporal hand gesture recognition system in complex environment

AbstractGesture recognition nowadays has grabbed the attention of researchers as they represent human behaviour in multiple practical ways. Amongst a variety of gestures available, hand gestures play an essential role in the field of human‐computer interaction when recognised efficiently in complex and dynamic environments. In this paper, we propose a dynamic hand gesture recognition system to recognise hand gestures appearing in different indoor and outdoor environments. Hand detection and tracking uses a two‐level system resulting in the formation of gesture trajectory in challenging conditions in which existing detection and tracking algorithms could not do so. A set of 45 features is provided as input to the various classification techniques. The redundancy problem has been reduced by selecting a set of optimum features using the analysis of variance method, which ranks the list of features. An incremental feature selection technique calculates recognition accuracy by selecting features according to rankings. This system provides an accuracy of 96.32% when used with machine learning and 97.5% when used with deep learning techniques. Recognition accuracy is calculated for various environments, including an extra hand, multiple persons in the video frame, and outdoor environment. All machine‐learning classifiers are combined using classifier combination to calculate the accuracy according to the majority‐voting rule. Based on the experimental results, it has been observed that deep learning provides better results compared to machine learning.

New combined DT-CWT and HOG descriptor for static and dynamic hand gesture recognition.

In recent years, researchers have been focusing on developing Human-Computer Interfaces that are fast, intuitive, and allow direct interaction with the computing environment. One of the most natural ways of communication is hand gestures. In this context, many systems were developed to recognize hand gestures using numerous vision-based techniques, these systems are highly affected by acquisition constraints, such as resolution, noise, lighting condition, hand shape, and pose. To enhance the performance under such constraints, we propose a static and dynamic hand gesture recognition system, which utilizes the Dual-Tree Complex Wavelet Transform to produce an approximation image characterized by less noise and redundancy. Subsequently, the Histogram of Oriented Gradients is applied to the resulting image to extract relevant information and produce a compact features vector. For classification, we compare the performance of three Artificial Neural Networks, namely, MLP, PNN, and RBNN. Random Decision Forest and SVM classifiers are also used to ameliorate the efficiency of our system. Experimental evaluation is performed on four datasets composed of alphabet signs and dynamic gestures. The obtained results demonstrate the efficiency of the combined features, for which the achieved recognition rates were comparable to the state-of-the-art.

User-Definable Dynamic Hand Gesture Recognition Based on Doppler Radar and Few-Shot Learning

In recent years, the radar-based dynamic hand gesture recognition (DHGR) system has been widely used in the non-contact interaction with smart electronic devices because of its advantages of safety, privacy security and robustness to different illumination environments. In order to achieve high-precision gesture recognition, Machine Learning algorithms, especially the deep learning algorithms are generally chosen by researchers. However, most deep learning models are trained based on large gesture dataset and only accept certain predefined specified gestures as input, i.e., user-defined gestures are not allowed. In this work, a neural network model trained with meta-learning method is proposed to deal with the few-shot classification task and realize the user-definable DHGR. Instead of learning a mapping between the unknown input gesture and the fixed predefined classes, the proposed model is trained to learn to compare the input gesture with arbitrary gestures entered by users, and then decide which class of gesture is the most probable one. Without retraining, the model can allow the user to enter the self-defined gestures for one or a few times and then recognize these gestures next time. To the best of our knowledge, this is the first work to apply the meta few-shot learning to the DHGR problem in the true sense. In this model, an embedding module is used in the network to extract the features of the input micro-Doppler spectrograms, and a comparison module is used to execute the feature-level comparison. Finally, a weighting module is proposed to weigh the comparison results and make the prediction. The evaluation result shows that with only one support sample for each class (i.e., for the one-shot tasks), the proposed model achieves an accuracy of 91% for 5 gesture classes and 90% for 10 gesture classes. When using 3 support samples (i.e., for the 3-shot tasks), the accuracies for 5 and 10 gestures are further improved to 96% and 92%, respectively.

Dynamic Gesture Recognition System with Gesture Spotting Based on Self-Organizing Maps

In this paper, a real-time dynamic hand gesture recognition system with gesture spotting function is proposed. In the proposed system, input video frames are converted to feature vectors, and they are used to form a posture sequence vector that represents the input gesture. Then, gesture identification and gesture spotting are carried out in the self-organizing map (SOM)-Hebb classifier. The gesture spotting function detects the end of the gesture by using the vector distance between the posture sequence vector and the winner neuron’s weight vector. The proposed gesture recognition method was tested by simulation and real-time gesture recognition experiment. Results revealed that the system could recognize nine types of gesture with an accuracy of 96.6%, and it successfully outputted the recognition result at the end of gesture using the spotting result.

Algorithm of local features fusion and modified covariance-matrix technique for hand motion position estimation and hand gesture trajectory tracking approach

Nowadays, visual recognition based dynamic hand gesture tracking has gained very considerable attention. Hand gestures can play an important role as a non-touchable communication tool between machines and humans based on using the affordable built-in webcam. The need for replacing touch-based computing devices interaction has been increasing in many fields like healthcare, security, and generally as interface-based devices controlling. Especially with COVID19 outbreak spreading around the world, where people take a risk and avoid dealing with electronic consumer machines that required hand touching. However, in any dynamic hand gesture recognition system, dynamic hand gesture tracking is a very hard task, where position estimation over video frames for freely moving hand in the air is quite challenging. This shortcoming is due to hand great scale changes, posture variations, and translation problems. Hence, to tackle these difficulties and extract gesture features for gesture recognition phase accurately, this paper proposed an algorithm of dynamic hand trajectory tracking for gesture recognition. The presented algorithm proposes local features fusion based on Gabor-Canny- Hog features embedded an updated compact covariance matrix technique as sophisticated feature-based tracking, utilizing video sequences of IBGHT dataset. As a result, the proposed approach has shown adorable optimization achieving an accuracy rate of 96.97%, overcoming the problems of hand appearance variation in the complicated environment.

TWO-STREAM CONVOLUTIONAL NETWORK FOR DYNAMIC HAND GESTURE RECOGNITION USING CONVOLUTIONAL LONG SHORT-TERM MEMORY NETWORKS

Human action and gesture recognition provides important and worth information for interaction between human and device ambient that monitors living, healthcare facilities or entertainment activities in smart homes. Recent years, there were many machine learning model application studies to recognize human action and gesture. In this paper, we propose a dynamic hand gesture recognition system in video based on two stream-convolution network (ConvNet) architecture. Specifically, we research the state-of-the-art approaches using to recognize dynamic hand gesture in video and propose an improvement method to enhance performance of model which is suitable for uses such as indoor environment in this paper. Our contribution is improvement of two stream ConvNet to achieve better performance. The results show that the proposal model improves execution speed and memory resource usage comparing to existing models.

A Dynamic Hand Gesture Recognition System Incorporating Orientation-based Linear Extrapolation Predictor and Velocity-assisted Longest Common Subsequence Algorithm

A Dynamic Hand Gesture Recognition System Incorporating Orientation-based Linear Extrapolation Predictor and Velocity-assisted Longest Common Subsequence Algorithm

Dynamic hand gesture recognition using vision-based approach for human–computer interaction

In this work, a vision-based approach is used to build a dynamic hand gesture recognition system. Various challenges such as complicated background, change in illumination and occlusion make the detection and tracking of hand difficult in any vision-based approaches. To overcome such challenges, a hand detection technique is developed by combining three-frame differencing and skin filtering. The three-frame differencing is performed for both colored and grayscale frames. The hand is then tracked using modified Kanade---Lucas---Tomasi feature tracker where the features were selected using the compact criteria. Velocity and orientation information were added to remove the redundant feature points. Finally, color cue information is used to locate the final hand region in the tracked region. During the feature extraction, 44 features were selected from the existing literatures. Using all the features could lead to overfitting, information redundancy and dimension disaster. Thus, a system with optimal features was selected using analysis of variance combined with incremental feature selection. These selected features were then fed as an input to the ANN, SVM and kNN model. These individual classifiers were combined to produce classifier fusion model. Fivefold cross-validation has been used to evaluate the performance of the proposed model. Based on the experimental results, it may be concluded that classifier fusion provides satisfactory results (92.23 %) compared to other individual classifiers. One-way analysis of variance test, Friedman's test and Kruskal---Wallis test have also been conducted to validate the statistical significance of the results.

Effect of variation in gesticulation pattern in dynamic hand gesture recognition system

This paper presents a hand gesture recognition system which addresses the effect of variations in gesture pattern during gesticulation. Different gestures can be gesticulated in various patterns which increase the difficulties in recognizing the gestures. We have proposed two new features such as left sector trajectory features and right sector trajectory features which are able to recognize gestures even with the presence of variations in the gesticulation pattern. The effectiveness of the proposed system is illustrated by different experiments with our own gesture database. A comparative study has been made with the proposed features and three state-of-art features such as orientation; combination of location, orientation, velocity; and combination of ellipse and position features. The performance of the system was evaluated using this proposed set of features for different individual classifiers such as ANN, SVM, k-NN, Naïve Bayes and ELM. Finally, the decisions of the individual classifiers were combined using major voting rule to result in classifier fusion model. Based on the experimental results it may be concluded that classifier fusion provides satisfactory results compared to other individual classifiers. An accuracy of 91.07% was achieved using the classifier fusion technique as compared to baseline CRF (79.45%) and HCRF (83.07%) models.

Hand Gesture Recognition System for Creating & Controlling Media Player using Mat Lab Tool

In this paper we have discussed a dynamic hand gesture recognition system for creating and controlling the media player. The system is made possible by using the concepts of threshold and color detection. The media player is created by ActiveX Control. Being independent of restricted background and noise, this system doesn't use a database for performing an operation. This obviously reduces the space and time complexity, resulting in an efficient system which is capable of accepting dynamic gestures to control the media player.

Online Dynamic Gesture Recognition for Human Robot Interaction

This paper presents an online dynamic hand gesture recognition system with an RGB-D camera, which can automatically recognize hand gestures against complicated background. For background subtraction, we use a model-based method to perform human detection and segmentation in the depth map. Since a robust hand tracking approach is crucial for the performance of hand gesture recognition, our system uses both color information and depth information in the process of hand tracking. To extract spatio-temporal hand gesture sequences in the trajectory, a reliable gesture spotting scheme with detection on change of static postures is proposed. Then discrete HMMs with Left-Right Banded (LRB) topology are utilized to model and classify gestures based on multi-feature representation and quantization of the hand gesture sequences. Experimental evaluations on two self-built databases of dynamic hand gestures show the effectiveness of the proposed system. Furthermore, we develop a human-robot interactive system, and the performance of this system is demonstrated through interactive experiments in the dynamic environment.

A Dynamic Gesture and Posture Recognition System

This paper presents a real time dynamic hand gesture and posture recognition system based on a neural network and a Hidden Markov Model. For skin color segmentation an adaptive online trained skin color model is used, while the hand posture recognition is accomplished through a likelihood-based classification technique of geometric features. A novel trajectory smoothing technique based on Self Organized Neural Network is introduced to improve HMM classification performance of dynamic gestures. The aim of the proposed system is the creation of a visual dictionary combining hand postures and dynamic gestures. The system has been successfully tested with many people under varying light conditions and different web cameras.

Dynamic Hand Gesture Recognition using PCA, Pruning and ANN

Gesture recognition has the potential to be a natural and powerful tool supporting efficient and intuitive interaction between the human and the computer. The primary goal of hand gesture recognition research is to create a system which can identify specific hand gestures and use them to convey information for device control. This paper proposes a visionbased dynamic hand gesture recognition system using PCA, pruning and ANN. The system will find hand location in each frame. The hand segmentation is done using color marker for PCA. Skin color detection method is used for pruning and ANN. After connecting all these hand locations, the template is generated at the end of Segmentation. Template is used to extract the feature for training and testing purpose. The implementation uses the eigenspace which is determined by processing the eigenvalues and eigenvectors of the image set. The experimental results show ANN method gives more recognition rate than PCA and pruning.

Hand gesture tracking and recognition system using Lucas–Kanade algorithms for control of consumer electronics

Dynamic hand gesture tracking and recognition system can simplify the way humans interact with computers and many other non-critical consumer electronic equipments. This system is based on the well-known “Wave Controller” technology developed at the University of Wollongong [1–3] and certainly a step forward in video gaming and consumer electronics control interfaces. Currently, computer interfacing mainly involves keyboard, mouse, joystick or gaming wheels and occasionally voice recognition for user input. These modes of interaction have constrained the artistic ability of many users, as they are required to respond to the computer through pressing buttons or moving other apparatus. Voice recognition is seen as unreliable and impractical in areas where more than one user is present. All these drawbacks can be tackled by using a reliable hand gesture tracking and recognition system based on both Lucas–Kanade and Moment Invariants approaches. This will facilitate interaction between users and computers and other consumer electronic equipments in real time. This will further enhance the user experience as users are no longer have any physical connection to the equipment being controlled. In this research, we have compared our proposed moment invariant based algorithm with template based and Fourier descriptor based methods to highlight the advantages and limitations of the proposed system.

A fuzzy rule-based approach to spatio-temporal hand gesture recognition

Gesture based applications widely range from replacing the traditional mouse as a position device to virtual reality and communication with the deaf. The article presents a fuzzy rule based approach to spatio-temporal hand gesture recognition. This approach employs a powerful method based on hyperrectangutar composite neural networks (HRCNNs) for selecting templates. Templates for each hand shape are represented in the form of crisp IF-THEN rules that are extracted from the values of synaptic weights of the corresponding trained HRCNNs. Each crisp IF-THEN rule is then fuzzified by employing a special membership function in order to represent the degree to which a pattern is similar to the corresponding antecedent part. When an unknown gesture is to be classified, each sample of the unknown gesture is tested by each fuzzy rule. The accumulated similarity associated with all samples of the input is computed for each hand gesture in the vocabulary, and the unknown gesture is classified as the gesture yielding the highest accumulative similarity. Based on the method we can implement a small-sized dynamic hand gesture recognition system. Two databases which consisted of 90 spatio-temporal hand gestures are utilized for verifying its performance. An encouraging experimental result confirms the effectiveness of the proposed method.