Approach For Hand Gesture Recognition Research Articles

Real-Time Hand Gesture Monitoring Model Based on MediaPipe's Registerable System.

Hand gesture recognition plays a significant role in human-to-human and human-to-machine interactions. Currently, most hand gesture detection methods rely on fixed hand gesture recognition. However, with the diversity and variability of hand gestures in daily life, this paper proposes a registerable hand gesture recognition approach based on Triple Loss. By learning the differences between different hand gestures, it can cluster them and identify newly added gestures. This paper constructs a registerable gesture dataset (RGDS) for training registerable hand gesture recognition models. Additionally, it proposes a normalization method for transforming hand gesture data and a FingerComb block for combining and extracting hand gesture data to enhance features and accelerate model convergence. It also improves ResNet and introduces FingerNet for registerable single-hand gesture recognition. The proposed model performs well on the RGDS dataset. The system is registerable, allowing users to flexibly register their own hand gestures for personalized gesture recognition.

An extended variational autoencoder for cross-subject electromyograph gesture recognition

Surface electromyographic hand gesture recognition has gained significant attention in recent years, especially within the field of human–computer interfaces. However, cross-subject tasks remain challenging due to inherent individual differences. To address this, a novel approach for hand gesture recognition is proposed that leverages a subject-generalized variational autoencoder. This approach involves an extended variational autoencoder designed to disentangle input data into three distinct feature-specific representations. The primary classifier within the variational autoencoder focuses on gesture recognition, while two auxiliary classifiers work together to extract subject-specific and gesture-specific features. The gesture-specific features capture generalized characteristics applicable across all subjects, enabling direct application to new subjects. To enhance accuracy and stability, a competitive voting strategy is implemented. The effectiveness of the proposed method was evaluated using a dataset comprising six representative gestures performed by eight subjects. Comparative analysis with baseline models shows that our approach outperforms others, demonstrating superior generalization with an average accuracy of 90.52% in cross-subject validation.

A Review on Sign Language Recognition and Learning System

This paper presents innovative approaches to various aspects of sign language recognition and learning, catering to the diverse needs within this field. It introduces a system utilizing 2D image sampling and concatenation, trained with convolutional neural networks, achieving accurate and robust sign recognition even with low-cost cameras. Another system, SignQuiz, offers a cost-effective web-based solution for learning finger-spelled signs in Indian Sign Language (ISL), outperforming traditional printed mediums. Additionally, a dynamic hand gesture recognition system employing deep learning architectures demonstrates superior performance over existing methods, addressing challenges such as hand segmentation and sequence modeling. Furthermore, an efficient deep convolutional neural network approach for hand gesture recognition, including transfer learning, achieves high recognition rates across various datasets. Finally, a robust model for static sign recognition in ISL, utilizing CNNs and evaluating on a diverse dataset, attains exceptional accuracy and outperforms previous works. These contributions collectively advance the field of sign language recognition and learning, offering solutions that are accurate, cost-effective, and efficient, thereby facilitating better communication and interaction for the hearing-impaired community. Key Words: CNN, Sign language recognition, Hand gesture recognition, Machine learning, Transfer Learning, SVM.

A novel sEMG-based dynamic hand gesture recognition approach via residual attention network

A novel sEMG-based dynamic hand gesture recognition approach via residual attention network

Human Gesture Recognition Based on Millimeter-Wave Radar Using Improved C3D Convolutional Neural Network

<p>In this paper, we propose a time sequential IC3D convolutional neural network approach for hand gesture recognition based on frequency modulated continuous wave (FMCW) radar. Firstly, the FMCW radar is used to collect the echoes of human hand gestures. A two-dimensional fast Fourier transform calculates the range and velocity information of hand gestures in each frame signal to construct the Range-Doppler heat map dataset of hand gestures. Then, we design an IC3D network for feature extraction and classification of the dynamic gesture heat map. Finally, the experiment results show that the gesture recognition system designed in this paper effectively solves the problems of the difficulty of human gesture feature extraction and low utilization of time series information, and the average recognition accuracy rate can reach more than 99.8%.</p> <p> </p>

Deep Learning Approach for Hand Gesture Recognition: Applications in Deaf Communication and Healthcare

Deep Learning Approach for Hand Gesture Recognition: Applications in Deaf Communication and Healthcare

A two-branch hand gesture recognition approach combining atrous convolution and attention mechanism

A two-branch hand gesture recognition approach combining atrous convolution and attention mechanism

Hand Gestures Identification for Fine-Grained Human Activity Recognition in Smart Homes

Most developed countries are facing important demographic issues related to ageing populations. Maintaining elders at home while ensuring their safety and well-being often constitutes the main goal of these countries. An interesting solution to this challenge is to develop a smart home, able to monitor the routines of the resident, to recognize the on-going activities, and to provide support when required. In the literature, most works focus on monitoring high-level behaviors such as eating, sleeping, etc. However, to provide live guidance, the system needs to have a far more detailed recognition process able to identify the specific steps of the on-going task and the erroneous executions. In this paper, we propose an algorithmic approach for hand gesture recognition designed to be used as the core component of a fine-grained activity recognition model. It is based on the processing of inertial data collected from a wristband equipped with triaxial accelerometer and gyroscope, and machine learning techniques. A simple set of gestures for cooking activities as been defined, enabling characterizing high-level cooking tasks. To do that, we constructed a labelled dataset of atomic gestures performed by participants that we made available to the scientific community. We obtained an average accuracy of 0.83 in recognizing the gestures with the leave-one-subject-out strategy.

Hand Gesture Recognition Based on Trajectories Features and Computation-Efficient Reused LSTM Network

Touchless hand gesture recognition using radar sensor is considered to be an attractive and effective technique for human-machine interaction. In this paper, we propose a hand gesture recognition approach based-on range-Doppler-angle trajectories and a reused long short-term memory (RLSTM) network using a 77GHz frequency modulated continuous wave (FMCW) multiple-input-multiple-output (MIMO) radar. To overcome amounts of gesture interferences, a gesture desktop is designed and a potential hand gesture detection approach is followed to determine whether it is a potential hand gesture. If the gesture happens in the assumed gesture desktop, the range-Doppler-angle trajectories are extracted by using the discretize Fourier transform (DFT), multiple signal classification (MUSIC) algorithm and the Kalman filtering. These signatures cannot only significantly reduce the dimension for the following neural network but also exploit the three dimensional information provided by the MIMO radar. Finally, a LSTM network with the reused forward propagation approach is proposed to exploit the spatial, temporal and Doppler information and simplify the network architecture. The experimental results show that the proposed approach can achieve an average accuracy of 99.4% and recognize a new person's hand gestures with an average accuracy of 98.0% for 9 hand gestures.

Hand-Gesture Recognition Based on EMG and Event-Based Camera Sensor Fusion: A Benchmark in Neuromorphic Computing.

Hand gestures are a form of non-verbal communication used by individuals in conjunction with speech to communicate. Nowadays, with the increasing use of technology, hand-gesture recognition is considered to be an important aspect of Human-Machine Interaction (HMI), allowing the machine to capture and interpret the user's intent and to respond accordingly. The ability to discriminate between human gestures can help in several applications, such as assisted living, healthcare, neuro-rehabilitation, and sports. Recently, multi-sensor data fusion mechanisms have been investigated to improve discrimination accuracy. In this paper, we present a sensor fusion framework that integrates complementary systems: the electromyography (EMG) signal from muscles and visual information. This multi-sensor approach, while improving accuracy and robustness, introduces the disadvantage of high computational cost, which grows exponentially with the number of sensors and the number of measurements. Furthermore, this huge amount of data to process can affect the classification latency which can be crucial in real-case scenarios, such as prosthetic control. Neuromorphic technologies can be deployed to overcome these limitations since they allow real-time processing in parallel at low power consumption. In this paper, we present a fully neuromorphic sensor fusion approach for hand-gesture recognition comprised of an event-based vision sensor and three different neuromorphic processors. In particular, we used the event-based camera, called DVS, and two neuromorphic platforms, Loihi and ODIN + MorphIC. The EMG signals were recorded using traditional electrodes and then converted into spikes to be fed into the chips. We collected a dataset of five gestures from sign language where visual and electromyography signals are synchronized. We compared a fully neuromorphic approach to a baseline implemented using traditional machine learning approaches on a portable GPU system. According to the chip's constraints, we designed specific spiking neural networks (SNNs) for sensor fusion that showed classification accuracy comparable to the software baseline. These neuromorphic alternatives have increased inference time, between 20 and 40%, with respect to the GPU system but have a significantly smaller energy-delay product (EDP) which makes them between 30× and 600× more efficient. The proposed work represents a new benchmark that moves neuromorphic computing toward a real-world scenario.

A novel hybrid bidirectional unidirectional LSTM network for dynamic hand gesture recognition with Leap Motion

Due to the recent development of machine learning and sensor innovations, hand gesture recognition systems become promisingfor the digital entertainment field. In this paper, we propose a dynamic hand gesture recognition approach using touchless hand motions over a Leap Motion device. First, we analyze the sequential time series data gathered from Leap Motion using Long Short-Term Memory (LSTM) recurrent neural networks for recognition purposes. We exploit basic unidirectional LSTM and bidirectional LSTM separately. Then, we propound novel architecture by combining the aforementioned models with additional components to give a final prediction network, named Hybrid Bidirectional Unidirectional LSTM (HBU-LSTM). The suggested network improves the model performance significantly by considering the spatial and temporal dependencies between the Leap Motion data and the network layers during the forward and backward pass. The recognition models are examined on two available benchmark datasets, named the LeapGestureDB dataset and the RIT dataset. Experiments demonstrate the potential of the proposed HBU-LSTM network for dynamic hand gesture recognition, with an average recognition rate reaching approximately 90%. Our suggested approach reaches superior performance, in terms of accuracy and computational complexity, over some existing methods for hand gesture recognition.

Design and evaluation of a hand gesture recognition approach for real-time interactions

Hand gestures are a natural and intuitive form for human-environment interaction and can be used as an input alternative in human-computer interaction (HCI) to enhance usability and naturalness. Many existing approaches have employed vision -based systems to detect and recognize hand gestures. However, vision-based systems usually require users to move their hands within restricted space, where the optical device can capture the motion of hands. Also, vision-based systems may suffer from self-occlusion issues due to sophisticated finger movements. In this work, we use a sensor-based motion tracking system to capture 3D hand and finger motions. To detect and recognize hand gestures, we propose a novel angular-velocity method, which is directly applied to real-time 3D motion data streamed by the sensor-based system. Our approach is capable of recognizing both static and dynamic gestures in real-time. We assess the recognition accuracy and execution performance with two interactive applications that require gesture input to interact with the virtual environment. Our experimental results show high recognition accuracy, high execution performance, and high-levels of usability.

Hand Gesture Recognition for Sign Language Using 3DCNN

Recently, automatic hand gesture recognition has gained increasing importance for two principal reasons: the growth of the deaf and hearing-impaired population, and the development of vision-based applications and touchless control on ubiquitous devices. As hand gesture recognition is at the core of sign language analysis a robust hand gesture recognition system should consider both spatial and temporal features. Unfortunately, finding discriminative spatiotemporal descriptors for a hand gesture sequence is not a trivial task. In this study, we proposed an efficient deep convolutional neural networks approach for hand gesture recognition. The proposed approach employed transfer learning to beat the scarcity of a large labeled hand gesture dataset. We evaluated it using three gesture datasets from color videos: 40, 23, and 10 classes were used from these datasets. The approach obtained recognition rates of 98.12%, 100%, and 76.67% on the three datasets, respectively for the signer-dependent mode. For the signer-independent mode, it obtained recognition rates of 84.38%, 34.9%, and 70% on the three datasets, respectively.

A Random Forest-based Approach for Hand Gesture Recognition with Wireless Wearable Motion Capture Sensors

Abstract Gesture Recognition has a prominent importance in smart environment and home automation. Thanks to the availability of Machine Learning approaches it is possible for users to define gestures that can be associated with commands for the smart environment. In this paper we propose a Random Forest-based approach for Gesture Recognition of hand movements starting from wireless wearable motion capture data. In the presented approach, we evaluate different feature extraction procedures to handle gestures and data with different duration. To enhance reproducibility of our results and to foster research in the Gesture Recognition area, we share the dataset that we have collected and exploited for the present work.

TS-I3D Based Hand Gesture Recognition Method With Radar Sensor

Aiming at the problems of the noise impact on the parametric image of hand gestures, the difficulty of gesture feature extraction, and the inefficient utilization of continuous gesture time sequential information, we propose a time sequential inflated 3 dimensions (TS-I3D) convolutional neural network approach for hand gesture recognition based on frequency modulated continuous wave (FMCW) radar sensor. Specifically, the FMCW radar is used to acquire the hand gesture data, and the range and speed of the gesture in each frame signal are calculated by 2 dimensions fast Fourier transform. Then, the range-Doppler map (RDM) is generated based on the relationship between motion parameters and frequency. The interference in RDM caused by people and the external environment is filtered out and the peak of hand gesture in RDM is further enhanced by wavelet transform. Finally, TS-I3D network is designed to extract the range and speed change information of the continuous gestures. The experimental results show that the average recognition accuracy rate of the hand gestures of the proposed method is 96.17%.

Deep Fisher discriminant learning for mobile hand gesture recognition

Gesture recognition becomes a popular analytics tool for extracting the characteristics of user movement and enables numerous practical applications in the biometrics field. Despite recent advances in this technique, complex user interaction and the limited amount of data pose serious challenges to existing methods. In this paper, we present a novel approach for hand gesture recognition based on user interaction on mobile devices. We have developed two deep models by integrating Bidirectional Long-Short Term Memory (BiLSTM) network and Bidirectional Gated Recurrent Unit (BiGRU) with Fisher criterion, termed as F-BiLSTM and F-BiGRU respectively. These two Fisher discriminative models can classify user’s gesture effectively by analyzing the corresponding acceleration and angular velocity data of hand motion. In addition, we build a large Mobile Gesture Database (MGD) containing 5547 sequences of 12 gestures. With extensive experiments, we demonstrate the superior performance of the proposed method compared to the state-of-the-art BiLSTM and BiGRU on MGD database and two other benchmark databases (i.e., BUAA mobile gesture and SmartWatch gesture). The source code and MGD database will be made publicly available at https://github.com/bczhangbczhang/Fisher-Discriminant-LSTM.

Hand Motion and Posture Recognition in a Network of Calibrated Cameras

This paper presents a vision-based approach for hand gesture recognition which combines both trajectory and hand posture recognition. The hand area is segmented by fixed-range CbCr from cluttered and moving backgrounds and tracked by Kalman Filter. With the tracking results of two calibrated cameras, the 3D hand motion trajectory can be reconstructed. It is then modeled by dynamic movement primitives and a support vector machine is trained for trajectory recognition. Scale-invariant feature transform is employed to extract features on segmented hand postures, and a novel strategy for hand posture recognition is proposed. A gesture vector is introduced to recognize hand gesture as an entirety which combines the recognition results of motion trajectory and hand postures where a support vector machine is trained for gesture recognition based on gesture vectors.

A New Hand Gesture Recognition Approach for Robotic Assistants Based on Mobile Devices

Nowadays, the mobile devices (smartphones and tablets) are developed with important improvements in processing and memory capacity as well as camera resolution, massive storage or wireless connectivity. On those grounds, in developing countries these devices constitute an important alternative to develop assistive technologies to provide support in several areas as education, health care, and the elderly. Given that, in this paper we propose a new approach to handle the gesture recognition in robotic assistants that use a mobile device as main processor. Our approach uses a robust descriptor based on polygonal approximation, convex hull techniques and HU moments, and has achieved 93% precision in real scenarios with different light conditions.

HAGR-D: A Novel Approach for Gesture Recognition with Depth Maps.

The hand is an important part of the body used to express information through gestures, and its movements can be used in dynamic gesture recognition systems based on computer vision with practical applications, such as medical, games and sign language. Although depth sensors have led to great progress in gesture recognition, hand gesture recognition still is an open problem because of its complexity, which is due to the large number of small articulations in a hand. This paper proposes a novel approach for hand gesture recognition with depth maps generated by the Microsoft Kinect Sensor (Microsoft, Redmond, WA, USA) using a variation of the CIPBR (convex invariant position based on RANSAC) algorithm and a hybrid classifier composed of dynamic time warping (DTW) and Hidden Markov models (HMM), called the hybrid approach for gesture recognition with depth maps (HAGR-D). The experiments show that the proposed model overcomes other algorithms presented in the literature in hand gesture recognition tasks, achieving a classification rate of 97.49% in the MSRGesture3D dataset and 98.43% in the RPPDI dynamic gesture dataset.

Performance Comparison of Transforms using Row Mean and Column Mean for Hand Gesture Recognition

computer interaction is a major issue in research industry. In order to offer a way to enable untrained users to interact with computer more easily and efficiently gesture based interface has been paid more attention. This paper presents a new approach for hand gesture recognition. An approach consists of three modules: a) Preprocessing of the image b) Feature extraction c) Pattern matching for gesture recognition. Feature extraction is based on feature vector of transformed image using Discrete Cosine Transform, Walsh Transform, Haar transform and Kekre's transform. This transforms are applied on column mean and row mean of the images and various percentage of feature vectors are generated such as 100%, 50%, 25%, 12.5% and 6.25%. Results found to be better than existing system.