Hand Gesture Recognition Research Articles

Wearable high-density EMG sleeve for complex hand gesture classification and continuous joint angle estimation

High-density electromyography (HD-EMG) can provide a natural interface to enhance human–computer interaction (HCI). This study aims to demonstrate the capability of a novel HD-EMG forearm sleeve equipped with up to 150 electrodes to capture high-resolution muscle activity, decode complex hand gestures, and estimate continuous hand position via joint angle predictions. Ten able-bodied participants performed 37 hand movements and grasps while EMG was recorded using the HD-EMG sleeve. Simultaneously, an 18-sensor motion capture glove calculated 23 joint angles from the hand and fingers across all movements for training regression models. For classifying across the 37 gestures, our decoding algorithm was able to differentiate between sequential movements with 97.3±0.3%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$97.3 \\pm 0.3\\%$$\\end{document} accuracy calculated on a 100 ms bin-by-bin basis. In a separate mixed dataset consisting of 19 movements randomly interspersed, decoding performance achieved an average bin-wise accuracy of 92.8±0.8%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$92.8 \\pm 0.8\\%$$\\end{document}. When evaluating decoders for use in real-time scenarios, we found that decoders can reliably decode both movements and movement transitions, achieving an average accuracy of 93.3±0.9%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$93.3 \\pm 0.9\\%$$\\end{document} on the sequential set and 88.5±0.9%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$88.5 \\pm 0.9\\%$$\\end{document} on the mixed set. Furthermore, we estimated continuous joint angles from the EMG sleeve data, achieving a R2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$R^2$$\\end{document} of 0.884±0.003\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$0.884 \\pm 0.003$$\\end{document} in the sequential set and 0.750±0.008\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$0.750 \\pm 0.008$$\\end{document} in the mixed set. Median absolute error (MAE) was kept below 10° across all joints, with a grand average MAE of 1.8±0.04∘\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$1.8 \\pm 0.04^\\circ$$\\end{document} and 3.4±0.07∘\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$3.4 \\pm 0.07^\\circ$$\\end{document} for the sequential and mixed datasets, respectively. We also assessed two algorithm modifications to address specific challenges for EMG-driven HCI applications. To minimize decoder latency, we used a method that accounts for reaction time by dynamically shifting cue labels in time. To reduce training requirements, we show that pretraining models with historical data provided an increase in decoding performance compared with models that were not pretrained when reducing the in-session training data to only one attempt of each movement. The HD-EMG sleeve, combined with sophisticated machine learning algorithms, can be a powerful tool for hand gesture recognition and joint angle estimation. This technology holds significant promise for applications in HCI, such as prosthetics, assistive technology, rehabilitation, and human–robot collaboration.

Dynamic Adaptation in Deep Learning for Enhanced Hand Gesture Recognition

Dynamic Adaptation in Deep Learning for Enhanced Hand Gesture Recognition

An overview of hand gesture recognition based on computer vision

Hand gesture recognition emerges as one of the foremost sectors which has gone through several developments within pattern recognition. Numerous studies and research endeavors have explored methodologies grounded in computer vision within this domain. Despite extensive research endeavors, there is still a need for a more thorough evaluation of the efficiency of various methods in different environments along with the challenges encountered during the application of these methods. The focal point of this paper is the comparison of different research in the domain of vision-based hand gesture recognition. The objective is to find out the most prominent methods by reviewing efficiency. Concurrently, the paper delves into presenting potential solutions for challenges faced in different research. A comparative analysis particularly centered around traditional methods and convolutional neural networks like random forest, long short-term memory (LSTM), heatmap, and you only look once (YOLO). considering their efficacy. Where convolutional neural network-based algorithms performed best for recognizing the gestures and gave effective solutions for the challenges faced by the researchers. In essence, the findings of this review paper aim to contribute to future implementations and the discovery of more efficient approaches in the gesture recognition sector.

Innovative healthcare solutions: robust hand gesture recognition of daily life routines using 1D CNN.

Hand gestures are an effective communication tool that may convey a wealth of information in a variety of sectors, including medical and education. E-learning has grown significantly in the last several years and is now an essential resource for many businesses. Still, there has not been much research conducted on the use of hand gestures in e-learning. Similar to this, gestures are frequently used by medical professionals to help with diagnosis and treatment. We aim to improve the way instructors, students, and medical professionals receive information by introducing a dynamic method for hand gesture monitoring and recognition. Six modules make up our approach: video-to-frame conversion, preprocessing for quality enhancement, hand skeleton mapping with single shot multibox detector (SSMD) tracking, hand detection using background modeling and convolutional neural network (CNN) bounding box technique, feature extraction using point-based and full-hand coverage techniques, and optimization using a population-based incremental learning algorithm. Next, a 1D CNN classifier is used to identify hand motions. After a lot of trial and error, we were able to obtain a hand tracking accuracy of 83.71% and 85.71% over the Indian Sign Language and WLASL datasets, respectively. Our findings show how well our method works to recognize hand motions. Teachers, students, and medical professionals can all efficiently transmit and comprehend information by utilizing our suggested system. The obtained accuracy rates highlight how our method might improve communication and make information exchange easier in various domains.

DEEP LEARNING-BASED HAND GESTURE RECOGNITION FOR SPEECH SYNTHESIS IN TELUGU

In a world increasingly reliant on technology, individuals had born with hearing impairments face significant communication challenges, leading to feelings of isolation and dependency. This paper addresses the pressing need to empower the deaf and mute community by proposing an innovative solution – Deep Learning-Based Hand Gesture Recognition for Speech Synthesis in Telugu. Deaf and mute individuals encounter barriers in expressing themselves verbally, hindering their integration into mainstream society. Conventional methods often fall short in providing effective communication channels, exacerbating the challenges faced by this community. The critical need for a comprehensive solution that facilitates seamless communication in their native language, Telugu is evident. Current sign language solutions lack precision, failing to capture nuanced gestures, limiting accuracy. Importantly, they overlook converting gestures into spoken Telugu, leaving a gap for the deaf and mute community. These systems also face real-time processing challenges, hindering natural communication in users native language. Our innovative approach utilizes advanced technology, specifically Deep Learning and a Convolutional Neural Network (CNN). This system significantly improves understanding of hand movements, achieving a remarkable 90% accuracy. When someone uses hand gestures, our system converts them into spoken Telugu, enhancing communication for deaf and mute individuals. This substantial improvement empowers native Telugu speakers by 80%, allowing them to express themselves more naturally and actively participate in daily life.

Large measurement range curvature sensor based on all-fiber Mach-Zehnder interferometer using triple-ring-core few-mode fiber.

This study experimentally demonstrates a large measurement range curvature sensor based on a Mach-Zehnder interferometer (MZI) in a triple-ring-core fiber (TRCF). The sensor is fabricated by fusion splicing a segment of TRCF between two pieces of single-mode fiber (SMF), forming the SMF-TRCF-SMF sandwich structure. Since the TRCF can support the propagation of a few guided modes, the fundamental mode interferes with high-order modes in the sensing part to produce a periodic interference spectrum. Bending causes the effective refractive index of the mode to change, which shifts the interference spectrum. The bending measurement is achieved by monitoring the wavelength variation of interference dips. The proposed sensors can realize curvature measurements up to 100 m-1, with a bending sensitivity of -225.9 pm/m-1. It is believed that the proposed MZI sensor may have potential applications in robot hand gesture recognition and control.

Quantized CNN-based efficient hardware architecture for real-time hand gesture recognition

Nowadays, Convolutional Neural Networks (CNN) have been widely adopted for vision-based hand gesture recognition. Several existing CNN architectures designed for gesture classification perform well with high accuracy but require a high memory footprint and processing when deployed on low-power embedded devices. To address this issue, we present a quantized CNN-based efficient framework for meeting real-life hand gesture recognition challenges. The proposed quantized CNN is designed and implemented using the FINN-based pipelined streaming architecture on an FPGA. Moreover, hardware-based optimizations are used to minimize the resources needed and to achieve fast memory access. Experimental results demonstrate that the developed recognition system achieves an average accuracy of 92% on the numeral database of Indian Sign Language (ISL). Additionally, our optimized design attains an inference latency of 0.85ms for real-time single gesture prediction on the PYNQ Zynq Ultrascale (ZU) FPGA, consuming only 3.63 W of power. The proposed design achieves a better trade-off between hardware resource utilization and speed performance, over previous designs.

Personalized Drone Interaction : Adaptive Hand Gesture Control with Facial Authentication

This paper presents a novel system for personalized drone interaction, integrating adaptive hand gesture control with facial authentication. Utilizing the DJI Tello drone equipped with a 5 MP camera, the system employs advanced computer vision and machine learning techniques to ensure secure and intuitive control. Facial recognition using the Histogram of Oriented Gradients (HOG) method and FaceNet model verifies user identity, while MediaPipe and a custom convolutional neural network (CNN) facilitate accurate hand gesture recognition. The system’s real-time processing capabilities ensure seamless and responsive user interaction. Experimental results demonstrate the system’s robustness and accuracy in various scenarios, highlighting its potential for diverse applications such as security, entertainment, and personal assistance.

Human–Robot Collaborative Manufacturing Cell with Learning-Based Interaction Abilities

This paper presents a collaborative manufacturing cell implemented in a laboratory setting, focusing on developing learning-based interaction abilities to enhance versatility and ease of use. The key components of the system include 3D real-time volumetric monitoring for safety, visual recognition of hand gestures for human-to-robot communication, classification of physical-contact-based interaction primitives during handover operations, and detection of hand–object interactions to anticipate human intentions. Due to the nature and complexity of perception, deep-learning-based techniques were used to enhance robustness and adaptability. The main components are integrated in a system containing multiple functionalities, coordinated through a dedicated state machine. This ensures appropriate actions and reactions based on events, enabling the execution of specific modules to complete a given multi-step task. An ROS-based architecture supports the software infrastructure among sensor interfacing, data processing, and robot and gripper controllers nodes. The result is demonstrated by a functional use case that involves multiple tasks and behaviors, paving the way for the deployment of more advanced collaborative cells in manufacturing contexts.

Adaptive shift graph convolutional neural network for hand gesture recognition based on 3D skeletal similarity

Adaptive shift graph convolutional neural network for hand gesture recognition based on 3D skeletal similarity

Hand Gesture Recognition System using Deep Learning

This paper presents a comprehensive exploration of a hand gesture recognition system centered on the YOLOv5 object detection algorithm. The system is meticulously designed to fulfill specific objectives, including real-time gesture detection, high accuracy, scalability, robustness to environmental variability, and seamless integration with other system components. Drawing upon the strengths of YOLOv5, the proposed system aims to provide a reliable and efficient solution for hand gesture recognition, thereby enhancing human- computer interaction and enabling various applications across different domains. The methodology encompasses several stages, including data acquisition, preprocessing, object detection using YOLOv5, integration with other system modules, evaluation, and validation. Through a series of experiments and extensive testing in real-world scenarios, the effectiveness and usability of the system are thoroughly assessed, demonstrating its potential for practical deployment and widespread adoption. Additionally, this paper discusses the implications of the findings, identifies areas for further research and development, and offers insights into the future directions of hand gesture recognition systems utilizing YOLOv5. Overall, this study contributes to advancing the field of human-computer interaction and gesture recognition technology, paving the way for innovative applications and enhanced user experiences in various domains. Keywords— Hand Gesture Recognition, YOLOv5, Real-time Detection, Human-Computer Interaction, Object Detection, Machine Learning

A Methodical Approach to Hardware Platform Design for sEMG Recording and Classification Hand Gesture Recognition using sEMG

A Methodical Approach to Hardware Platform Design for sEMG Recording and Classification Hand Gesture Recognition using sEMG

Advancements and Challenges in Hand Gesture Recognition: A Comprehensive Review

Hand gesture recognition is a quickly developing field with many uses in human-computer interaction, sign language recognition, virtual reality, gaming, and robotics. This paper reviews different ways to model hands, such as vision-based, sensor-based, and data glove-based techniques. It emphasizes the importance of accurate hand modeling and feature extraction for capturing and analyzing gestures. Key features like motion, depth, color, shape, and pixel values and their relevance in gesture recognition are discussed. Challenges faced in hand gesture recognition include lighting variations, complex backgrounds, noise, and real-time performance. Machine learning algorithms are used to classify and recognize gestures based on extracted features. The paper emphasizes the need for further research and advancements to improve hand gesture recognition systems' robustness, accuracy, and usability. This review offers valuable insights into the current state of hand gesture recognition, its applications, and its potential to revolutionize human-computer interaction and enable natural and intuitive interactions between humans and machines. In simpler terms, hand gesture recognition is a way for computers to understand what people are saying with their hands. It has many potential applications, such as allowing people to control computers without touching them or helping people with disabilities communicate. The paper reviews different ways to develop hand gesture recognition systems and discusses the challenges and opportunities in this area.

High‐Performance Textile‐Based Capacitive Strain Sensors via Enhanced Vapor Phase Polymerization of Pyrrole and Their Application to Machine Learning‐Assisted Hand Gesture Recognition

Sensors based on everyday textiles are extremely promising for wearable applications. The present work focuses on high‐performance textile‐based capacitive strain sensors. Specifically, a conductive textile is obtained via vapor‐phase polymerization of pyrrole, in which the usage of methanol co‐vapor and the addition of imidazole to the iron chloride oxidant solution are shown to maximize conductivity. A technique to provide insulation and mechanical resistance using thermoplastic polyurethane and polystyrene‐block‐polyisoprene‐block‐polystyrene/barium titanate composite is developed. Such insulated conductive elastics are then used to fabricate highly sensitive twisted yarn capacitive sensors. A textile glove is subsequently embedded with such sensors. The wireless measurement and transmission system demonstrate efficacy in capturing capacitance variations upon strain and monitoring hand motions. A machine learning model to recognize 12 gestures is implemented—100% classification accuracy is obtained.

Resource-Efficient Deep Learning: Fast Hand Gestures on Microcontrollers

Hand gesture recognition using a camera provides an intuitive and promising means of human-computer interaction and allows operators to execute commands and control machines with simple gestures. Research in hand gesture recognition-based control systems has garnered significant attention, yet the deploying of microcontrollers into this domain remains relatively insignificant. In this study, we propose a novel approach utilizing micro-hand gesture recognition built on micro-bottleneck Residual and micro-bottleneck Conv blocks. Our proposed model, comprises only 42K parameters, is optimized for size to facilitate seamless operation on resource-constrained hardware. Benchmarking conducted on STM32 microcontrollers showcases remarkable efficiency, with the model achieving an average prediction time of just 269ms, marking a 7× faster over the state-of-art model. Notably, despite its compact size and enhanced speed, our model maintains competitive performance result, achieving an accuracy of 99.6% on the ASL dataset and 92% on OUHANDS dataset. These findings underscore the potential for deploying advanced control methods on compact, cost-effective devices, presenting promising avenues for future research and industrial applications.

Cognitive neuropsychological and neuroanatomic predictors of naturalistic action performance in left hemisphere stroke: a retrospective analysis.

Individuals who have experienced a left hemisphere cerebrovascular accident (LCVA) have been shown to make errors in naturalistic action tasks designed to assess the ability to perform everyday activities such as preparing a cup of coffee. Naturalistic action errors in this population are often attributed to limb apraxia, a common deficit in the representation and performance of object-related actions. However, naturalistic action impairments are also observed in right hemisphere stroke and traumatic brain injury, populations infrequently associated with apraxia, and errors across all these populations are influenced by overall severity. Based on these and other data, an alternative (though not mutually exclusive) account is that naturalistic action errors in LCVA are also a consequence of deficits in general attentional resource availability or allocation. In this study, we conducted a retrospective analysis of data from a large group of 51 individuals with LCVA who had completed a test of naturalistic action, along with a battery of tests assessing praxis, attention allocation and control, reasoning, and language abilities to determine which of these capacities contribute uniquely to naturalistic action impairments. Using a regularized regression method, we found that naturalistic action impairments are predicted by both praxis deficits (hand posture sequencing and gesture recognition), as well as attention allocation and control deficits (orienting and dividing attention), along with language comprehension ability and age. Using support vector regression-lesion symptom mapping (SVR-LSM), we also demonstrated that naturalistic action impairments are associated with lesions to posterior middle temporal gyrus and anterior inferior parietal lobule - regions known to be implicated in praxis; as well the middle frontal gyrus that has been implicated in both praxis and attention allocation and control. Taken together, these findings support the hypothesis that naturalistic action impairments in LCVA are a consequence of apraxia as well as deficits in attention allocation and control.

Continuous hand gesture segmentation and acknowledgement of hand gesture path for innovative effort interfaces

Human-computer interaction (HCI) has revolutionized the way we interact with computers, making it more intuitive and user-friendly. It is a dynamic field that has found it is applications in various industries, including multimedia and gaming, where hand gestures are at the forefront. The advent of ubiquitous computing has further heightened the interest in using hand gestures as input. However, recognizing continuous hand gestures presents a set of challenges, primarily stemming from the variable duration of gestures and the lack of clear starting and ending points. Our main objective is to propose a solution: the framework for “continuous palm motion analysis and retrieval” based on “Spatial-temporal and path knowledge”. Framework harnesses the power of cognitive deep learning networks (DLN), offering a significant advancement in the continuous hand gesture recognition domain. we conducted rigorous experiments using a diverse video dataset capturing hand gestures for boasting an impressive F-score of up to 0.99. The potential of our framework to significantly enhance the accuracy and reliability of hand gesture recognition in real-world applications.

Unsupervised Feature Extraction From Raw Data for Gesture Recognition With Wearable Ultralow-Power Ultrasound.

Wearable ultrasound (US) is a novel sensing approach that shows promise in multiple application domains, and specifically in hand gesture recognition (HGR). In fact, US enables to collect information from deep musculoskeletal structures at high spatiotemporal resolution and high signal-to-noise ratio, making it a perfect candidate to complement surface electromyography for improved accuracy performance and on-the-edge classification. However, existing wearable solutions for US-based gesture recognition are not sufficiently low power for continuous, long-term operation. On top of that, practical hardware limitations of wearable US devices (limited power budget, reduced wireless throughput, and restricted computational power) set the need for the compressed size of models for feature extraction and classification. To overcome these limitations, this article presents a novel end-to-end approach for feature extraction from raw musculoskeletal US data suited for edge computing, coupled with an armband for HGR based on a truly wearable (12 cm2, 9 g), ultralow-power (ULP) (16 mW) US probe. The proposed approach uses a 1-D convolutional autoencoder (CAE) to compress raw US data by 20× while preserving the main amplitude features of the envelope signal. The latent features of the autoencoder are used to train an XGBoost classifier for HGR on datasets collected with a custom US armband, considering armband removal/repositioning in between sessions. Our approach achieves a classification accuracy of 96%. Furthermore, the proposed unsupervised feature extraction approach offers generalization capabilities for intersubject use, as demonstrated by testing the pretrained encoder on a different subject and conducting posttraining analysis, revealing that the operations performed by the encoder are subject-independent. The autoencoder is also quantized to 8-bit integers and deployed on a ULP wearable US probe along with the XGBoost classifier, allowing for a gesture recognition rate ≥ 25 Hz and leading to 21% lower power consumption [at 30 frames/s (FPS)] compared to the conventional approach (raw data transmission and remote processing).

An Air Gesture Based Keyboard for Visually Impaired People Using Machine Learning and Embedded System Approach

This report outlines the development of an air gesture keyboard utilizing machine learning technology, designed to assist visually impaired individuals in entering text more easily. This innovative system removes the need for physical keyboards, instead employing a remote-based system that translates hand gestures into text through the use of an accelerometer, Arduino, and switches. There is a significant need for alternative input methods for visually impaired individuals due to the limitations of existing technologies. The proposed air gesture keyboard captures and interprets hand motions in the air, enabling users to input text without physical contact. A literature review was conducted to examine related research on hand gesture recognition using various techniques.

Sign Language Recognition (SLR)

Abstract— Sign Language is mainly used by deaf (hard hearing) and dumb people to exchange information between their own community and with other people. It is a language where people use their hand gestures to communicate as they can’t speak or hear. Sign Language Recognition (SLR) deals with recognizing the hand gestures acquisition and continues till text or speech is generated for corresponding hand gestures. Here hand gestures for sign language can be classified as static and dynamic. However, static hand gesture recognition is simpler than dynamic hand gesture recognition, but both recognition is important to the human community. We can use Deep Learning Computer Vision to recognize the hand gestures by building Deep Neural Network architectures (Convolution Neural Network Architectures. Index Terms – HTML, CSS, Python, Pip, Mediapip, opencv, NumPy