Hand Gesture Recognition Research Articles

An Ensemble Approach with Evolutionary Algorithm for Hand Posture Classification

Grasping is a fundamental action in daily life and particularly evident during mealtime situations where various grasping actions occur with tableware such as chopsticks, spoons, forks, bowls, and cups, each serving specific purposes. While tableware usage varies across regions and cultures, recognizing grasping actions is crucial for assessing performance in daily activities. In this study, we focus on assessing grasping functionality in terms of tableware usage during meals and propose a method for identifying hand movements. In recent years, there has been a surge in developing approaches for hand pose estimation and gesture recognition using deep learning. However, these approaches encounter common challenges, including the need for large-scale datasets, hyperparameter tuning, significant time and computational costs, and limited applicability to incremental learning. To address these challenges, we propose an ensemble approach employing extreme learning machines to recognize grasp postures. In addition, we apply spatiotemporal modeling to extract the relationship between grasp postures and the surrounding tools during mealtimes.

Hand Gesture Recognition from Surface Electromyography Signals with Graph Convolutional Network and Attention Mechanisms

Hand Gesture Recognition from Surface Electromyography Signals with Graph Convolutional Network and Attention Mechanisms

Gesture Driven Gaming: A Deep Dive into Computer Vision-Based Hand Gesture Recognition

Gesture Driven Gaming: A Deep Dive into Computer Vision-Based Hand Gesture Recognition

Hand gestures classification of sEMG signals based on BiLSTM-metaheuristic optimization and hybrid U-Net-MobileNetV2 encoder architecture

Surface electromyography (sEMG) data has been extensively utilized in deep learning algorithms for hand movement classification. This paper aims to introduce a novel method for hand gesture classification using sEMG data, addressing accuracy challenges seen in previous studies. We propose a U-Net architecture incorporating a MobileNetV2 encoder, enhanced by a novel Bidirectional Long Short-Term Memory (BiLSTM) and metaheuristic optimization for spatial feature extraction in hand gesture and motion recognition. Bayesian optimization is employed as the metaheuristic approach to optimize the BiLSTM model’s architecture. To address the non-stationarity of sEMG signals, we employ a windowing strategy for signal augmentation within deep learning architectures. The MobileNetV2 encoder and U-Net architecture extract relevant features from sEMG spectrogram images. Edge computing integration is leveraged to further enhance innovation by enabling real-time processing and decision-making closer to the data source. Six standard databases were utilized, achieving an average accuracy of 90.23% with our proposed model, showcasing a 3–4% average accuracy improvement and a 10% variance reduction. Notably, Mendeley Data, BioPatRec DB3, and BioPatRec DB1 surpassed advanced models in their respective domains with classification accuracies of 88.71%, 90.2%, and 88.6%, respectively. Experimental results underscore the significant enhancement in generalizability and gesture recognition robustness. This approach offers a fresh perspective on prosthetic management and human-machine interaction, emphasizing its efficacy in improving accuracy and reducing variance for enhanced prosthetic control and interaction with machines through edge computing integration.

Recognition of Hand Gestures Using Image with Histogram Feature Extraction and Euclidean Distance Classification Method

Recognition of Hand Gestures Using Image with Histogram Feature Extraction and Euclidean Distance Classification Method

AirStrum: A virtual guitar using real-time hand gesture recognition and strumming technique

AirStrum: A virtual guitar using real-time hand gesture recognition and strumming technique

Hand Gesture Recognition Using Frequency‐Modulated Continuous Wave Radar on Tactile Displays for the Visually Impaired

Touchscreens are essential parts of many electronics in daily lives of sighted people in the digital information era. On the other hand, visually impaired users rely on tactile displays as one of the key communication devices to interact with the digital world. However, due to their working mechanism and the uneven surface of tactile displays, one of the key features of screens for sighted users is surprisingly challenging to implement: precision touch input. To overcome this, a hand gesture recognition system is developed using a frequency‐modulated continuous wave millimeter‐wave radar. A multifeature encoder method is used to obtain the range and velocity information from the radar to translate the data into spectrogram images. Gesture recognition is implemented for common input gestures: single/double‐click, swipe‐right/left, scroll‐up/down, zoom‐in/out, and rotate‐anticlockwise/clockwise. The gesture recognition and classification are based on machine learning, support vector machines, deep learning, and convolutional neural network approaches. The chosen model You‐Only‐Look‐Once (YOLOv8) shows a high accuracy of 97.1% by iterating only 30 epochs with only 500 collected data samples per gesture. This research paves the way toward using radar sensors not only for tactile displays but also for other digital devices in human–computer interaction.

Gesture Recognition for Interactive Presentation Control: A Deep Learning and Edge Computing Approach on Raspberry Pi

Gesture recognition is one of the important tech- nologies for intuitive HCI applications, for ex- ample, in interactive presentations. This re- view focuses on recent work on gesture recog- nition systems, deep learning-based methods for presentation control by hand gestures, on edge devices like the Raspberry Pi. This work de- scribes both static and dynamic frameworks for gesture recognition and human-machine interac- tion techniques based on deep learning.Recent studies show the usability of real-time hand gesture recognition using Python-based systems via Raspberry Pi, stressing portability and ef- ficiency in solutions. The review discusses var- ious deep learning frameworks such as Tensor- Flow, which can be used in image preprocess- ing and model training to enhance the accuracy of gesture detection. Techniques cover wear- able devices to computer vision and demon- strate the flexibility of gesture recognition in a variety of hardware platforms. Advances in edge computing enable complex gesture recog- nition on low- power devices, with reduced la- tency, and enables improved accessibility.The re- view addresses challenges such as gesture type distinction, responsiveness, and environmental variability in developing reliable gesture-based systems. In this regard, it emphasizes the Rasp- berry Pi as an edge computing solution for in- teractive presentation control in HCI. Keywords Gesture Recognition Human-Computer Inter- action (HCI) Edge Computing Deep Learning Real-Time Gesture Detection

Comparative analysis of force sensitive resistor circuitry for use in force myography systems for hand gesture recognition

Wearable technologies for hand gesture classification are becoming increasingly prominent due to the growing need for more natural, human-centered control of complex devices. This need is particularly evident in emerging fields such as virtual reality and bionic prostheses, which require precise control with minimal delay. One method used for hand gesture recognition is force myography (FMG), which utilizes non-invasive pressure sensors to measure radial muscle forces on the skin’s surface of the forearm during hand movements. These sensors, typically force-sensitive resistors (FSRs), require additional circuitry to generate analog output signals, which are then classified using machine learning to derive corresponding control signals for the device. The performance of hand gesture classification can be influenced by the characteristics of this output signal, which may vary depending on the circuitry used. Our study examined three commonly used circuits in FMG systems: the voltage divider (VD), unity gain amplifier (UGA), and transimpedance amplifier (TIA). We first conducted benchtop testing of FSRs to characterize the impact of this circuitry on linearity, deadband, hysteresis, and drift, all metrics with the potential to influence an FMG system’s performance. To evaluate the circuit’s performance in hand gesture classification, we constructed an FMG band with 8 FSRs, using an adjustable Velcro strap and interchangeable circuitry. Wearing the FMG band, participants (N = 15) were instructed to perform 10 hand gestures commonly used in daily living. Our findings indicated that the UGA circuit outperformed others in minimizing hysteresis, drift and deadband with comparable results to the VD, while the TIA circuit excelled in ensuring linearity. Further, contemporary machine learning algorithms used to detect hand gestures were unaffected by the circuitry employed. These results suggest that applications of FMG requiring precise sensing of force values would likely benefit from use of the UGA. Alternatively, if hand gesture state classification is the only use case, developers can take advantage of benefits offered from using less complex circuitry such as the VD.

Posture-invariant myoelectric control with self-calibrating random forests.

Myoelectric control systems translate different patterns of electromyographic (EMG) signals into the control commands of diverse human-machine interfaces via hand gesture recognition, enabling intuitive control of prosthesis and immersive interactions in the metaverse. The effect of arm position is a confounding factor leading to the variability of EMG characteristics. Developing a model with its characteristics and performance invariant across postures, could largely promote the translation of myoelectric control into real world practice. Here we propose a self-calibrating random forest (RF) model which can (1) be pre-trained on data from many users, then one-shot calibrated on a new user and (2) self-calibrate in an unsupervised and autonomous way to adapt to varying arm positions. Analyses on data from 86 participants (66 for pre-training and 20 in real-time evaluation experiments) demonstrate the high generalisability of the proposed RF architecture to varying arm positions. Our work promotes the use of simple, explainable, efficient and parallelisable model for posture-invariant myoelectric control.

Wi-TCG: a WiFi gesture recognition method based on transfer learning and conditional generative adversarial networks

Abstract With the rapid progress of WiFi technology, WiFi-based wireless sensing technology has opened up new ways for contactless human-computer interaction. However, hand gesture recognition technology faces low-quality data sets, insufficient model robustness, poor universality when the application scenario changes, high training costs, and weak generalization ability. To this end, this study innovatively proposes the Wi-TCG method, which combines transfer learning and conditional generative adversarial network (CGAN) to optimize WiFi gesture recognition. This method uses commercial Wi-Fi devices to collect channel state information (CSI) of gesture actions. It innovatively extracts Doppler shift image data as the input of CGAN to generate virtual data with similar characteristics to expand the training sample set. The network is fine-tuned using transfer learning techniques to recognize multiple gesture action categories in different scenarios accurately. In tests of two new natural scenes and six new gesture categories, the Wi-TCG method achieved a high recognition accuracy of 93.1%, providing strong support for applying WiFi-based wireless sensing technology in contactless human-computer interaction.

A Node Point Approach for Real-Time Hand Gesture Recognition Using Support Vector Machines (SVM)

Gesture recognition systems have become increasingly popular in recent years due to their ability to improve the user experience and accessibility in real time. However, accurate and efficient hand gesture recognition remains a challenge, especially in dynamic environments. This study presents real-time hand gestures and poses using computer vision. Fundamentally, the system can identify and categorize a wide range of hand movements through real-time analysis of the node connect points of the user's fingers. Every successive point is rendered in such a way that associations between several points have been oriented, and differences between each gesture may be classified. Apart from this, the machine learning model is to be trained on such joint points of the user’s fingers using angle approximation in between them and gradient as per pixels per inch according to a hand gesture-capturing adapter like a camera. For this task to be done, we aim to use the Support Vector Machine (SVM) algorithm for recognizing the classifying parameters of live gestures. The study utilizes support vector machines (SVM) to deliberately cluster signals according to their node focuses. To enhance the accuracy and resilience of the whole model, a support vector machine is incorporated into the classifier to maximize the classification function of the courier-based domain.

Efficient Hand Gesture Recognition Using Artificial Intelligence and IMU-Based Wearable Device

Efficient Hand Gesture Recognition Using Artificial Intelligence and IMU-Based Wearable Device

Uncertainty-Aware Hand Gesture Recognition for Safety-Critical and Emergency Human-Robot Interaction

Uncertainty-Aware Hand Gesture Recognition for Safety-Critical and Emergency Human-Robot Interaction

Multichannel EMG-based gesture recognition utilizing advanced machine learning techniques: A random forest classifier for high-precision signal classification

This research examines how advanced machine learning algorithms can be used to classify multichannel electromyographic (EMG) signals with a high level of accuracy to assist in recognizing hand gestures. The goal is to create a robust and scalable system for gesture-based virtual control using EMG signals with potential applications in assistive technologies, rehabilitation, and human-computer interaction. Data were gathered using a MYO Thalmic bracelet containing eight EMG sensors on thirty-six subjects, and a Random Forest classifier was trained to identify seven distinct types of hand gestures (rest, fist clench, wrist flexion/extension, and radial/ulnar deviations). The machine learning pipeline included extensive preprocessing (i.e., EMG signal normalization and signal feature extraction; root mean square, waveform length, and zero crossing rate) and several hyperparameter tuning procedures to improve model performance. The Random Forest model (100 decision trees) achieved an overall classification accuracy of 98.68%, with a range of accuracies for each class (e.g., 95.2% wrist flexion and 91.8% ulnar deviation) when evaluated using cross-validation (i.e., average F1-score = 0.92, precision = 0.94, recall = .91). Overall, the study provides strong evidence for the effectiveness of ensemble learning methods at analyzing complex, multidimensional EMG signals. The high classification accuracy reported, in particular, demonstrates that the system could function for real-time recognition of hand gestures in a virtual environment. Ultimately, the initial work sets the stage for future exploration of a model that may be integrated with actuation models to control prosthetic limbs, virtual actors/avatars, and robotic devices. By demonstrating a scalable and efficient method of gesture recognition using EMG signals, these early findings enable future pathways and possibilities to design innovative, assistive solutions for digital systems that increase accessibility and interaction for users who are motor impaired or have a limited range of motion.

The ASL Dataset for Real-Time Recognition and Integration with LLM Services

This study aims to investigate the impact of hand gesture recognition techniques on the efficiency of American Sign Language (ASL) interpretation, addressing a critical gap in the existing literature. The research seeks new insights into the challenges of automated sign language recognition, contributing to a deeper understanding of accessibility in communication for the deaf and hard-of-hearing community. The study employs a quantitative approach, using a dataset comprising hand gesture images representing the static letters of the ASL alphabet collected from multiple users. Data were collected from various individuals to ensure diversity and analyzed using machine learning models to evaluate their effectiveness in recognizing ASL signs. The results reveal that the machine learning models implemented achieved a high accuracy rate in recognizing hand gestures, indicating that person-specific variations do not significantly hinder performance. These findings provide evidence that the proposed dataset and methodologies can improve the reliability of sign language recognition systems, offering significant implications for the development of more inclusive communication technologies. This research offers a novel perspective on sign language recognition, providing valuable insight that extends the current understanding of gesture-based communication systems. The study’s findings contribute to advancements in accessibility technologies, highlighting areas for future research and practical applications in improving communication for the Deaf and hard of hearing community.

An Artificial Intelligence-Assisted Flexible and Wearable Mechanoluminescent Strain Sensor System

HighlightsThe sandwich-structured flexible mechanoluminescent sensor (SFLC) film shows great application potential as wireless wearable strain sensor and encryption device.System-level integration of SFLC film with deep learning-based artificial intelligence enables fast and accurate interpretation of color data to strain values with automatic correction of errors caused by varying color temperatures.The smart glove wearable sensor based on the SFLC film combined with deep learning neural network enables fast and accurate hand gesture recognition.

Enhancing manual inspection in semiconductor manufacturing with integrated augmented reality solutions

On-site routine inspection often remains a manual operation in the semiconductor manufacturing industry because implementing automated solutions can be costly and technically challenging in such a highly controlled and complex environment. The manual inspection is prone to errors due to the impact of demanding physical and mental workloads. This paper presents an integrated Augmented Reality (AR) solution developed to assist manual inspection tasks in the supporting areas of semiconductor manufacturing, referred to as the sub-fab. The solution is accessible to a human worker wearing an AR headset during the inspection process at the location. We propose a system framework to deploy computational intelligences of varying granularity provided by the solution across cloud, edge, and device levels, accommodating constraints within the sub-fab. A machine maintenance module helps estimate and monitor the health condition of running scrubbers. Incorrect intentions performed by the worker on the scrubber control panel are detected through hand gesture recognition. This instantly prompts warning messages in the AR headset to prevent subsequent wrong actions. The solution can also identify abnormal device states through 6D pose estimation of objects enabled by machine learning models. A test scenario demonstrates how these functional features enhance the inspection efficiency and quality by reducing human workloads. This work demonstrates that semiconductor manufacturing may require AR-assisted functions different from those needed or common in other industrial sectors. It also highlights the potential of AR technology for reducing operational human errors in manual tasks.

High-sensitivity and damage redundant detection capable gel-state [BMIM][BF4] electronic skin for aerospace applications

Electronic skin (e-skin) holds significant potential for applications in robotics, Internet-of-things, and health monitoring. However, conventional e-skins often exhibit decreased sensitivity and delayed response in ultra-low temperature environments due to the freezing of conductive materials. Moreover, an increased brittleness can cause substrate damage, limiting their application in cryogenic conditions. To address existing challenges, an ionic liquid [BMIM][BF4] is cross-linked with N,N'-Bis(2-hydroxyethyl)oxamide to form an ionic gel for this study. The gel exhibits excellent electrical performance at −71 °C, significantly expanding the operating temperature range of e-skins. Additionally, gelation decreases resistive drift and leakage inherent to ionic liquids. The prepared sensing units demonstrate high sensitivity to pressure loading across an ultra-wide temperature range of −50–50 °C, with linear sensing within the range of 48–32,000 Pa and a rapid response time of 0.05 s. Integration of the units into large-scaled e-skin enables precise recognition of static and dynamic pressure loads in ultra-low temperature environments (−50 °C). Furthermore, a space glove model assembled using the sensing units achieves accurate recognition of hand gestures in extreme conditions. The sensing units retain over 60 % of their pressure response even when damaged, and demonstrate resilience to environmental factors including low temperatures and vacuum conditions. Along with exceptional performance and environmental resilience under harsh conditions, the gel-state [BMIM][BF4] e-skin shows great potential in deep space exploration, polar expeditions, and other challenging environments.

Research on gesture recognition technology based on machine learning

Abstract. Hand Gesture Recognition (HGR), as a significant technological advancement in the field of Human-Computer Interaction (HCI), aims to develop systems capable of accurately recognizing and interpreting human gestures for a diverse range of applications, including device control, virtual reality, gesture passwords, and gesture interaction. With the continuous advancement of machine learning algorithms, especially deep learning techniques, machine learning-based gesture recognition has garnered widespread attention. This paper presents a review of the development of gesture recognition techniques from traditional approaches to the current mainstream deep learning-based methods, and outlines the challenges and technical difficulties encountered. It analyzes several of the most popular classification techniques, including Naive Bayes, K-Nearest Neighbors (KNN), Random Forests, XGBoost, Support Vector Classifiers (SVCs), and Convolutional Neural Networks (CNNs). Furthermore, this paper examines the application of these algorithms in both dynamic and static gesture recognition and compares their performance and suitability in different scenarios. The results demonstrate that the accuracy and robustness of gesture recognition systems can be markedly enhanced through the prudent selection and optimization of the algorithms, which serves as a valuable reference for future research and applications.