Hand Gesture Recognition System Research Articles

Comparison of SEMG-Based Hand Gesture Classifiers

Machine learning techniques have shown success in classifying hand gestures. As the prevalence of prosthetic devices continues to rise, the adoption of non-invasive technologies, such as surface electromyography (sEMG), becomes paramount. This study systematically assesses the isolated influence of classification algorithms within hand gesture recognition (HGR) systems using sEMG data and dynamic time warping (DTW) based features. This approach effectively handles temporal variations in sEMG signals by leveraging DTW, ensuring input features are invariant to gesture speed. Six supervised learning classifiers were evaluated: the multilayer perceptron, support vector machine, logistic regression, linear discriminant analysis, k-nearest neighbors, and decision tree. Cross-validation was employed to fine-tune the segmentation hyperparameters, significantly improving results. To ensure reproducibility, the source code has been made available, the proposed system design has been detailed, and the evaluation protocols have been described. Our findings indicate that logistic regression outperformed other classifiers in this setup, achieving 95.2% accuracy in classifying six hand movements from ten healthy individuals, representing a 1.6% improvement over the best previously reported performance using the same publicly available dataset. Future research will assess the proposed HGR system’s generalization capability on larger datasets suitable for training more complex classifiers, including deep learning models.

Enhancement algorithm for surface electromyographic-based gesture recognition based on real-time fusion of muscle fatigue features

This study aims to optimize surface electromyography-based gesture recognition technique, focusing on the impact of muscle fatigue on the recognition performance. An innovative real-time analysis algorithm is proposed in the paper, which can extract muscle fatigue features in real time and fuse them into the hand gesture recognition process. Based on self-collected data, this paper applies algorithms such as convolutional neural networks and long short-term memory networks to provide an in-depth analysis of the feature extraction method of muscle fatigue, and compares the impact of muscle fatigue features on the performance of surface electromyography-based gesture recognition tasks. The results show that by fusing the muscle fatigue features in real time, the algorithm proposed in this paper improves the accuracy of hand gesture recognition at different fatigue levels, and the average recognition accuracy for different subjects is also improved. In summary, the algorithm in this paper not only improves the adaptability and robustness of the hand gesture recognition system, but its research process can also provide new insights into the development of gesture recognition technology in the field of biomedical engineering.

Exploring User-Defined Gestures as Input for Hearables and Recognizing Ear-Level Gestures with IMUs

Hearables are highly functional earphone-type wearables; however, existing input methods using stand-alone hearables are limited in the number of commands, and there is a need to extend device operation through hand gestures. In previous research on hearables for hand input, user understanding and gesture recognition systems have been developed. However, in the realm of user understanding, investigation concerning hand input with hearables remains incomplete, and existing recognition systems have not demonstrated proficiency in discerning user-defined gestures. In this study, we conducted a gesture elicitation study (GES) assuming hand input using hearables under six conditions (three interaction areas x two device shapes). Then, we extracted ear-level gestures that the device's built-in IMU sensor could recognize from the user-defined gestures and investigated the recognition performance. The results of sitting experiments showed that the gesture recognition rate for in-ear devices was 91.0% and that for ear-hook devices was 74.7%.

Developing a real-time hand exoskeleton system that controlled by a hand gesture recognition system via wireless sensors

Numerous studies have been conducted using wearable sensors on gesture recognition and classification methods for the control of robotic arms, prostheses, and exoskeleton systems. The primary goal of this study is to classify 32 different hand gestures in real-time using electromyography (EMG) signals. EMG signals obtained by measuring the electrical activity of the muscles were simulated wirelessly on a hand exoskeleton. A Myo Armband placed on the upper arm was used to generate these signals for evaluating and validating the system. The signals were normalized according to maximum voluntary contraction (MVC), and relevant features such as MAV, STD, VAR, RMS, IEMG, ZC, and WL were selected using the univariate feature selection. A support vector machine (SVM) was then used for classification. The 32 different hand gestures were classified to an accuracy level of 99.9%. In short, this proposed approach can be effectively used in real-time hand gesture recognition and generating real-time actions for the hand exoskeleton system.

Real-Time Hand Gesture Recognition: A Comprehensive Review of Techniques, Applications, and Challenges

Abstract Real-time Hand Gesture Recognition (HGR) has emerged as a vital technology in human-computer interaction, offering intuitive and natural ways for users to interact with computer-vision systems. This comprehensive review explores the advancements, challenges, and future directions in real-time HGR. Various HGR-related technologies have also been investigated, including sensors and vision technologies, which are utilized as a preliminary step in acquiring data in HGR systems. This paper discusses different recognition approaches, from traditional handcrafted feature methods to state-of-the-art deep learning techniques. Learning paradigms have been analyzed such as supervised, unsupervised, transfer, and adaptive learning in the context of HGR. A wide range of applications has been covered, from sign language recognition to healthcare and security systems. Despite significant developments in the computer vision domain, challenges remain in areas such as environmental robustness, gesture complexity, computational efficiency, and user adaptability. Lastly, this paper concludes by highlighting potential solutions and future research directions trying to develop more robust, efficient, and user-friendly real-time HGR systems.

Next-Gen Dynamic Hand Gesture Recognition: MediaPipe, Inception-v3 and LSTM-Based Enhanced Deep Learning Model

Gesture recognition is crucial in computer vision-based applications, such as drone control, gaming, virtual and augmented reality (VR/AR), and security, especially in human–computer interaction (HCI)-based systems. There are two types of gesture recognition systems, i.e., static and dynamic. However, our focus in this paper is on dynamic gesture recognition. In dynamic hand gesture recognition systems, the sequences of frames, i.e., temporal data, pose significant processing challenges and reduce efficiency compared to static gestures. These data become multi-dimensional compared to static images because spatial and temporal data are being processed, which demands complex deep learning (DL) models with increased computational costs. This article presents a novel triple-layer algorithm that efficiently reduces the 3D feature map into 1D row vectors and enhances the overall performance. First, we process the individual images in a given sequence using the MediaPipe framework and extract the regions of interest (ROI). The processed cropped image is then passed to the Inception-v3 for the 2D feature extractor. Finally, a long short-term memory (LSTM) network is used as a temporal feature extractor and classifier. Our proposed method achieves an average accuracy of more than 89.7%. The experimental results also show that the proposed framework outperforms existing state-of-the-art methods.

Hand Gesture Recognition Using Deep Learning

Hand gesture recognition (HGR) has gained significant attention due to its potential for various applications. This paper explores the use of deep learning, specifically Convolutional Neural Networks (CNNs), for HGR using the TensorFlow library. We investigate existing research on CNN-based HGR, focusing on image classification tasks. We then provide a brief overview of CNNs and their suitability for image recognition. Subsequently, we describe the typical workflow of a deep learning-based HGR system, including data preprocessing, hand detection, feature extraction with CNNs, and classification. We highlight the advantages of using TensorFlow to build and train CNN models for HGR. Finally, we conclude by summarizing the key findings from related work and mentioning the specific dataset and number of gestures classified in our research. This work contributes to the growing body of research on CNN-based HGR using TensorFlow and emphasizes its potential for developing accurate and efficient HGR systems.

Hand Gesture Recognition System using Deep Learning

This paper presents a comprehensive exploration of a hand gesture recognition system centered on the YOLOv5 object detection algorithm. The system is meticulously designed to fulfill specific objectives, including real-time gesture detection, high accuracy, scalability, robustness to environmental variability, and seamless integration with other system components. Drawing upon the strengths of YOLOv5, the proposed system aims to provide a reliable and efficient solution for hand gesture recognition, thereby enhancing human- computer interaction and enabling various applications across different domains. The methodology encompasses several stages, including data acquisition, preprocessing, object detection using YOLOv5, integration with other system modules, evaluation, and validation. Through a series of experiments and extensive testing in real-world scenarios, the effectiveness and usability of the system are thoroughly assessed, demonstrating its potential for practical deployment and widespread adoption. Additionally, this paper discusses the implications of the findings, identifies areas for further research and development, and offers insights into the future directions of hand gesture recognition systems utilizing YOLOv5. Overall, this study contributes to advancing the field of human-computer interaction and gesture recognition technology, paving the way for innovative applications and enhanced user experiences in various domains. Keywords— Hand Gesture Recognition, YOLOv5, Real-time Detection, Human-Computer Interaction, Object Detection, Machine Learning

Advancements and Challenges in Hand Gesture Recognition: A Comprehensive Review

Hand gesture recognition is a quickly developing field with many uses in human-computer interaction, sign language recognition, virtual reality, gaming, and robotics. This paper reviews different ways to model hands, such as vision-based, sensor-based, and data glove-based techniques. It emphasizes the importance of accurate hand modeling and feature extraction for capturing and analyzing gestures. Key features like motion, depth, color, shape, and pixel values and their relevance in gesture recognition are discussed. Challenges faced in hand gesture recognition include lighting variations, complex backgrounds, noise, and real-time performance. Machine learning algorithms are used to classify and recognize gestures based on extracted features. The paper emphasizes the need for further research and advancements to improve hand gesture recognition systems' robustness, accuracy, and usability. This review offers valuable insights into the current state of hand gesture recognition, its applications, and its potential to revolutionize human-computer interaction and enable natural and intuitive interactions between humans and machines. In simpler terms, hand gesture recognition is a way for computers to understand what people are saying with their hands. It has many potential applications, such as allowing people to control computers without touching them or helping people with disabilities communicate. The paper reviews different ways to develop hand gesture recognition systems and discusses the challenges and opportunities in this area.

Multimodal Gesture Recognition with Spatio-Temporal Features Fusion Based on YOLOv5 and MediaPipe

As a natural, intuitive and easy-to-learn mode of interaction, gesture plays an important role in communication. Hand detection, containing multimodal information, includes static and dynamic detection and involves intricate spatial relationship problems such as different hand sizes, complex joints, occlusion and self-occlusion. This study focused on a multimodal hand gesture recognition system based on YOLOv5 and MediaPipe with fused spatio-temporal features. First, the Mediapipe and OpenCV libraries were employed to implement hand keypoint detection. Subsequently, the human–computer interaction (HCI) of volume control was realized by identifying the distance between thumb and index. Finally, model training was conducted based on the YOLOv5 algorithm, and the recognition of different gesture categories was realized. The performance was evaluated and compared through YOLOv5s, YOLOv5m, and YOLOv5l. The gesture recognition system interface visualization was achieved through pyqt5. Experiments show that the average detection accuracy of the model is 99.4% and the recognition speed is around 0.2[Formula: see text]s.

Efficient Learning for hearing-impaired: Two Way Sign Language Translator using AI

This research paper describes the design and im- plementation of a two-way sign language translator for deaf- mute and hearing individuals. The device uses a hand gesture recognition system to translate sign language into text for hearing individuals and converts text into sign language videos for deaf- mute individuals. The paper goes over the technical challenges and solutions involved in creating a legitimate system that recognizes gestures and a sign language video generation system. The results of the user testing with deaf-mute and hearing individuals show that the device has high accuracy and is user-friendly. The two-way sign language translator has the possibility of enhancing communication and accessibility in daily interactions for deaf-mute people.

Hand Gesture Recognition System Based on Indian Sign Language Using SVM and CNN

Hand gestures (HGs) have been extensively acknowledged as a communication standard for the deaf and dumb school system in the Indian sign language (ISL). We suggested an automated Indian sign language recognition (ISLR) method for English in this study. Initially, the hand region is segmented using the Grasshopper optimization algorithm (GOA) based on the skin color model method. The segmentation’s efficacy is further assessed using three techniques: GOA-based skin color detection algorithm (SCDA), particle swarm optimization-based SCDA (PSO-SCDA), and artificial bee colony-based SCDA (ABC-SCDA). Finally, a database for the retrieved motions representing distinct English alphabets is created. For gesture recognition, the system is trained, and a template-based matching approach is used. The classification technique employs the support vector machine (SVM) and the convolution neural network (CNN). Our suggested recognition method achieved the highest accuracy of 97.85% with GOA-SCDA, compared to 89.29% and 93.96% with PSO-SCDA and ABC-SCDA, respectively. Furthermore, CNN outperformed SVM in classification, with a high accuracy of 99.2% and precision of 81.8%.

Hand gesture-based automatic door security system using squeeze and excitation residual networks

Viruses can be transmitted due to various aspects; one spreads through airborne droplets or the touch of multiple objects. This can occur in any area, including the entrance to the house or access to a room or deposit box. The spread of viruses that cause diseases like Covid-19 has caused many human casualties, and there is still the possibility of similar conditions appearing in the future. Several things need to be done to reduce the chances of spreading disease due to viruses, including developing contactless security support methods. This paper proposes a security system using hand gesture recognition using squeeze and excitation residual networks (SE-ResNet). This research offers a hand gesture recognition system for an automatic door system using SE-ResNet and the residual network (ResNet).

Electromyography signal based hand gesture classification system using Hilbert Huang transform and deep neural networks

This research aims to provide the groundwork for smartly categorizing hand movements for use with prosthetic hands. The hand motions are classified using surface electromyography (sEMG) data. In reaction to a predetermined sequence of fibre activation, every single one of our muscles contracts. They could be useful in developing control protocols for bio-control systems, such human-computer interaction and upper limb prostheses. When focusing on hand gestures, data gloves and vision-based approaches are often used. The data glove technique requires tedious and unnatural user engagement, whereas the vision-based solution requires significantly more expensive sensors. This research offered a Deep Neural Network (DNN) automated hand gesticulation recognition system based on electromyography to circumvent these restrictions. This work primarily aims to augment the concert of the hand gesture recognition system via the use of an artificial classifier. To advance the recognition system's classification accuracy, this study explains how to build models of neural networks and how to use signal processing methods. By locating the Hilbert Huang Transform (HHT), one may get the essential properties of the signal. When training a DNN classifier, these characteristics are sent into it. The investigational results reveal that the suggested technique accomplishes a better categorization rate (98.5 % vs. the alternatives).

A memory-friendly class-incremental learning method for hand gesture recognition using HD-sEMG

Hand gesture recognition (HGR) plays a vital role in human-computer interaction. The integration of high-density surface electromyography (HD-sEMG) and deep neural networks (DNNs) has significantly improved the robustness and accuracy of HGR systems. These methods are typically effective for a fixed set of trained gestures. However, the need for new gesture classes over time poses a challenge. Introducing new classes to DNNs can lead to a substantial decrease in accuracy for previously learned tasks, a phenomenon known as “catastrophic forgetting,” especially when the training data for earlier tasks is not retained and retrained. This issue is exacerbated in embedded devices with limited storage, which struggle to store the large-scale data of HD-sEMG. Class-incremental learning (CIL) is an effective method to reduce catastrophic forgetting. However, existing CIL methods for HGR rarely focus on reducing memory load. To address this, we propose a memory-friendly CIL method for HGR using HD-sEMG. Our approach includes a lightweight convolutional neural network, named SeparaNet, for feature representation learning, coupled with a nearest-mean-of-exemplars classifier for classification. We introduce a priority exemplar selection algorithm inspired by the herding effect to maintain a manageable set of exemplars during training. Furthermore, a task-equal-weight exemplar sampling strategy is proposed to effectively reduce memory load while preserving high recognition performance. Experimental results on two datasets demonstrate that our method significantly reduces the number of retained exemplars to only a quarter of that required by other CIL methods, accounting for less than 5 ​% of the total samples, while still achieving comparable average accuracy.

Conformal, stretchable, breathable, wireless epidermal surface electromyography sensor system for hand gesture recognition and rehabilitation of stroke hand function

Surface electromyography (sEMG) plays a significant role in the everyday practice of clinic hand function rehabilitation. The materials and design of current typical clinic sEMG electrodes are rigid Ag/AgCl or flexible polyimide (PI) film, which cannot provide a stable interface between electrodes and skin for adequate long-term high-quality data. Thus, conformal, soft, breathable, wireless epidermal sEMG sensor systems have broad potential relevance to clinic rehabilitation settings. Herein, we demonstrate a stretchable epidermal sEMG sensor array system with optimized materials and structure strategies for hand gesture recognition and hand function rehabilitation. The optimized serpentine structures with marvelous stretchability and improved fill ratio, provide lower impedance and high-quality sEMG signals. Moreover, the easy-to-use airhole method further ensures stable and long-term contact with the skin for recording. In addition, integrated with a customized flexible wireless data acquisition system, the capability for real-time 8-channel sEMG monitoring is developed, and taking together with the CNN-based algorithm, the system can automatically and reliably realize the 7 kinds of hand gestures with an accuracy of 81.02%. Moreover, the low-cost yet high-performance epidermal sEMG sensor system demonstrated its conceptual feasibility in quantitatively evaluation of stroke patient’s hand and facilitating human-robot collaboration in hand rehabilitation by proof-of-the-concept clinical testing.

Adapted Contrastive Predictive Coding Framework for Accessible Smart Home Control System Based on Hand Gestures Recognition

Smart home control systems have been widely used to control multiple smart home devices such as smart TVs, smart HVAC, and smart bulbs. While targeting the design of a smart home environment, the accessibility of the entire system is crucial to achieving a comfortable and accessible environment for all home individuals. Thus, in this paper, we are aiming to improve the daily quality of life of the elderly and Disabled individuals by improving the user experience of a vision-based accessible home control system based on an integrated self-supervised and supervised learning framework that provides a robust capability of extracting the features to enhance the overall performance via adapting the contrastive predictive coding concepts and deep learning for real-time hand gesture recognition system. The proposed framework provides an accessible smart home control system with a significant improvement in the user experience.

Hand Gesture Recognition System Using Deep Learning

Hand Gesture Recognition Systems have undergone significant advancements, ushering in a new era of human-computer interaction. This paper offers a thorough examination of the current state of the art in hand gesture recognition, addressing both the notable progress achieved and the persistent challenges. By leveraging state-of-the-art technologies such as computer vision and deep learning, the paper explores the methodologies employed in data collection, preprocessing, and the implementation of various algorithms. The research delves into the complexities of popular hand gesture datasets, emphasizing their role in training and testing models. A critical analysis of different algorithms and models, including Hidden Markov Models, Support Vector Machines, and Neural Networks, is presented. The paper scrutinizes their strengths and limitations, providing insights into the delicate balance between accuracy and real-time processing. Furthermore, it investigates the diverse applications of hand gesture recognition, spanning from enriching human-computer interaction to its pivotal role in virtual reality, gaming, and robotics. Despite these advancements, challenges persist, such as occlusion, varying lighting conditions, and the imperative for real-time processing. The hardware utilized in hand gesture recognition systems, including depth sensors, RGB-D cameras, and wearable devices, is examined. Evaluation metrics, such as accuracy, precision, recall, and the F1 score, are employed to evaluate system performance.

A Review on the Development of Library/Office Assistant Robots

In today's rapidly evolving landscape, the integration of assistive robots in libraries is gaining momentum. This review paper scrutinizes the evolution of library assistant robots, emphasizing their significance in enhancing operational efficiency and user experiences. Drawing upon a comprehensive analysis of existing literature, we examine various methodologies employed in the development of library assistant robots, encompassing both hardware and software components. From line-following mechanisms to sophisticated hand gesture recognition systems, the methodologies explored showcase a diverse array of approaches to address the unique needs of library environments. Our contribution lies in comparing and contrasting different approaches and technologies utilized in the development of library assistant robots. By synthesizing insights from multiple research endeavors, we offer a comprehensive overview of the advancements in this field, highlighting key technological innovations and their potential implications for future developments. Keywords— Library Assistant Robot, Pattern Recognition Robots; Visually-Guided Grasping.

GestureFusion: A Gesture-based Interaction System

Abstract: In an era of increasingly sophisticated human-computer interaction, the ability to comprehend and interpret hand gestures has become a pivotal element for bridging the gap between humans and machines. This project, titled ”GestureFusion: A Gesture-based Interaction System,” seeks to contribute to this rapidly evolving field by developing a comprehensive system for recognizing and categorizing hand gestures through the application of computer vision techniques. The primary objective of this project is to create a versatile and extensible hand gesture recognition system that can be employed in various domains such as virtual reality, robotics, sign language interpretation, and human-computer interface design. To achieve this, the project employs state-of-the-art computer vision algorithms and deep learning techniques. Key components of this project include data collection and annotation, model training and optimization, the development of a user-friendly application programming interface (API) for integration into diverse applications.