Hand Gesture Recognition System Research Articles

Hand gesture-based automatic door security system using squeeze and excitation residual networks

Viruses can be transmitted due to various aspects; one spreads through airborne droplets or the touch of multiple objects. This can occur in any area, including the entrance to the house or access to a room or deposit box. The spread of viruses that cause diseases like Covid-19 has caused many human casualties, and there is still the possibility of similar conditions appearing in the future. Several things need to be done to reduce the chances of spreading disease due to viruses, including developing contactless security support methods. This paper proposes a security system using hand gesture recognition using squeeze and excitation residual networks (SE-ResNet). This research offers a hand gesture recognition system for an automatic door system using SE-ResNet and the residual network (ResNet).

Electromyography signal based hand gesture classification system using Hilbert Huang transform and deep neural networks

This research aims to provide the groundwork for smartly categorizing hand movements for use with prosthetic hands. The hand motions are classified using surface electromyography (sEMG) data. In reaction to a predetermined sequence of fibre activation, every single one of our muscles contracts. They could be useful in developing control protocols for bio-control systems, such human-computer interaction and upper limb prostheses. When focusing on hand gestures, data gloves and vision-based approaches are often used. The data glove technique requires tedious and unnatural user engagement, whereas the vision-based solution requires significantly more expensive sensors. This research offered a Deep Neural Network (DNN) automated hand gesticulation recognition system based on electromyography to circumvent these restrictions. This work primarily aims to augment the concert of the hand gesture recognition system via the use of an artificial classifier. To advance the recognition system's classification accuracy, this study explains how to build models of neural networks and how to use signal processing methods. By locating the Hilbert Huang Transform (HHT), one may get the essential properties of the signal. When training a DNN classifier, these characteristics are sent into it. The investigational results reveal that the suggested technique accomplishes a better categorization rate (98.5 % vs. the alternatives).

A memory-friendly class-incremental learning method for hand gesture recognition using HD-sEMG

Hand gesture recognition (HGR) plays a vital role in human-computer interaction. The integration of high-density surface electromyography (HD-sEMG) and deep neural networks (DNNs) has significantly improved the robustness and accuracy of HGR systems. These methods are typically effective for a fixed set of trained gestures. However, the need for new gesture classes over time poses a challenge. Introducing new classes to DNNs can lead to a substantial decrease in accuracy for previously learned tasks, a phenomenon known as “catastrophic forgetting,” especially when the training data for earlier tasks is not retained and retrained. This issue is exacerbated in embedded devices with limited storage, which struggle to store the large-scale data of HD-sEMG. Class-incremental learning (CIL) is an effective method to reduce catastrophic forgetting. However, existing CIL methods for HGR rarely focus on reducing memory load. To address this, we propose a memory-friendly CIL method for HGR using HD-sEMG. Our approach includes a lightweight convolutional neural network, named SeparaNet, for feature representation learning, coupled with a nearest-mean-of-exemplars classifier for classification. We introduce a priority exemplar selection algorithm inspired by the herding effect to maintain a manageable set of exemplars during training. Furthermore, a task-equal-weight exemplar sampling strategy is proposed to effectively reduce memory load while preserving high recognition performance. Experimental results on two datasets demonstrate that our method significantly reduces the number of retained exemplars to only a quarter of that required by other CIL methods, accounting for less than 5 ​% of the total samples, while still achieving comparable average accuracy.

Conformal, stretchable, breathable, wireless epidermal surface electromyography sensor system for hand gesture recognition and rehabilitation of stroke hand function

Surface electromyography (sEMG) plays a significant role in the everyday practice of clinic hand function rehabilitation. The materials and design of current typical clinic sEMG electrodes are rigid Ag/AgCl or flexible polyimide (PI) film, which cannot provide a stable interface between electrodes and skin for adequate long-term high-quality data. Thus, conformal, soft, breathable, wireless epidermal sEMG sensor systems have broad potential relevance to clinic rehabilitation settings. Herein, we demonstrate a stretchable epidermal sEMG sensor array system with optimized materials and structure strategies for hand gesture recognition and hand function rehabilitation. The optimized serpentine structures with marvelous stretchability and improved fill ratio, provide lower impedance and high-quality sEMG signals. Moreover, the easy-to-use airhole method further ensures stable and long-term contact with the skin for recording. In addition, integrated with a customized flexible wireless data acquisition system, the capability for real-time 8-channel sEMG monitoring is developed, and taking together with the CNN-based algorithm, the system can automatically and reliably realize the 7 kinds of hand gestures with an accuracy of 81.02%. Moreover, the low-cost yet high-performance epidermal sEMG sensor system demonstrated its conceptual feasibility in quantitatively evaluation of stroke patient’s hand and facilitating human-robot collaboration in hand rehabilitation by proof-of-the-concept clinical testing.

Hand Gesture Recognition System Using Deep Learning

Hand Gesture Recognition Systems have undergone significant advancements, ushering in a new era of human-computer interaction. This paper offers a thorough examination of the current state of the art in hand gesture recognition, addressing both the notable progress achieved and the persistent challenges. By leveraging state-of-the-art technologies such as computer vision and deep learning, the paper explores the methodologies employed in data collection, preprocessing, and the implementation of various algorithms. The research delves into the complexities of popular hand gesture datasets, emphasizing their role in training and testing models. A critical analysis of different algorithms and models, including Hidden Markov Models, Support Vector Machines, and Neural Networks, is presented. The paper scrutinizes their strengths and limitations, providing insights into the delicate balance between accuracy and real-time processing. Furthermore, it investigates the diverse applications of hand gesture recognition, spanning from enriching human-computer interaction to its pivotal role in virtual reality, gaming, and robotics. Despite these advancements, challenges persist, such as occlusion, varying lighting conditions, and the imperative for real-time processing. The hardware utilized in hand gesture recognition systems, including depth sensors, RGB-D cameras, and wearable devices, is examined. Evaluation metrics, such as accuracy, precision, recall, and the F1 score, are employed to evaluate system performance.

A Review on the Development of Library/Office Assistant Robots

In today's rapidly evolving landscape, the integration of assistive robots in libraries is gaining momentum. This review paper scrutinizes the evolution of library assistant robots, emphasizing their significance in enhancing operational efficiency and user experiences. Drawing upon a comprehensive analysis of existing literature, we examine various methodologies employed in the development of library assistant robots, encompassing both hardware and software components. From line-following mechanisms to sophisticated hand gesture recognition systems, the methodologies explored showcase a diverse array of approaches to address the unique needs of library environments. Our contribution lies in comparing and contrasting different approaches and technologies utilized in the development of library assistant robots. By synthesizing insights from multiple research endeavors, we offer a comprehensive overview of the advancements in this field, highlighting key technological innovations and their potential implications for future developments. Keywords— Library Assistant Robot, Pattern Recognition Robots; Visually-Guided Grasping.

GestureFusion: A Gesture-based Interaction System

Abstract: In an era of increasingly sophisticated human-computer interaction, the ability to comprehend and interpret hand gestures has become a pivotal element for bridging the gap between humans and machines. This project, titled ”GestureFusion: A Gesture-based Interaction System,” seeks to contribute to this rapidly evolving field by developing a comprehensive system for recognizing and categorizing hand gestures through the application of computer vision techniques. The primary objective of this project is to create a versatile and extensible hand gesture recognition system that can be employed in various domains such as virtual reality, robotics, sign language interpretation, and human-computer interface design. To achieve this, the project employs state-of-the-art computer vision algorithms and deep learning techniques. Key components of this project include data collection and annotation, model training and optimization, the development of a user-friendly application programming interface (API) for integration into diverse applications.

Hand Gestures for Personal Computer Control

Abstract: Hand gestures have emerged as a promising modality for enhancing personal computer (PC) control, offering intuitive and natural interaction methods. This research paper explores the design, implementation, and evaluation of a hand gesture recognition system for PC interaction. We review existing methods of PC control and discuss the limitations of traditional input modalities. The paper outlines the process of hand gesture recognition, including data acquisition, preprocessing, feature extraction, and classification. We describe the design considerations and implementation details of the gesture recognition system, highlighting the hardware and software requirements. Empirical evaluations are presented to assess the system's performance in terms of recognition accuracy, response time, and user satisfaction. Furthermore, we explore potential applications of hand gesture control across various domains and discuss challenges and future directions in the field. This research contributes to advancing the state-of-the-art in HCI by demonstrating the feasibility and effectiveness of hand gestures for PC interaction

OBT-Trace: An approach to trace and recognition object motion through prosthetic hand gestures

The development of prosthetic hands has advanced significantly in recent years, aiming to provide more intuitive and functional solutions for individuals with upper limb amputations. This research presents a novel approach to prosthetic hand control by integrating 3D hand gesture recognition with object manipulation and recognition capabilities. Our proposed system utilizes a pre-trained object recognition model, based on transfer learning, to enable the prosthetic hand to perceive and identify objects in its vicinity. The model leverages a vast dataset of objects, enabling the prosthetic hand to recognize a wide array of everyday items, thus enhancing its versatility.In addition, the prosthetic hand incorporates a sophisticated 3D hand gesture recognition system, allowing users to control the hand's movements and actions seamlessly. By recognizing specific gestures, such as grasping, lifting, and releasing, users can intuitively interact with their environment and perform various tasks with ease. This research leverages the synergy between gesture recognition and object recognition, creating a powerful framework for prosthetic hand control. The system's adaptability and versatility make it suitable for a broad range of applications, from assisting with daily tasks to enhancing the quality of life for individuals with upper limb amputations.The results of this study demonstrate the feasibility and effectiveness of combining 3D hand gesture recognition with pre-trained object recognition through transfer learning. This approach opens up new possibilities for enhancing prosthetic hand functionality and usability, ultimately improving the lives of those who rely on these devices for daily living. The proposed model combines the features of YOLO V7 object detection with pre-trained models. The proposed model achieves 99.8% of accuracy compared to the existing models.

End-to-End Ultrasonic Hand Gesture Recognition.

As the number of electronic gadgets in our daily lives is increasing and most of them require some kind of human interaction, this demands innovative, convenient input methods. There are limitations to state-of-the-art (SotA) ultrasound-based hand gesture recognition (HGR) systems in terms of robustness and accuracy. This research presents a novel machine learning (ML)-based end-to-end solution for hand gesture recognition with low-cost micro-electromechanical (MEMS) system ultrasonic transducers. In contrast to prior methods, our ML model processes the raw echo samples directly instead of using pre-processed data. Consequently, the processing flow presented in this work leaves it to the ML model to extract the important information from the echo data. The success of this approach is demonstrated as follows. Four MEMS ultrasonic transducers are placed in three different geometrical arrangements. For each arrangement, different types of ML models are optimized and benchmarked on datasets acquired with the presented custom hardware (HW): convolutional neural networks (CNNs), gated recurrent units (GRUs), long short-term memory (LSTM), vision transformer (ViT), and cross-attention multi-scale vision transformer (CrossViT). The three last-mentioned ML models reached more than 88% accuracy. The most important innovation described in this research paper is that we were able to demonstrate that little pre-processing is necessary to obtain high accuracy in ultrasonic HGR for several arrangements of cost-effective and low-power MEMS ultrasonic transducer arrays. Even the computationally intensive Fourier transform can be omitted. The presented approach is further compared to HGR systems using other sensor types such as vision, WiFi, radar, and state-of-the-art ultrasound-based HGR systems. Direct processing of the sensor signals by a compact model makes ultrasonic hand gesture recognition a true low-cost and power-efficient input method.

Hand Gesture Recognition for Video Player

Hand Gesture Recognition (HGR) is an emerging technology that has numerous applications in various fields, including human-computer interaction and robotics. In this project, we developed a Hand Gesture Recognition system for video players that recognize hand gestures and perform actions such as play, pause, rewind, and fast-forward. The system was developed using Python programming language and the OpenCV, mediapipe, pyautogui libraries. A dataset of hand gestures was created by recording videos using a webcam and annotating the frames with corresponding labels. The Hand Gesture Recognition system achieved an accuracy of 92% on the test set and was able to accurately recognize hand gestures and perform the corresponding actions on a video player. The system has the potential to be used as a novel way to control video players, especially in situations where the user cannot use a mouse or keyboard. In conclusion, the Hand Gesture Recognition system developed in this project provides a promising solution for controlling video players using hand gestures. The system achieved a high level of accuracy in recognizing gestures and performing actions, and can potentially be used in a variety of applications. Further improvements and refinements can be made to the system in the future to make it even more effective and user- friendly.

A Review on Sign Language Recognition and Learning System

This paper presents innovative approaches to various aspects of sign language recognition and learning, catering to the diverse needs within this field. It introduces a system utilizing 2D image sampling and concatenation, trained with convolutional neural networks, achieving accurate and robust sign recognition even with low-cost cameras. Another system, SignQuiz, offers a cost-effective web-based solution for learning finger-spelled signs in Indian Sign Language (ISL), outperforming traditional printed mediums. Additionally, a dynamic hand gesture recognition system employing deep learning architectures demonstrates superior performance over existing methods, addressing challenges such as hand segmentation and sequence modeling. Furthermore, an efficient deep convolutional neural network approach for hand gesture recognition, including transfer learning, achieves high recognition rates across various datasets. Finally, a robust model for static sign recognition in ISL, utilizing CNNs and evaluating on a diverse dataset, attains exceptional accuracy and outperforms previous works. These contributions collectively advance the field of sign language recognition and learning, offering solutions that are accurate, cost-effective, and efficient, thereby facilitating better communication and interaction for the hearing-impaired community. Key Words: CNN, Sign language recognition, Hand gesture recognition, Machine learning, Transfer Learning, SVM.

Hand Gesture Gaming

Hand gesture recognition has emerged as a promising interface for interactive gaming applications, offering intuitive and immersive experiences. This research paper explores the development and implementation of a hand gesture recognition system using Python for gaming purposes. The study investigates various machine learning techniques, particularly convolutional neural networks (CNNs), for accurate detection and classification of hand gestures from video input. The research methodology involves collecting a diverse dataset of hand gesture samples, encompassing common gaming gestures such as swipes, taps, and rotations. Preprocessing techniques are employed to enhance the quality and consistency of the dataset. Subsequently, deep learning architectures, including CNNs, are trained on the dataset to recognize and classify hand gestures in real-time.

Gaming using different hand gestures using artificial neural network

INTRODUCTION: Gaming has evolved over the years, and one of the exciting developments in the industry is the integration of hand gesture recognition.
 OBJECTIVES: This paper proposes gaming using different hand gestures using Artificial Neural Networks which allows players to interact with games using natural hand movements, providing a more immersive and intuitive gaming experience.
 METHODS: Introduces two modules: recognition and analysis of gestures. The gesture recognition module identifies the gestures, and the analysis module assesses them to execute game controls based on the calculated analysis.
 RESULTS: The main results obtained in this paper are enhanced accessibility, higher accuracy and improved performance.
 CONCLUSION: To communicate with any of the traditional systems, physical contact is necessary. In the hand gesture recognition system, the same functionality can be interpreted by gestures without requiring physical contact with the interfaced devices.

HGR-FYOLO: a robust hand gesture recognition system for the normal and physically impaired person using frozen YOLOv5

HGR-FYOLO: a robust hand gesture recognition system for the normal and physically impaired person using frozen YOLOv5

Adaptive Trajectories’ Constant False Alarm Rate Mirror Filters and Elevation Angle Evaluation for Multiple-Input Multiple-Output Radar-Based Hand Gesture Recognition

Gesture recognition technology has been quickly developed in the field of human–computer interaction. The multiple-input multiple-output (MIMO) radar is popular in gesture recognition because of its notable spatial resolution. This work proposes a MIMO radar-based hand gesture recognition algorithm with low complexity. We leverage low-complexity adaptive signal processing to extract trajectory information and minimize noise to create a system that can be applied in real-world applications with small training datasets. First, a spectrum analysis is utilized on range-Doppler maps (RDMs), and a cell-averaging constant false alarm rate (CA-CFAR) with mirror filters is applied to improve the robustness of noise. Then, the features related to the distance, speed, direction, and elevation angle of the moving object are determined using the proposed adaptive signal analysis techniques. For classification, the random forest algorithm is implemented. The proposed system can precisely distinguish and identify eight gestures, including waving, moving to the left or right, patting, pushing, pulling, and rotating clockwise or anti-clockwise, with an accuracy of 95%. Experiments demonstrate the capability of the proposed hand gesture recognition system to classify different movements precisely.

High-Accuracy Hand Gesture Recognition on the Wrist Tendon Group Using Pneumatic Mechanomyography (pMMG)

Hand gesture recognition has received considerable attention as an intuitive interaction method in recent years. This research introduces a new wearable hand gesture recognition system that employs pneumatic mechanomyography (pMMG) to directly monitor the wrist tendon group, which transmits muscle force to the fingers. The experimental findings demonstrate that the proposed method provides raw observations proportional to the finger flexion force, with high R-squared values exceeding 0.94. The performance of the proposed system was evaluated by conducting a hand gesture experiment consisting of 28 hand gestures. The proposed method achieved an average accuracy of 98.12%, surpassing the surface electromyography (sEMG) system's accuracy of 93.89%. Furthermore, the fusion of pMMG and sEMG sensors yielded an accuracy of 99.18%. The results suggest that the proposed approach exhibits the potential to enhance the accuracy and efficiency of hand gesture recognition systems.

Controlling Media Player with Hands: A Transformer Approach and a Quality of Experience Assessment

In this article, we propose a Hand Gesture Recognition (HGR) system based on a novel deep transformer (DT) neural network for media player control. The extracted hand skeleton features are processed by separate transformers for each finger in isolation to better identify the finger characteristics to drive the following classification. The achieved HGR accuracy (0.853) outperforms state-of-the-art HGR approaches when tested on the popular NVIDIA dataset. Moreover, we conducted a subjective assessment involving 30 people to evaluate the Quality of Experience (QoE) provided by the proposed DT-HGR for controlling a media player application compared with two traditional input devices, i.e., mouse and keyboard. The assessment participants were asked to evaluate objective (accuracy) and subjective (physical fatigue, usability, pragmatic quality, and hedonic quality) measurements. We found that (i) the accuracy of DT-HGR is very high (91.67%), only slightly lower than that of traditional alternative interaction modalities; and that (ii) the perceived quality for DT-HGR in terms of satisfaction, comfort, and interactivity is very high, with an average Mean Opinion Score (MOS) value as high as 4.4, whereas the alternative approaches did not reach 3.8, which encourages a more pervasive adoption of the natural gesture interaction.

Hand Gesture Recognition Using FSK Radar Sensors.

Hand gesture recognition, which is one of the fields of human-computer interaction (HCI) research, extracts the user's pattern using sensors. Radio detection and ranging (RADAR) sensors are robust under severe environments and convenient to use for hand gestures. The existing studies mostly adopted continuous-wave (CW) radar, which only shows a good performance at a fixed distance, which is due to its limitation of not seeing the distance. This paper proposes a hand gesture recognition system that utilizes frequency-shift keying (FSK) radar, allowing for a recognition method that can work at the various distances between a radar sensor and a user. The proposed system adopts a convolutional neural network (CNN) model for the recognition. From the experimental results, the proposed recognition system covers the range from 30 cm to 180 cm and shows an accuracy of 93.67% over the entire range.

Leveraging Self-Attention Mechanism for Deep Learning in Hand-Gesture Recognition System

This research addresses the complex challenge of recognizing hand gestures irrespective of the user's body posture, a crucial issue in medical treatment for people with speech impairments and human-machine interfaces where precise gesture interpretation is vital. The aim is to engineer an advanced hand gesture recognition system, effective across various body positions and camera viewpoints. A novel flexible camera arrangement was employed, integrating a CNN-Transformer hybrid model, leveraging the strengths of Convolutional Neural Networks and the self-attention mechanism of Transformers. Developed using Python and the PyTorch deep learning framework, the system focuses on sophisticated image processing techniques. A thorough literature review on gesture recognition systems and multi-view analysis was conducted to inform the development. The system demonstrated exceptional accuracy in recognizing hand gestures in diverse body postures and from multiple camera perspectives, significantly outperforming existing methods. It marked a significant advancement in decoding complex gestures, a key aspect for medical applications and intricate human-machine is primarily interactions. This beneficial for people with speech impairments, rehabilitation, and in human-machine interfaces, poised to revolutionize patient care and enhance interaction with advanced machinery and computer systems.