Gesture Recognition Technology Research Articles

IoT Based Gesture Control Light System Using Python

The Gesture Control Light System is an innovative, user-friendly interface designed to enhance the interaction between humans and electronic devices. By utilizing hand gestures as input commands, this system enables users to control lighting appliances without physical contact. It integrates gesture recognition technology, primarily using infrared sensors or cameras, to interpret specific hand movements like swipes or waves, translating them into actions such as turning lights on/off, adjusting brightness, or changing colour modes. This touchless approach offers convenience and hygiene benefits, especially in settings where hands-free control is advantageous, such as in smart homes, medical facilities, or during cooking tasks. The system's intuitive design also contributes to energy efficiency, enabling automation and personalized lighting preferences. With its practical applications and potential to improve user experience, the Gesture Control Light System represents a step forward in the evolution of smart technology and human-machine interaction.

IOT Gesture Recognition-Based Virtual Keyboard and Mouse

Recognition-based virtual mouse and keyboard systems use gesture recognition technology to control mouse movements and keyboard inputs without physical devices. Instead of using a physical mouse or keyboard, users can perform hand or finger gestures in the air, which are captured and interpreted by a camera or sensor system. Gesture abstracts in this context refer to the specific hand or finger movements that are recognized and associated with certain mouse or keyboard actions. These gestures are predefined and programmed into the recognition system, allowing users to perform specific movements to simulate the actions of a traditional mouse or keyboard

Research on gesture recognition technology based on machine learning

Abstract. Hand Gesture Recognition (HGR), as a significant technological advancement in the field of Human-Computer Interaction (HCI), aims to develop systems capable of accurately recognizing and interpreting human gestures for a diverse range of applications, including device control, virtual reality, gesture passwords, and gesture interaction. With the continuous advancement of machine learning algorithms, especially deep learning techniques, machine learning-based gesture recognition has garnered widespread attention. This paper presents a review of the development of gesture recognition techniques from traditional approaches to the current mainstream deep learning-based methods, and outlines the challenges and technical difficulties encountered. It analyzes several of the most popular classification techniques, including Naive Bayes, K-Nearest Neighbors (KNN), Random Forests, XGBoost, Support Vector Classifiers (SVCs), and Convolutional Neural Networks (CNNs). Furthermore, this paper examines the application of these algorithms in both dynamic and static gesture recognition and compares their performance and suitability in different scenarios. The results demonstrate that the accuracy and robustness of gesture recognition systems can be markedly enhanced through the prudent selection and optimization of the algorithms, which serves as a valuable reference for future research and applications.

Three-dimensional dynamic gesture recognition method based on convolutional neural network

With the rapid advancement of virtual reality, dynamic gesture recognition technology has become an indispensable and critical technique for users to achieve human–computer interaction in virtual environments. The recognition of dynamic gestures is a challenging task due to the high degree of freedom and the influence of individual differences and the change of gesture space. To solve the problem of low recognition accuracy of existing networks, an improved dynamic gesture recognition algorithm based on ResNeXt architecture is proposed. The algorithm employs three-dimensional convolution techniques to effectively capture the spatiotemporal features intrinsic to dynamic gestures. Additionally, to enhance the model’s focus and improve its accuracy in identifying dynamic gestures, a lightweight convolutional attention mechanism is introduced. This mechanism not only augments the model’s precision but also facilitates faster convergence during the training phase. In order to further optimize the performance of the model, a deep attention submodule is added to the convolutional attention mechanism module to strengthen the network’s capability in temporal feature extraction. Empirical evaluations on EgoGesture and NvGesture datasets show that the accuracy of the proposed model in dynamic gesture recognition reaches 95.03% and 86.21%, respectively. When operating in RGB mode, the accuracy reached 93.49% and 80.22%, respectively. These results underscore the effectiveness of the proposed algorithm in recognizing dynamic gestures with high accuracy, showcasing its potential for applications in advanced human–computer interaction systems.

Enhancement algorithm for surface electromyographic-based gesture recognition based on real-time fusion of muscle fatigue features

This study aims to optimize surface electromyography-based gesture recognition technique, focusing on the impact of muscle fatigue on the recognition performance. An innovative real-time analysis algorithm is proposed in the paper, which can extract muscle fatigue features in real time and fuse them into the hand gesture recognition process. Based on self-collected data, this paper applies algorithms such as convolutional neural networks and long short-term memory networks to provide an in-depth analysis of the feature extraction method of muscle fatigue, and compares the impact of muscle fatigue features on the performance of surface electromyography-based gesture recognition tasks. The results show that by fusing the muscle fatigue features in real time, the algorithm proposed in this paper improves the accuracy of hand gesture recognition at different fatigue levels, and the average recognition accuracy for different subjects is also improved. In summary, the algorithm in this paper not only improves the adaptability and robustness of the hand gesture recognition system, but its research process can also provide new insights into the development of gesture recognition technology in the field of biomedical engineering.

Hand Gesture Recognition Using Ultrasonic Array with Machine Learning

In the field of gesture recognition technology, accurately detecting human gestures is crucial. In this research, ultrasonic transducers were utilized for gesture recognition. Due to the wide beamwidth of ultrasonic transducers, it is difficult to effectively distinguish between multiple objects within a single beam. However, they are effective at accurately identifying individual objects. To leverage this characteristic of the ultrasonic transducer as an advantage, this research involved constructing an ultrasonic array. This array was created by arranging eight transmitting transducers in a circular formation and placing a single receiving transducer at the center. Through this, a wide beam area was formed extensively, enabling the measurement of unrestricted movement of a single hand in the X, Y, and Z axes. Hand gesture data were collected at distances of 10 cm, 30 cm, 50 cm, 70 cm, and 90 cm from the array. The collected data were trained and tested using a customized Convolutional Neural Network (CNN) model, demonstrating high accuracy on raw data, which is most suitable for immediate interaction with computers. The proposed system achieved over 98% accuracy.

High-precision dynamic gesture recognition based on microfiber sensor and EMT-Net

Dynamic gesture recognition, which utilizes flexible wearable sensors and deep learning, is invaluable for human–computer interaction. Nevertheless, the primary challenges persist are the rapid detection of intricate gestures and the accurate recognition of dynamic signals. In this study, we suggest utilizing a microfiber sensor to identify the variations in wrist skin and detect dynamic gesture. In order to tackle the issue of insufficient feature extraction in the detected signals, resulting in reduced accuracy in recognition, we introduce a network dubbed EMT-Net (improve multi-head attention transformer network). This network utilizes a transformer encoder to capture and represent the characteristics of dynamic gesture signal and uses a CNN to classify the encoded features. To ensure that the model comprehensively captures the temporal and statistical characteristics of the signals, we enhance the multi-head attention mechanism by restricting certain attention heads to concentrate solely on the statistical features of the signals while allowing others to focus on the temporal features and global dependencies. Furthermore, because of the varying discriminatory abilities of different characteristics, we have developed an attention module to redistribution the attention weights on statistical features. The experimental results demonstrate that the microfiber sensor effectively recognizes ten distinct forms of dynamic gesture signals. Simultaneously, EMT-Net attains proficient identification with an accuracy of 98.80%, precision of 98.81%, recall of 98.80%, and an F1 score of 98.80%. The application value of this dynamic gesture recognition technology, which utilizes microfiber sensors and EMT-Net, is significant. The forthcoming alterations in human–computer interaction, virtual reality, and various other domains are anticipated.

Human Muscle sEMG Signal and Gesture Recognition Technology Based on Multi-Stream Feature Fusion Network

Surface electromyography signals have significant value in gesture recognition due to their ability to reflect muscle activity in real time. However, existing gesture recognition technologies have not fully utilized surface electromyography signals, resulting in unsatisfactory recognition results. To this end, firstly, a Butterworth filter was adopted to remove high-frequency noise from the signal. A combined method of moving translation threshold was introduced to extract effective signals. Then, a gesture recognition model based on multi-stream feature fusion network was constructed. Feature extraction and fusion were carried out through multiple parallel feature extraction paths, combined with convolutional neural networks and residual attention mechanisms. Compared to popular methods of the same type, this new recognition method had the highest recognition accuracy of 92.1% and the lowest recognition error of 5%. Its recognition time for a single-gesture image was as short as 4s, with a maximum Kappa coefficient of 0.92. Therefore, this method combining multi-stream feature fusion networks can effectively improve the recognition accuracy and robustness of gestures and has high practical value.

Student Evaluation Model Based on Emotion Recognition through Classroom Monitoring

The improvement and perfection of higher education work relies on correct feedback from student evaluation of teaching, however, problems such as strong subjectivity and generalization of the indicator system are common in college student evaluation. Therefore, this article designs a student evaluation model based on emotion recognition through classroom monitoring. First, use classroom surveillance to collect class videos of teachers and students. After that the emotional information of teachers and students in class is captured through emotion recognition technology by calling Baidu AI Open Platform API and gesture recognition technology through improved you only watch once (YOWO) algorithm model, then entering into the database for statistics. Finally, statistical data is used to analyze teaching methods and teaching attitudes such as content mastery and enthusiasm, as well as learning effects such as student initiative and knowledge mastery, and multiple indicators of student evaluation are presented on a web page in the form of bar charts, pie charts, etc through plotting library. As a result, this model improves the objectivity and reliability of the evaluation results and provides data support for the quantification of the indicator system, which makes the evaluation results more intuitive and easy to understand.

Assisting autonomous precision medical rehabilitation through gesture recognition devices

With the aging of the population and the increase of various chronic diseases, there is a growing demand for effective and accessible medical rehab solutions. In order to overcome the above problems, this project innovatively adopts the gesture recognition technology to design and fabricate the realization of an autonomous accurate medical rehab gesture recognition assistance system. In terms of hardware, this paper uses Arduino as the main controller, Paj7620 as the gesture recognition device, and OLED 0.96 display as the system display. In terms of software, this paper utilizes Arduino IDE software to successfully write the code and realize all the functions. In terms of combining with actual medical rehab needs, the team chose elbow fracture rehab as a typical type of rehab, and designed a three-device obtuse triangle placement scheme for precise monitoring of rehab training. Initial simulation test results show that over 70% of participants reported an improvement in their rehabilitation outcomes, and about 90% found the system easy to use. The results of this project will directly benefit patients requiring long-term rehabilitation.

Synergistic Integration of Self-Powered, Flexible Wearables with Advanced Gesture Recognition for Deaf-Mute Communication

This research confronts two pivotal challenges in advancing assistive communication devices for the deaf and mute community: enhancing the longevity of device operation through self-powered sensor technology and improving gesture recognition accuracy with sophisticated algorithms. The study introduces a novel design for flexible, wearable devices that harness kinetic and thermal energy, converting them into electrical energy to ensure continuous operation. This approach eliminates the need for frequent recharging, thereby increasing the autonomy of these assistive devices. In addressing the second challenge, the research refines deep learning algorithms to significantly boost the precision and accuracy of gesture recognition. These algorithms are trained on an extensive dataset that includes a broad spectrum of sign language gestures, captured by sensors on the wearable device. This methodology not only promises a breakthrough in the self-sufficiency of wearable assistive devices but also improves the effectiveness and usability of sign language to speech translation. Such advancements facilitate more seamless, barrier-free communication for individuals with hearing and speech impairments. Preliminary results demonstrate the potential of integrating self-powering capabilities with advanced gesture recognition technology, marking a considerable advancement in assistive technology.

Hand Gesture Recognition System using Deep Learning

This paper presents a comprehensive exploration of a hand gesture recognition system centered on the YOLOv5 object detection algorithm. The system is meticulously designed to fulfill specific objectives, including real-time gesture detection, high accuracy, scalability, robustness to environmental variability, and seamless integration with other system components. Drawing upon the strengths of YOLOv5, the proposed system aims to provide a reliable and efficient solution for hand gesture recognition, thereby enhancing human- computer interaction and enabling various applications across different domains. The methodology encompasses several stages, including data acquisition, preprocessing, object detection using YOLOv5, integration with other system modules, evaluation, and validation. Through a series of experiments and extensive testing in real-world scenarios, the effectiveness and usability of the system are thoroughly assessed, demonstrating its potential for practical deployment and widespread adoption. Additionally, this paper discusses the implications of the findings, identifies areas for further research and development, and offers insights into the future directions of hand gesture recognition systems utilizing YOLOv5. Overall, this study contributes to advancing the field of human-computer interaction and gesture recognition technology, paving the way for innovative applications and enhanced user experiences in various domains. Keywords— Hand Gesture Recognition, YOLOv5, Real-time Detection, Human-Computer Interaction, Object Detection, Machine Learning

Research on Intelligent Wheelchair Multimode Human–Computer Interaction and Assisted Driving Technology

The traditional wheelchair focuses on the “human-chair” motor function interaction to ensure the elderly and people with disabilities’ basic travel. For people with visual, hearing, physical disabilities, etc., the current wheelchairs show shortcomings in terms of accessibility and independent travel for this group. Therefore, this paper develops an intelligent wheelchair with multimodal human–computer interaction and autonomous navigation technology. Firstly, it researches the multimodal human–computer interaction technology of occupant gesture recognition, speech recognition, and head posture recognition and proposes a wheelchair control method of three-dimensional head posture mapping the two-dimensional plane. After testing, the average accuracy of the gesture, head posture and voice control modes of the motorized wheelchair proposed in this study reaches more than 95 percent. Secondly, the LiDAR-based smart wheelchair indoor autonomous navigation technology is investigated to realize the autonomous navigation of the wheelchair by constructing an environment map, using A* and DWA algorithms for global and local path planning, and adaptive Monte Carlo simulation algorithms for real-time localization. Experiments show that the position error of the wheelchair is within 10 cm, and the heading angle error is less than 5° during the autonomous navigation. The multimode human–computer interaction and assisted driving technology proposed in this study can partially compensate and replace the functional deficiencies of the disabled population and improve the quality of life of the elderly and disabled population.

Implementation of Gesture-Based Virtual Keyboard and Mouse

The development of human-computer interaction has seen a major change in the last few years, moving in the direction of more user-friendly and effective interfaces. This tendency has prompted researchers to investigate new input techniques, especially in the field of gesture-based control systems. By utilizing both hand and eye gestures to create a gesture-based virtual mouse and keyboard, this paper introduces a novel method of human-computer interaction. The suggested system offers users a smooth and simple way to interact with computers by combining hand gesture recognition and eye-tracking technology. Users can control cursor movement and keyboard inputs without the use of physical peripherals by using the system to track their hand and eye movements and interpret them as commands. The incorporation of eye-tracking technology improves the system's accuracy and responsiveness, enabling more precise cursor control and interaction with elements displayed on the screen. Hand gesture recognition, on the other hand, gives users an ergonomic and natural way to input data, allowing them to carry out commands with straightforward motions. The architecture of the system combines real-time processing capabilities to guarantee low latency and high responsiveness with sophisticated machine-learning algorithms for gesture recognition. Additionally, users can define their gesture commands according to their needs and preferences because the system is flexible and customizable. The suggested gesture-based virtual mouse and keyboard system has undergone rigorous testing and evaluation, and the results show promising performance and usability in a range of scenarios and applications. This novel method of interacting with computers has the potential to improve user experience, accessibility, and productivity in a variety of computing contexts. Keywords: Human-Computer Interaction, Gesture Recognition, Eye-Tracking

IoT-based Smart Screen Control Using Hand Gesture

In today's dynamic educational environment, the demand for innovative tools to enhance teaching methodologies is paramount. This research paper proposes a novel IoT-based smart screen control system tailored explicitly for educational settings [10]. At its core lies hand gesture recognition technology, powered by the MediaPipe library, facilitating seamless real-time detection and tracking of hand gestures for intuitive presentation control. Integrating camera sensors and microcontrollers [14], the system ensures precise data transmission to computers interfacing with smart boards. An advanced algorithm synchronizes hand feed and movement flawlessly [14], offering users a glitch-free experience. The user-friendly interface empowers educators to customize settings and gestures, fostering personalized interactions. Furthermore, the system enables control over various types of presentations and slides, as well as zooming functionality within any opened interface [4]. Through a comprehensive literature review and implementation of state-of-the-art methodologies [12], this research revolutionizes human-computer interaction in educational environments. By adhering to stringent citation guidelines and considering the quality and relevance of references, this paper contributes to advancing knowledge in the field [1, 2, 4, 10, 12, 14]. The proposed system sets a new standard for efficiency and effectiveness in educational settings, promising to reshape the teaching landscape and inspire creativity in classrooms. Key Words: Media Pipe, IoT, camera sensors, microcontrollers, presentation control, customization, educational environments, human-computer interaction, OpenCV, Python.

Driving Aid for Rotator Cuff Injured Patients using Hand Gesture Recognition

Gesture recognition is a way for computers to understand how humans move and express themselves without using traditional methods like typing or clicking. Instead of relying on text or graphics, gesture recognition focuses on reading body movements, such as those made by the hands or face. Currently, there is a specific interest in recognizing hand gestures by analyzing the veins on the back of the hand. Scientists have found that each person has a unique arrangement of veins beneath the skin of their hand. When the hand moves, the position of these veins changes, and this change is considered a gesture. These gestures are then translated into specific actions or tasks by coding the hand movements. This technology is particularly helpful for individuals with rotator cuff injuries. The rotator cuff is a group of muscles and tendons in the shoulder that can get injured, causing pain and limiting movement. People with these injuries may have difficulty steering a car, especially if their job or sport involves repetitive overhead motions. With gesture recognition technology, a person can control the car by simply moving their wrist, eliminating the need to use the shoulder. In summary, gesture recognition technology reads the unique patterns of hand veins to interpret hand movements, making it a practical solution for individuals with rotator cuff injuries who may struggle with certain tasks, like steering a car.

Computer Vision Gestures Recognition System Using Centralized Cloud Server

Gesture recognition is a progressive technology that enables communication between humans and computers without any physical interaction. In recent years, gesture recognition technology has provided remarkable advancements in the technology world. A computer vision cloud-based gesture recognition system represents the potential of cloud computing to process and analyze the collected data. This paper examines the various growing aspects of gesture recognition using computer vision and cloud technology. Here we introduce a computer vision-based gesture processing unit that works on a cloud server and is used to enable real-time interaction between the physical world and the virtual server. It will help to use high-end gesture processing technology for every kind of device user. All the data will be collected using vision- based devices and monitored and processed on the centralized server. So the user doesn't have to be a highly efficient device. While using this system, the user can experience both the recent cutting-edge cloud infrastructure as well as the recent gesture recognition system. Not only humans but also object gestures can be recognized by using this proposed system. So this paper aims to represent a significant step toward a cloud-based interaction between human gestures and digital devices.

A Hybrid Image Augmentation Technique for User- and Environment-Independent Hand Gesture Recognition Based on Deep Learning

This research stems from the increasing use of hand gestures in various applications, such as sign language recognition to electronic device control. The focus is the importance of accuracy and robustness in recognizing hand gestures to avoid misinterpretation and instruction errors. However, many experiments on hand gesture recognition are conducted in limited laboratory environments, which do not fully reflect the everyday use of hand gestures. Therefore, the importance of an ideal background in hand gesture recognition, involving only the signer without any distracting background, is highlighted. In the real world, the use of hand gestures involves various unique environmental conditions, including differences in background colors, varying lighting conditions, and different hand gesture positions. However, the datasets available to train hand gesture recognition models often lack sufficient variability, thereby hindering the development of accurate and adaptable systems. This research aims to develop a robust hand gesture recognition model capable of operating effectively in diverse real-world environments. By leveraging deep learning-based image augmentation techniques, the study seeks to enhance the accuracy of hand gesture recognition by simulating various environmental conditions. Through data duplication and augmentation methods, including background, geometric, and lighting adjustments, the diversity of the primary dataset is expanded to improve the effectiveness of model training. It is important to note that the utilization of the green screen technique, combined with geometric and lighting augmentation, significantly contributes to the model’s ability to recognize hand gestures accurately. The research results show a significant improvement in accuracy, especially with implementing the proposed green screen technique, underscoring its effectiveness in adapting to various environmental contexts. Additionally, the study emphasizes the importance of adjusting augmentation techniques to the dataset’s characteristics for optimal performance. These findings provide valuable insights into the practical application of hand gesture recognition technology and pave the way for further research in tailoring techniques to datasets with varying complexities and environmental variations.

Research on Dynamic Gesture Recognition and Control System based on Machine Vision

Hand gesture recognition and control is a new type of human-computer interaction that can provide a more convenient and efficient operation mode by utilizing non-contact gesture recognition technology. This paper presents a lightweight dynamic gesture recognition method for intelligent office presentation control. First, we introduce the concept of hand gesture recognition and go over key gesture recognition technologies like classification. The structure, process, and evaluation index of the gesture recognition algorithm are described in detail using a convolutional neural network model. During the experiment's algorithm verification phase, we test and analyze the algorithm using the Python language, compilation environment, and data set. In the control experiment, we evaluated the system's ability to control the office application's start, play, next, previous, and exit functions. We achieve 96.3% accuracy on the test set. Experimental results show that the system can recognize a wide range of hand gestures and accurately control the presentation.

AirMouse and AirWrite Intuitive Digital Interaction

The primary purpose of this project is to redefine the way people will interact with their systems more easily, and this will be achieved by eliminating the touchpads and keyboards to connect to the computer. This cutting-edge system eliminates the need for a regular mouse or a keyboard, allowing for easier device user-interface adaptation. The implementation of the latest gesture recognition technology lets the system not only pick up and decipher the movements of users' hands but also translate them into wider operations that are similar to those of a mouse and keyboard. Central to the design of the metaphoric system is the "virtual mouse," which enables the cursor movement and the performance of various mouse buttons without touching anything physically. So, it is a virtual mouse with an on-the-screen cursor that reacts to hand gestures in real-time. The system utilizes certain hand gestures enough to enable the effortless execution of actions commonly associated with a traditional mouse including left-clicking, right-clicking, and dragging across the screen. The virtual mouse feature is enhanced by the air writing technique, commonly acknowledged as a milestone in input development. This capability precisely reads and decodes the movements developed in the space by the fingertips while conveying them to the screen as written words. The air writing system would predict the characters from the user's gestures, creating a fresh, secure, and user-friendly method that favors the option of air typing rather than relying on a physical keyboard. This project brings innovation in human-computer interaction when implemented using powerful computer vision and machine learning techniques and promotes accessibility and usability of the system where the traditional inputs become inapplicable. On the other hand, its feasible justifications range from medical uses to pioneering in virtual reality, gaming, and education that drives brilliantly the new options in interaction across various fields.