Gesture Recognition Algorithm Research Articles

Implementation of Gesture-Based Virtual Keyboard and Mouse

The development of human-computer interaction has seen a major change in the last few years, moving in the direction of more user-friendly and effective interfaces. This tendency has prompted researchers to investigate new input techniques, especially in the field of gesture-based control systems. By utilizing both hand and eye gestures to create a gesture-based virtual mouse and keyboard, this paper introduces a novel method of human-computer interaction. The suggested system offers users a smooth and simple way to interact with computers by combining hand gesture recognition and eye-tracking technology. Users can control cursor movement and keyboard inputs without the use of physical peripherals by using the system to track their hand and eye movements and interpret them as commands. The incorporation of eye-tracking technology improves the system's accuracy and responsiveness, enabling more precise cursor control and interaction with elements displayed on the screen. Hand gesture recognition, on the other hand, gives users an ergonomic and natural way to input data, allowing them to carry out commands with straightforward motions. The architecture of the system combines real-time processing capabilities to guarantee low latency and high responsiveness with sophisticated machine-learning algorithms for gesture recognition. Additionally, users can define their gesture commands according to their needs and preferences because the system is flexible and customizable. The suggested gesture-based virtual mouse and keyboard system has undergone rigorous testing and evaluation, and the results show promising performance and usability in a range of scenarios and applications. This novel method of interacting with computers has the potential to improve user experience, accessibility, and productivity in a variety of computing contexts. Keywords: Human-Computer Interaction, Gesture Recognition, Eye-Tracking

Computer Interactive Gesture Recognition Model Based on Improved YOLOv5 Algorithm

The expansion of virtual and augmented reality, intelligent assistance technology, and other fields has led to an increased demand for human–computer interaction methods that are more natural and intuitive. Gesture recognition has become an important research direction. Traditional gesture recognition methods are mainly based on image processing and pattern recognition techniques. However, there are certain challenges to its accuracy and robustness in complex backgrounds. In addition, the temporal correlation and spatial information in gesture sequences have not been fully utilized, which limits the performance of gesture recognition systems. In response to the above issues, this study first utilizes the Ghost module for feature extraction based on the You Only Look Once version 5 (YOLOv5) algorithm. Then drawing inspiration from the idea of densely connected networks, feature map stitching is carried out, and a human–machine interactive gesture recognition algorithm is designed by combining it with a hybrid attention mechanism. The experimental results showed that the average accuracy of the algorithm tended to converge after 160 iterations, and the final MAP value converged to 92.19%. Compared to the standard YOLOv5 algorithm, its iteration speed had been improved by 12.5%, and the MAP value had been improved by 4.63%. The designed human–computer interaction gesture recognition algorithm has higher accuracy and smaller error, and has certain application potential in the field of machine vision.

Controlling 3D Models in VR using Hand Gestures

Abstract: The fusion of hand gesture detection technology with virtual reality (VR) environments enables users to interact organically with three-dimensional models. Utilizing VR headsets equipped with advanced hand tracking algorithms, individuals can engage with virtual objects through intuitive gestures, enriching their immersive experiences. Software development entails carrying out sophisticated gesture recognition algorithms and the assignment of specific actions to these gestures, such as grasping, rotating, and resizing 3D models. These interactions mirror real-world manipulations, empowering users to seamlessly explore and modify digital content. Furthermore, gestural menu systems streamline user interactions by offering convenient access to additional features. Rigorous testing and optimization processes ensure precision, responsiveness, and optimal performance, enhancing the overall user satisfaction.

Research on Dynamic Gesture Recognition and Control System based on Machine Vision

Hand gesture recognition and control is a new type of human-computer interaction that can provide a more convenient and efficient operation mode by utilizing non-contact gesture recognition technology. This paper presents a lightweight dynamic gesture recognition method for intelligent office presentation control. First, we introduce the concept of hand gesture recognition and go over key gesture recognition technologies like classification. The structure, process, and evaluation index of the gesture recognition algorithm are described in detail using a convolutional neural network model. During the experiment's algorithm verification phase, we test and analyze the algorithm using the Python language, compilation environment, and data set. In the control experiment, we evaluated the system's ability to control the office application's start, play, next, previous, and exit functions. We achieve 96.3% accuracy on the test set. Experimental results show that the system can recognize a wide range of hand gestures and accurately control the presentation.

DMS-yolov8 A gesture recognition algorithm based on improved yolov81

Aiming at the problems of slow speed and low accuracy of traditional neural network systems for real-time gesture recognition in complex backgrounds., this paper proposes DMS-yolov8-a gesture recognition method to improve yolov8. This algorithm replaces the Bottleneck convolution module in the backbone network of yolov8 with variable row convolution DCNV2, and increases the feature convolution range without increasing the computation amount through a more flexible feeling field. in addition, the self-developed MPCA attention module is added after the feature output layer of the backbone layer, which improves the problem of recognizing the accuracy of difference gestures in complex backgrounds by effectively combining the feature information of the contextual framework, taking into account the multi-scale problem of the gestures in the image, this paper introduces the SPPFCSPS module, which realizes multi-feature fusion and improves real-time accuracy of detection. Finally, the model proposed in this paper is compared with other models, and the proposed DMS-yolov8 model achieves good results on both publicly available datasets and homemade datasets, with the average accuracy up to 97.4% and the average mAP value up to 96.3%, The improvements proposed in this paper are effectively validated.

Gesture Detection System using OpenCV

In the rapidly evolving landscape of human-computer interaction, our project delves into the innovative realm of gesture detection technology. This study presents the development of a sophisticated Gesture Detection System, a responsive interface bridging the gap between human hand movements and digital actions. With the invaluable support of our project supervisor, Mrs. V. M. Khanapure, our project not only explores the intricate nuances of gesture recognition but also showcases the collaborative spirit of our team. Utilizing OpenCV and MediaPipe libraries, our system employs advanced computer vision algorithms for real-time hand tracking and precise gesture recognition. The project’s success is not only attributed to the technical acumen of our team members but also to the seamless cooperation and encourageme Among them. Furthermore, the availability of essential resources provided by Mr.V. Nitnaware Principal, PLGPL, significantly bolstered our project development process. This abstract encapsulates a journey of collaborative innovation, highlighting the symbiotic relationship between technology and teamwork. Through this Gesture Detection System, we aim to redefine user experiences, offering an intuitive and interactive interface that responds to human gestures with precision and fluidity

Development and Analysis of an Origami-Based Elastomeric Actuator and Soft Gripper Control with Machine Learning and EMG Sensors.

This study investigates the characteristics of a novel origami-based, elastomeric actuator and a soft gripper, which are controlled by hand gestures that are recognized through machine learning algorithms. The lightweight paper-elastomer structure employed in this research exhibits distinct actuation features in four key areas: (1) It requires approximately 20% less pressure for the same bending amplitude compared to pneumatic network actuators (Pneu-Net) of equivalent weight, and even less pressure compared to other actuators with non-linear bending behavior; (2) The control of the device is examined by validating the relationship between pressure and the bending angle, as well as the interaction force and pressure at a fixed bending angle; (3) A soft robotic gripper comprising three actuators is designed. Enveloping and pinch grasping experiments are conducted on various shapes, which demonstrate the gripper's potential in handling a wide range of objects for numerous applications; and (4) A gesture recognition algorithm is developed to control the gripper using electromyogram (EMG) signals from the user's muscles.

Football referee gesture recognition algorithm based on YOLOv8s.

Gesture serves as a crucial means of communication between individuals and between humans and machines. In football matches, referees communicate judgment information through gestures. Due to the diversity and complexity of referees' gestures and interference factors, such as the players, spectators, and camera angles, automated football referee gesture recognition (FRGR) has become a challenging task. The existing methods based on visual sensors often cannot provide a satisfactory performance. To tackle FRGR problems, we develop a deep learning model based on YOLOv8s. Three improving and optimizing strategies are integrated to solve these problems. First, a Global Attention Mechanism (GAM) is employed to direct the model's attention to the hand gestures and minimize the background interference. Second, a P2 detection head structure is integrated into the YOLOv8s model to enhance the accuracy of detecting smaller objects at a distance. Third, a new loss function based on the Minimum Point Distance Intersection over Union (MPDIoU) is used to effectively utilize anchor boxes with the same shape, but different sizes. Finally, experiments are executed on a dataset of six hand gestures among 1,200 images. The proposed method was compared with seven different existing models and 10 different optimization models. The proposed method achieves a precision rate of 89.3%, a recall rate of 88.9%, a mAP@0.5 rate of 89.9%, and a mAP@0.5:0.95 rate of 77.3%. These rates are approximately 1.4%, 2.0%, 1.1%, and 5.4% better than those of the newest YOLOv8s, respectively. The proposed method has right prospect in automated gesture recognition for football matches.

Adaptive Trajectories’ Constant False Alarm Rate Mirror Filters and Elevation Angle Evaluation for Multiple-Input Multiple-Output Radar-Based Hand Gesture Recognition

Gesture recognition technology has been quickly developed in the field of human–computer interaction. The multiple-input multiple-output (MIMO) radar is popular in gesture recognition because of its notable spatial resolution. This work proposes a MIMO radar-based hand gesture recognition algorithm with low complexity. We leverage low-complexity adaptive signal processing to extract trajectory information and minimize noise to create a system that can be applied in real-world applications with small training datasets. First, a spectrum analysis is utilized on range-Doppler maps (RDMs), and a cell-averaging constant false alarm rate (CA-CFAR) with mirror filters is applied to improve the robustness of noise. Then, the features related to the distance, speed, direction, and elevation angle of the moving object are determined using the proposed adaptive signal analysis techniques. For classification, the random forest algorithm is implemented. The proposed system can precisely distinguish and identify eight gestures, including waving, moving to the left or right, patting, pushing, pulling, and rotating clockwise or anti-clockwise, with an accuracy of 95%. Experiments demonstrate the capability of the proposed hand gesture recognition system to classify different movements precisely.

Research on gesture recognition algorithm based on MME-P3D.

A Multiscale-Motion Embedding Pseudo-3D (MME-P3D) gesture recognition algorithm has been proposed to tackle the issues of excessive parameters and high computational complexity encountered by existing gesture recognition algorithms deployed in mobile and embedded devices. The algorithm initially takes into account the characteristics of gesture motion information, integrating the channel attention (CE) mechanism into the pseudo-3D (P3D) module, thereby constructing a P3D-C feature extraction network that can efficiently extract spatio-temporal feature information while reducing the complexity of the algorithmic model. To further enhance the understanding and learning of the global gesture movement's dynamic information, a Multiscale Motion Embedding (MME) mechanism is subsequently designed. The experimental findings reveal that the MME-P3D model achieves recognition accuracies reaching up to 91.12% and 83.06% on the self-constructed conference gesture dataset and the publicly available Chalearn 2013 dataset, respectively. In comparison with the conventional 3D convolutional neural network, the MME-P3D model demonstrates a significant advantage in terms of parameter count and computational requirements, which are reduced by as much as 82% and 83%, respectively. This effectively addresses the limitations of the original algorithms, making them more suitable for deployment on embedded and mobile devices and providing a more effective means for the practical application of hand gesture recognition technology.

Enhance the AI Virtual System Accuracy with Novel Hand Gesture Recognition Algorithm Comparing to Convolutional Neural Network

The objective of this study is to enhance the precision of AI virtual systems by implementing Novel Hand Gesture Recognition techniques in comparison to Convolutional Neural Network. Materials and Methods: To recognise hand motions, a Convolutional Neural Network with distinct training and testing stages is utilized. The average Gpower for the test is between 0.05 and 0.85, or around 85%. Sample size is determined as 27,455 for each group using G Power 3.1 software (G Power setting parameters: α=0.05 and power=0.85). Results and Discussion: Novel Hand gesture recognition 92.60% identifies between objects and boosts the observed accuracy with a statistically non-significant value of p=0.123 (p>0.05) in comparison to the convolutional neural network's 88.59%. Conclusion: Comparison of the Novel Hand gesture Recognition algorithm and Convolutional Neural Network in terms of performance that shows Hand gesture recognition has 91.62% with better accuracy.

Blended English Teaching Model in Higher Education School Environment AR Constructive Technology

China's education modernization requirements clearly suggest exploring new teaching methods to improve teaching effectiveness. Based on this, this article investigates AR construction technology as a blended English teaching model in the university school environment. Mobile terminals are used to build the blended English teaching model, modeling tools are used to achieve split modeling, and the mobile terminals themselves are used to achieve the management and network synchronization work for the cloud classroom. For the gesture recognition model construction, 3D convolutional neural networks are used to separate and optimize the parameters. Finally, experiments are designed to simulate and analyze the AR construct of the blended English teaching model to determine accuracy. The simulation results show that the model can realize the simultaneous display of teacher explanation, and the improved algorithm of gesture recognition is improved in recognition rate, which can improve the effectiveness of blended English teaching.

Dynamic Interactive Design of Graphic Design Elements Based on Image Feature Extraction Algorithm

Abstract This paper first analyzes the use and expression of line and surface, color, and text as graphic design composition elements. Secondly, it analyzes the relationship between interaction design and other disciplines and constructs a dynamic interactive design model for graphic design. Then, the features of color, texture, shape and spatial relationship of the image are extracted by the feature extraction algorithm, and the gesture recognition algorithm is used to determine the operation requested by the user and execute it. Finally, data analysis was conducted on 32 randomly selected students from the design school of University H who applied the dynamic interactive design model. The results show that the dynamic interactive design has the highest rating in terms of smooth operation, with an average score of 6.893, and the lowest rating is in terms of low learning cost, with an average score of 5.087. This shows that the dynamic interactive design model of graphic elements can improve the smoothness of operation, thus improving the dynamic interactive experience of students and providing direction for the development of the field of graphic design.

Real-Time Dynamic Gesture Recognition Algorithm Based on Adaptive Information Fusion and Multi-Scale Optimization Transformer

Gesture recognition is a popular technology in the field of computer vision and an important technical mean of achieving human-computer interaction. To address problems such as the limited long-range feature extraction capability of existing dynamic gesture recognition networks based on convolutional operators, we propose a dynamic gesture recognition algorithm based on spatial pyramid pooling Transformer and optical flow information fusion. We take advantage of Transformer’s large receptive field to reduce model computation while improving the model’s ability to extract features at different scales by embedding spatial pyramid pooling. We use the optical flow algorithm with the global motion aggregation module to obtain an optical flow map of hand motion, and to extract the key frames based on the similarity minimization principle. We also design an adaptive feature fusion method to fuse the spatial and temporal features of the dual channels. Finally, we demonstrate the effectiveness of model components on model recognition enhancement through ablation experiments. We conduct training and validation on the SCUT-DHGA dynamic gesture dataset and on a dataset we collected, and we perform real-time dynamic gesture recognition tests using the trained model. The results show that our algorithm achieves high accuracy even while keeping the parameters balanced. It also achieves fast and accurate recognition of dynamic gestures in real-time tests.

Optimal video processing and soft computing algorithms for human hand gesture recognition from real-time video

Optimal video processing and soft computing algorithms for human hand gesture recognition from real-time video

Gesture Recognition of sEMG Signals Based on CNN-GRU Network

To improve the accuracy of surface electromyogram signal (sEMG) gesture recognition algorithm and solve the problem of manually extracting many features, this paper proposes a deep neural network-based gesture recognition method. A neural network integrating CNN and GRU was designed. The 8-channel sEMG data collected by the MYO armband is input to the CNN for feature extraction, and then the obtained feature sequence is input to the GRU network for gesture classification, and finally the recognition result of the gesture category is output. The experimental findings that the proposed technology reaches 76.41% recognition accuracy on the MyoUP dataset. This demonstrates the practicality of the suggested plan.

Retracted: Gesture Tracking and Recognition Algorithm for Dynamic Human Motion Using Multimodal Deep Learning

Retracted: Gesture Tracking and Recognition Algorithm for Dynamic Human Motion Using Multimodal Deep Learning

Smart-Data-Glove-Based Gesture Recognition for Amphibious Communication.

This study has designed and developed a smart data glove based on five-channel flexible capacitive stretch sensors and a six-axis inertial measurement unit (IMU) to recognize 25 static hand gestures and ten dynamic hand gestures for amphibious communication. The five-channel flexible capacitive sensors are fabricated on a glove to capture finger motion data in order to recognize static hand gestures and integrated with six-axis IMU data to recognize dynamic gestures. This study also proposes a novel amphibious hierarchical gesture recognition (AHGR) model. This model can adaptively switch between large complex and lightweight gesture recognition models based on environmental changes to ensure gesture recognition accuracy and effectiveness. The large complex model is based on the proposed SqueezeNet-BiLSTM algorithm, specially designed for the land environment, which will use all the sensory data captured from the smart data glove to recognize dynamic gestures, achieving a recognition accuracy of 98.21%. The lightweight stochastic singular value decomposition (SVD)-optimized spectral clustering gesture recognition algorithm for underwater environments that will perform direct inference on the glove-end side can reach an accuracy of 98.35%. This study also proposes a domain separation network (DSN)-based gesture recognition transfer model that ensures a 94% recognition accuracy for new users and new glove devices.

Kinect-based gesture interaction design method for smart terminal APP interface

Abstract In this paper, a static gesture classification model based on a convolutional neural network is proposed to achieve the goal of efficient interaction control and fast response time for gesture recognition, and hand feature extraction is achieved by using alternating convolution and pooling. In the convolutional neural network, image data is obtained by preprocessing operations such as adaptive thresholding and functions such as Gaussian filtering to provide data for the convolutional neural network structure to build the recognition model, which is used to quantify the edge characteristics of the image and construct the image edges with gradient amplitude to build the static gesture recognition algorithm for Kinect. In the accuracy rate for gesture interaction, the average recognition rate for slow movement is 96.5%, while when the interactive gesture is recognized at an accelerated speed, its recognition rate slips to 93.1%, and the total average recognition rate reaches 95.2%.

Design and research of music teaching system based on virtual reality system in the context of education informatization.

Virtual Reality (VR) technology uses computers to simulate the real world comprehensively. VR has been widely used in college teaching and has a huge application prospect. To better apply computer-aided instruction technology in music teaching, a music teaching system based on VR technology is proposed. First, a virtual piano is developed using the HTC Vive kit and the Leap Motion sensor fixed on the helmet as the hardware platform, and using Unity3D, related SteamVR plug-ins, and Leap Motion plug-ins as software platforms. Then, a gesture recognition algorithm is proposed and implemented. Specifically, the Dual Channel Convolutional Neural Network (DCCNN) is adopted to collect the user's gesture command data. The dual-size convolution kernel is applied to extract the feature information in the image and the gesture command in the video, and then the DCCNN recognizes it. After the spatial and temporal information is extracted, Red-Green-Blue (RGB) color pattern images and optical flow images are input into the DCCNN. The prediction results are merged to obtain the final recognition result. The experimental results reveal that the recognition accuracy of DCCNN for the Curwen gesture is as high as 96%, and the recognition accuracy varies with different convolution kernels. By comparison, it is found that the recognition effect of DCCNN is affected by the size of the convolution kernel. Combining convolution kernels of size 5×5 and 7×7 can improve the recognition accuracy to 98%. The research results of this study can be used for music teaching piano and other VR products, with extensive popularization and application value.