Gesture Recognition Accuracy Research Articles

Gesture Control Drone: Using Gloves

Gesture-controlled drones represent a significant advancement in human-computer interaction, allowing users to operate drones using simple hand movements without the need for traditional controllers. This technology utilizes computer vision, machine learning, and sensor-based systems to interpret gestures and translate them into drone commands such as takeoff, landing, movement, and hovering. A typical gesture-controlled drone employs a combination of cameras, accelerometers, gyroscopes, and deep learning models to recognize predefined gestures in real-time. Image processing techniques, such as OpenCV and deep neural networks, enhance the accuracy of gesture recognition, ensuring seamless communication between the user and the drone. Keywords--- Gesture Recognition, Drone Control, Computer Vision.

Body gesture recognition using crow search algorithm enhanced probabilistic neural network for human- computer interaction

Background Body gesture recognition has become a fundamental technique in Human-Computer Interaction (HCI). As human-machine interaction evolves, there is an increasing need for precise and efficient gesture detection systems. However, current methods face limitations such as accuracy constraints, high computational complexity, and limited adaptability. This study addresses these challenges by proposing an innovative approach to enhance the accuracy and efficiency of body gesture recognition systems. Methods The proposed system integrates advanced algorithms and techniques to improve performance. A Marker-Based Watershed Algorithm is employed for accurate image segmentation, enhancing region detection. Feature extraction uses a Convolutional Neural Network (CNN), while a Wavelet Transform-Based Pre-Processing technique improves input data quality. A unique component of this method is the application of the Crow Search Algorithm to optimize model efficiency. An Optimized Probabilistic Neural Network (PNN) is utilized for gesture classification, aiming to increase precision and computational effectiveness. Results The proposed approach achieves a gesture recognition accuracy rate of 99%. Compared to traditional methods such as Decision Trees (DT), Support Vector Machines (SVM), and Improved Neural Networks (INN), the Optimized PNN demonstrates a 2.21% improvement in overall accuracy. The implementation, carried out in Python, showcases the robustness and adaptability of the system across diverse HCI applications. Conclusions This work presents a comprehensive solution to the challenges of body gesture recognition by integrating cutting-edge algorithms. Combining the Marker-Based Watershed Algorithm, CNN-based feature extraction, and Crow Search Optimization significantly enhances the system’s accuracy and efficiency. By addressing the shortcomings of existing methods, this approach provides a more responsive, reliable, and flexible gesture recognition system, contributing to the advancement of HCI technologies. The results demonstrate the potential for improved human-computer interaction through more effective and precise gesture detection.

Intelligent Gesture Recognition Gloves for Real-Time Monitoring in Wireless Human-Computer Interaction.

Flexible sensors mimic the sensing ability of human skin, and have unique flexibility and adaptability, allowing users to interact with intelligent systems in a more natural and intimate way. To overcome the issues of low sensitivity and limited operating range of flexible strain sensors, this study presents a highly innovative preparation method to develop a conductive elastomeric sensor with a cracked thin film by combining polydimethylsiloxane (PDMS) with multiwalled carbon nanotubes (MCNT). This novel design significantly increases both the sensitivity and operating range of the sensor (strain range 0-50%; the maximum tensile sensitivity of this sensor reaches 4.97), marking a breakthrough in flexible sensor technology. The sensor can also be fabricated easily and inexpensively which is suitable for large-scale production, indicating its significant application potential. Combined with a dedicated hardware and software system, the sensors enable real-time monitoring of the finger status and accurate gesture recognition. Furthermore, machine-learning algorithms were used to identify American Sign Language gestures with an accuracy of up to 97%. This innovative human-machine interface provides important support for the future development of intelligent interactive systems, and is of great significance in promoting the progress of advanced human-computer interaction technologies.

Application of Fresnel Zone Model in Wireless Sensing Field

As a common model in the field of wireless sensing, the Fresnel zone model performs better than traditional deep learning in generalization ability, computational efficiency, robustness, and so on. It shows significant advantages in breath monitoring, gesture recognition, indoor behavior recognition, and so on. In this paper, the typical applications of the Fresnel band model in fine and coarse-grained wireless sensing tasks in recent ten years are listed. With the progress of the research, the concentric circle model, dynamic Fresnel region model, and through-wall wireless sensing model are developed in combination with other methods, which can well solve the wireless perception problem in non-line-of-sight scenes. Or combined with deep learning to develop new ways to solve specific problems. Over the course of development, the emergence of new methods makes the wireless perception scene more complex, Respiratory monitoring can be changed from one-person monitoring to multi-person detection, and the accuracy of gesture recognition and indoor behavior recognition is getting higher and higher.

Local Pyramid Vision Transformer: Millimeter-Wave Radar Gesture Recognition Based on Transformer with Integrated Local and Global Awareness

A millimeter-wave radar is widely accepted by the public due to its low susceptibility to interference, such as changes in light, and the protection of personal privacy. With the development of the deep learning theory, the deep learning method has been dominant in the millimeter-wave radar field, which usually uses convolutional neural networks for feature extraction. In recent years, transformer networks have also been highly valued by researchers due to their parallel processing capabilities and long-distance dependency modeling capabilities. However, traditional convolutional neural networks (CNNs) and vision transformers each have their limitations: CNNs usually overlook the global features of images and vision transformers may neglect local image continuity, and both of them may impede gesture recognition performance. In addition, whether CNN or transformer, their implementation is hindered by the scarcity of public radar gesture datasets. To address these limitations, this paper proposes a new recognition method using a local pyramid visual transformer (LPVT) based on millimeter-wave radar. LPVT can capture global and local features in dynamic gesture spectrograms, ultimately improving the recognition ability of gestures. In this paper, we mainly carried out the following two tasks: building the corresponding datasets and executing gesture recognition. First, we constructed a gesture dataset for training. In this stage, we use a 77 GHz radar to collect the echo signals of gestures and preprocess them to build a dataset. Second, we propose the LPVT network specifically designed for gesture recognition tasks. By integrating local sensing into the globally focused transformer, we improve its capacity to capture both global and local features in dynamic gesture spectrograms. The experimental results using the dataset we constructed show that the proposed LPVT network achieved a gesture recognition accuracy of 92.2%, which exceeds the performance of other networks.

Adaptive optimization of electromyographic channels for intelligent prosthetic hands based on individual differences

Intelligent prosthetic hands typically require an increase in the number of acquisition channels to improve gesture recognition accuracy, resulting in increased device complexity and cost. However, there are individual differences in muscle strength, body mass index, and exercise habits. Electromyographic prosthetic hands currently use standardized electromyographic channel configurations, which lack adaptability to individual differences. To address these issues, this paper proposes the electrode configuration adaptive optimization algorithm, which enhances and integrates traditional genetic algorithms and simulated annealing algorithms, and implements adaptive optimization solutions for different subjects. Experimental results show that the optimization outcomes differ among different subjects. Compared to a single optimization algorithm, the proposed algorithm can adaptively optimize the electrode configuration based on individual differences while ensuring recognition effectiveness, retaining electrode channel information that significantly contributes to gesture classification recognition, and meeting the stable recognition of their motion intentions by different subjects.

Hand Gesture Enabled Operation for Computer Using Deep Learning

This project aims to develop a system that detects hand gestures and prints corresponding instructions for computer operators. The system utilizes Deep learning algorithms CNN and mediapipe to recognize hand gestures, and then prints instructions for the operator to perform specific computer operations.Operations like Notepad Open,Shutdown,Volume Up,Volume Down etc.The system will consist of a webcam, a Deep learning model, and a printer. The webcam will capture images of hand gestures, which will be processed using the Deep learning model to identify the specific gesture. The system will then print the corresponding instruction on the printer. The system will be designed to be user-friendly and adaptable to different environments and hardware configurations.The system aims to improve efficiency, reduce errors, and enhance accessibility for computer operators. The proposed system consists of a hand gesture detection module, instruction printing module, and computer operation module. The system is trained using a dataset of hand gestures and corresponding computer operations. Experimental results show that the system achieves high accuracy in gesture recognition and instruction printing. The system has potential applications in various fields, including gaming, education, and healthcare. Key Words: Hand gesture detection, Deep learning, instruction printing, computer operation, accessibility, Mediapipe, CNN.

Sign Language to Speech Conversion

Human communication relies heavily on verbal and non-verbal cues, with sign language serving as a crucial method of interaction for individuals with hearing impairments. However, the communication barrier between sign language users and those unfamiliar with it remains a significant challenge. This paper presents an innovative approach to bridging this gap through an Arduino-based system that converts sign language gestures directly into speech without the use of a camera. By utilizing flex sensors and an accelerometer, this system detects hand gestures and movements, processes this data, and produces corresponding audio output. This approach offers a more accessible and portable solution compared to traditional camera-based systems, potentially revolutionizing real-time communication for the deaf community. The experimental results demonstrate a gesture recognition accuracy of 92% and a response time of under 500 milliseconds, indicating the system's viability for practical applications

Enhancement algorithm for surface electromyographic-based gesture recognition based on real-time fusion of muscle fatigue features

This study aims to optimize surface electromyography-based gesture recognition technique, focusing on the impact of muscle fatigue on the recognition performance. An innovative real-time analysis algorithm is proposed in the paper, which can extract muscle fatigue features in real time and fuse them into the hand gesture recognition process. Based on self-collected data, this paper applies algorithms such as convolutional neural networks and long short-term memory networks to provide an in-depth analysis of the feature extraction method of muscle fatigue, and compares the impact of muscle fatigue features on the performance of surface electromyography-based gesture recognition tasks. The results show that by fusing the muscle fatigue features in real time, the algorithm proposed in this paper improves the accuracy of hand gesture recognition at different fatigue levels, and the average recognition accuracy for different subjects is also improved. In summary, the algorithm in this paper not only improves the adaptability and robustness of the hand gesture recognition system, but its research process can also provide new insights into the development of gesture recognition technology in the field of biomedical engineering.

Design and Implementation of Gesture Music Composition System based on RealSense and Recurrent Neural Network

Aiming at the problems of slow human-computer interaction and low accuracy of hand gesture recognition (HGR) in music composition systems such as human-computer interaction, gesture recognition, and algorithmic composition, this paper designs an HGR music composition system based on real-sense technology and recurrent neural network. The system uses an Intel RealSense 3D camera to shoot the user's hand and extract the information on the hand joint points with Smart Wearable Biosensors (SWBSs). The joint points are used to construct a three-dimensional model of the hand bone, and the joint points' equivalent distance and joint deflection are calculated by the joint point information to match the corresponding Curwen HGR. The matching music data is input into the recurrent neural network. GRU is combined with the Markov chain algorithm to complete the composition and avoid the phenomenon of gradient disappearance or gradient explosion. The HGR recognition function and composition algorithm are simulated and tested to verify the method's feasibility. Cohort verification investigations and efficacy evaluations of wearable biosensors are essential to support their clinical acceptability. The research results show that the HGR music composition system based on real-sense technology and recurrent neural networks can effectively identify the tester's HGRs and assist the tester in completing the composition using intelligent biosensors. The composition mode is close to the professional composition process.

An Actively Vision-Assisted Low-Load Wearable Hand Function Mirror Rehabilitation System

The restoration of fine motor function in the hand is crucial for stroke survivors with hemiplegia to reintegrate into daily life and presents a significant challenge in post-stroke rehabilitation. Current mirror rehabilitation systems based on wearable devices require medical professionals or caregivers to assist patients in donning sensor gloves on the healthy side, thus hindering autonomous training, increasing labor costs, and imposing psychological burdens on patients. This study developed a low-load wearable hand function mirror rehabilitation robotic system based on visual gesture recognition. The system incorporates an active visual apparatus capable of adjusting its position and viewpoint autonomously, enabling the subtle monitoring of the healthy side’s gesture throughout the rehabilitation process. Consequently, patients only need to wear the device on their impaired hand to complete the mirror training, facilitating independent rehabilitation exercises. An algorithm based on hand key point gesture recognition was developed, which is capable of automatically identifying eight distinct gestures. Additionally, the system supports remote audio–video interaction during training sessions, addressing the lack of professional guidance in independent rehabilitation. A prototype of the system was constructed, a dataset for hand gesture recognition was collected, and the system’s performance as well as functionality were rigorously tested. The results indicate that the gesture recognition accuracy exceeds 90% under ten-fold cross-validation. The system enables operators to independently complete hand rehabilitation training, while the active visual system accommodates a patient’s rehabilitation needs across different postures. This study explores methods for autonomous hand function rehabilitation training, thereby offering valuable insights for future research on hand function recovery.

FM‐YOLOv8:Lightweight gesture recognition algorithm

AbstractIn practical production applications, the efficiency and success rate of gesture recognition directly affect the user experience and work efficiency. However, the existing gesture recognition models have the problem of a large number of model parameters and high computational complexity, which makes them unable to meet the needs of end‐to‐end industrial deployment. To solve these problems, this article proposes a gesture recognition model based on YOLOv8. First, FasterNet is adopted as the backbone of YOLOv8, which significantlydecrease the number of parameters and made the model more lightweight. By reducing the number of parameters, the computational complexity of the model can be reduced while maintaining the performance of the model, and the operation efficiency of the model can be improved. Second, recombination convolution ScConv is introduced to replace common convolution operations to further improve the model's efficiency. Recombination convolution can reduce the computation and make up for the loss of precision to some extent. Finally, the MDPIoU loss function is used to optimize target location and prediction, to improve the accuracy of the model. The MDPIoU loss function can better deal with the problem of target boundary frame positioning and prediction so that the model can locate and predict gestures more accurately in gesture recognition tasks. Experiments on a data set containing 10 types of gestures show that the number of parameters and floating point calculations of the improved network model are reduced by 45% and 42.7%, respectively, while the accuracy is unchanged. The improved model can be deployed on edge terminals, providing efficient and accurate gesture recognition.

Human Muscle sEMG Signal and Gesture Recognition Technology Based on Multi-Stream Feature Fusion Network

Surface electromyography signals have significant value in gesture recognition due to their ability to reflect muscle activity in real time. However, existing gesture recognition technologies have not fully utilized surface electromyography signals, resulting in unsatisfactory recognition results. To this end, firstly, a Butterworth filter was adopted to remove high-frequency noise from the signal. A combined method of moving translation threshold was introduced to extract effective signals. Then, a gesture recognition model based on multi-stream feature fusion network was constructed. Feature extraction and fusion were carried out through multiple parallel feature extraction paths, combined with convolutional neural networks and residual attention mechanisms. Compared to popular methods of the same type, this new recognition method had the highest recognition accuracy of 92.1% and the lowest recognition error of 5%. Its recognition time for a single-gesture image was as short as 4s, with a maximum Kappa coefficient of 0.92. Therefore, this method combining multi-stream feature fusion networks can effectively improve the recognition accuracy and robustness of gestures and has high practical value.

EchoGesture Communication: Gesture-based Systems for Individuals with Disabilities

EchoGesture Communication revolutionizes the interaction of differently-abled individuals using hand gestures. People with disabilities often face difficulties in using the conventional electronic gadgets. The proposed study, utilizes sensors, microcontroller, computer vision, and machine learning, to enable real-time recognition of hand gestures, facilitating effective communication. Additionally, Convolutional Neural Network (CNN) is used in the research to achieve accurate gesture recognition. The proposed system allows individuals with disability to communicate effectively using hand gestures.

Feature Extraction and Convolutional Neural Networks for Static Hand Gesture Recognition and Context Sentence Generation

Hand gestures based on human-computer interaction are both intuitive and versatile, with multiple and diverse applications including in smart homes, games, operating theaters and vehicle infotainment systems. This research presents a novel architecture by combining a convolutional neural network (CNN) and traditional feature extractors to examine the accuracy of static hand gesture recognition. This research provides three significant contributions. First, we use the Non-Dominated Sorting Genetic Algorithm II (NSGAII), an evolutionary algorithm to classify and select image features across five methods, including the Gabor filter, the Hu-moment, the Zernike moment, the Complex moment, and the Fourier moment. Experimental results demonstrated that the combination of the Gabor filter, the Hu moment, and the Zernike moment achieved the best result with an accuracy of 98.3% to 99.0%. The Zernike moment combined with the Hu-moment output had an accuracy of 95.5% to 98.0%. The second contribution proposes the use of the Multiple Feature Convolutional Neural Network (MFCNN) model to generate better image recognition through the combination of validation techniques and features descriptors. Extensive experimentation was conducted utilizing binary and grayscale, as well as two different validation techniques - the Holdout technique and the Cross-validation of leaving one subject out of the validation. The proposed architecture was evaluated on two dataset types and is compared with the state-of-the art convolutional neural networks (CNN). The Massey’s dataset, contained 2,524 images and 36 gestures, and the OUHANDs dataset contained 3,000 images and 10 gestures. Experimental results demonstrated a high recognition rate using descriptors with low computational cost and reduced size. The third contribution is the sequence sentences generation based on the Beam Search (BS) algorithm. The data obtained from CNN/Daily Mail documents and results of image recognition, i.e., the image’s label, were used to test various question size with four different sizes of questions, including 100, 1,000, 10,000, and 40,000. The experimental results showed that our method could achieve high-quality sentence generation.

Effects of Exercise on the Inter-Session Accuracy of sEMG-Based Hand Gesture Recognition.

Surface electromyography (sEMG) is commonly used as an interface in human-machine interaction systems due to their high signal-to-noise ratio and easy acquisition. It can intuitively reflect motion intentions of users, thus is widely applied in gesture recognition systems. However, wearable sEMG-based gesture recognition systems are susceptible to changes in environmental noise, electrode placement, and physiological characteristics. This could result in significant performance degradation of the model in inter-session scenarios, bringing a poor experience to users. Currently, for noise from environmental changes and electrode shifting from wearing variety, numerous studies have proposed various data-augmentation methods and highly generalized networks to improve inter-session gesture recognition accuracy. However, few studies have considered the impact of individual physiological states. In this study, we assumed that user exercise could cause changes in muscle conditions, leading to variations in sEMG features and subsequently affecting the recognition accuracy of model. To verify our hypothesis, we collected sEMG data from 12 participants performing the same gesture tasks before and after exercise, and then used Linear Discriminant Analysis (LDA) for gesture classification. For the non-exercise group, the inter-session accuracy declined only by 2.86%, whereas that of the exercise group decreased by 13.53%. This finding proves that exercise is indeed a critical factor contributing to the decline in inter-session model performance.

Hand Trajectory Recognition by Radar with a Finite-State Machine and a Bi-LSTM

Gesture plays an important role in human–machine interaction. However, the insufficient accuracy and high complexity of gesture recognition have blocked its widespread application. A gesture recognition method that combines state machine and bidirectional long short-term memory (Bi-LSTM) fusion neural network is proposed to improve the accuracy and efficiency. Firstly, gestures with large movements are categorized into simple trajectory gestures and complex trajectory gestures in advance. Afterwards, different recognition methods are applied for the two categories of gestures, and the final result of gesture recognition is obtained by combining the outputs of the two methods. The specific method used is a state machine that recognizes six simple trajectory gestures and a bidirectional LSTM fusion neural network that recognizes four complex trajectory gestures. Finally, the experimental results show that the proposed simple trajectory gesture recognition method has an average accuracy of 99.58%, and the bidirectional LSTM fusion neural network has an average accuracy of 99.47%, which can efficiently and accurately recognize 10 gestures with large movements. In addition, by collecting more gesture data from untrained participants, it was verified that the proposed neural network has good generalization performance and can adapt to the various operating habits of different users.

ChatMatch: Exploring the potential of hybrid vision–language deep learning approach for the intelligent analysis and inference of racket sports

Video understanding technology has become increasingly important in various disciplines, yet current approaches have primarily focused on lower comprehension level of video content, posing challenges for providing comprehensive and professional insights at a higher comprehension level. Video analysis plays a crucial role in athlete training and strategy development in racket sports. This study aims to demonstrate an innovative and higher-level video comprehension framework (ChatMatch), which integrates computer vision technologies with the cutting-edge large language models (LLM) to enable intelligent analysis and inference of racket sports videos. To examine the feasibility of this framework, we deployed a prototype of ChatMatch in the badminton in this study. A vision-based encoder was first proposed to extract the meta-features included the locations, actions, gestures, and action results of players in each frame of racket match videos, followed by a rule-based decoding method to transform the extracted information in both structured knowledge and unstructured knowledge. A set of LLM-based agents included namely task identifier, coach agent, statistician agent, and video manager, was developed through a prompt engineering and driven by an automated mechanism. The automatic collaborative interaction among the agents enabled the provision of a comprehensive response to professional inquiries from users. The validation findings showed that our vision models had excellent performances in meta-feature extraction, achieving a location identification accuracy of 0.991, an action recognition accuracy of 0.902, and a gesture recognition accuracy of 0.950. Additionally, a total of 100 questions were gathered from four proficient badminton players and one coach to evaluate the performance of the LLM-based agents, and the outcomes obtained from ChatMatch exhibited commendable results across general inquiries, statistical queries, and video retrieval tasks. These findings highlight the potential of using this approach that can offer valuable insights for athletes and coaches while significantly improve the efficiency of sports video analysis.

Stretchable Iontronic Tactile Sensing Fabric.

The iontronic tactile sensing modality has garnered significant attention due to its exceptional sensitivity, immunity to noise, and versatility in materials. Recently, various formats of iontronic tactile sensors have been developed, including droplets, polymer films, paper, ionic gels, and fabrics. However, the stretchability of the current iontronic pressure sensing fabric is inadequate, hindered by the limited stretchiness of the ionic functional fabric. Incorporating a stretchable tactile sensing implement could enhance the wear comfortability by preventing relative movement and ensuring intimate contact between the sensor and the skin. The research focuses on the development of a stretchable iontronic pressure sensing (SIPS) fabric for monitoring diverse aspects of body health and movement in wearable applications. The tactile sensing structure is generated at the iontronic interface between highly stretchable ionic and conductive fabrics. In particular, the ionic fabric is prepared by coating a layer of polyurethane/ionic liquid gel onto a Spandex fabric. To showcase its remarkable sensitivity, stretchability, and ability to detect diverse body information, several application scenarios have been demonstrated including an elastic wristband for precise pulse wave detection, a flexible belt with multitactile sensing channels for respiration and motion tracking purposes, and a stretchable fabric cuff equipped with a high-resolution sensing array comprising 32 × 32 units for accurate gesture recognition.

Gesture-Controlled Robotic Arm for Agricultural Harvesting Using a Data Glove with Bending Sensor and OptiTrack Systems.

This paper presents a gesture-controlled robotic arm system designed for agricultural harvesting, utilizing a data glove equipped with bending sensors and OptiTrack systems. The system aims to address the challenges of labor-intensive fruit harvesting by providing a user-friendly and efficient solution. The data glove captures hand gestures and movements using bending sensors and reflective markers, while the OptiTrack system ensures high-precision spatial tracking. Machine learning algorithms, specifically a CNN+BiLSTM model, are employed to accurately recognize hand gestures and control the robotic arm. Experimental results demonstrate the system's high precision in replicating hand movements, with a Euclidean Distance of 0.0131 m and a Root Mean Square Error (RMSE) of 0.0095 m, in addition to robust gesture recognition accuracy, with an overall accuracy of 96.43%. This hybrid approach combines the adaptability and speed of semi-automated systems with the precision and usability of fully automated systems, offering a promising solution for sustainable and labor-efficient agricultural practices.