Dynamic Gesture Recognition Research Articles

Reverse Sign Language Recognition System Using Machine Learning

Deaf and hard-of-hearing people communicate with each other and their society by using sign language. As this is a very natural method for pupils to stay in touch with computers, many academics are working on it to make it less complex and more convenient for use. So the main objective of gesture recognition research is to make systems which can interact and communicate while understanding human gestures and use them. In simple words to communicate information. Fast and extremely accurate hand detection and realtime hand gestures. Identification should be possible with vision-based hand gesture circumstances and interfaces. Learning and knowing sign movements and gestures is the kick start in making words and sentences for computer assisted sign language interpretation. Both dynamic and static sign actions are open and available. Both ways of gesture recognition are crucial to human culture, even if static gesture recognition is much easier than dynamic gesture recognition. When a human enters the value of alphabets and numerical value as input, the system immediately displays or outputs the appropriate recognised character shows the gesture on the monitor screen. In the following, research projects that have led to a proper system that uses convolutional neural networks to identify handwriting on the basis of the depth pictures and Hand Languages (Brain lipi) the collects. Keywords- Convolutional neural network,Text recognition ,Convert to image,StackGAN,simultaneous Tracking text and converting to image, gesture recognition,display output as sign gesture,training machine to(A-z to 1-0).

Dynamic gesture recognition using hand pose-based neural networks for sign language interpretation

Dynamic gesture recognition using hand pose-based neural networks for sign language interpretation

Three-dimensional dynamic gesture recognition method based on convolutional neural network

With the rapid advancement of virtual reality, dynamic gesture recognition technology has become an indispensable and critical technique for users to achieve human–computer interaction in virtual environments. The recognition of dynamic gestures is a challenging task due to the high degree of freedom and the influence of individual differences and the change of gesture space. To solve the problem of low recognition accuracy of existing networks, an improved dynamic gesture recognition algorithm based on ResNeXt architecture is proposed. The algorithm employs three-dimensional convolution techniques to effectively capture the spatiotemporal features intrinsic to dynamic gestures. Additionally, to enhance the model’s focus and improve its accuracy in identifying dynamic gestures, a lightweight convolutional attention mechanism is introduced. This mechanism not only augments the model’s precision but also facilitates faster convergence during the training phase. In order to further optimize the performance of the model, a deep attention submodule is added to the convolutional attention mechanism module to strengthen the network’s capability in temporal feature extraction. Empirical evaluations on EgoGesture and NvGesture datasets show that the accuracy of the proposed model in dynamic gesture recognition reaches 95.03% and 86.21%, respectively. When operating in RGB mode, the accuracy reached 93.49% and 80.22%, respectively. These results underscore the effectiveness of the proposed algorithm in recognizing dynamic gestures with high accuracy, showcasing its potential for applications in advanced human–computer interaction systems.

Machine Learning-Based Gesture Recognition Glove: Design and Implementation.

In the evolving field of human-computer interaction (HCI), gesture recognition has emerged as a critical focus, with smart gloves equipped with sensors playing one of the most important roles. Despite the significance of dynamic gesture recognition, most research on data gloves has concentrated on static gestures, with only a small percentage addressing dynamic gestures or both. This study explores the development of a low-cost smart glove prototype designed to capture and classify dynamic hand gestures for game control and presents a prototype of data gloves equipped with five flex sensors, five force sensors, and one inertial measurement unit (IMU) sensor. To classify dynamic gestures, we developed a neural network-based classifier, utilizing a convolutional neural network (CNN) with three two-dimensional convolutional layers and rectified linear unit (ReLU) activation where its accuracy was 90%. The developed glove effectively captures dynamic gestures for game control, achieving high classification accuracy, precision, and recall, as evidenced by the confusion matrix and training metrics. Despite limitations in the number of gestures and participants, the solution offers a cost-effective and accurate approach to gesture recognition, with potential applications in VR/AR environments.

High-precision dynamic gesture recognition based on microfiber sensor and EMT-Net

Dynamic gesture recognition, which utilizes flexible wearable sensors and deep learning, is invaluable for human–computer interaction. Nevertheless, the primary challenges persist are the rapid detection of intricate gestures and the accurate recognition of dynamic signals. In this study, we suggest utilizing a microfiber sensor to identify the variations in wrist skin and detect dynamic gesture. In order to tackle the issue of insufficient feature extraction in the detected signals, resulting in reduced accuracy in recognition, we introduce a network dubbed EMT-Net (improve multi-head attention transformer network). This network utilizes a transformer encoder to capture and represent the characteristics of dynamic gesture signal and uses a CNN to classify the encoded features. To ensure that the model comprehensively captures the temporal and statistical characteristics of the signals, we enhance the multi-head attention mechanism by restricting certain attention heads to concentrate solely on the statistical features of the signals while allowing others to focus on the temporal features and global dependencies. Furthermore, because of the varying discriminatory abilities of different characteristics, we have developed an attention module to redistribution the attention weights on statistical features. The experimental results demonstrate that the microfiber sensor effectively recognizes ten distinct forms of dynamic gesture signals. Simultaneously, EMT-Net attains proficient identification with an accuracy of 98.80%, precision of 98.81%, recall of 98.80%, and an F1 score of 98.80%. The application value of this dynamic gesture recognition technology, which utilizes microfiber sensors and EMT-Net, is significant. The forthcoming alterations in human–computer interaction, virtual reality, and various other domains are anticipated.

Next-Gen Dynamic Hand Gesture Recognition: MediaPipe, Inception-v3 and LSTM-Based Enhanced Deep Learning Model

Gesture recognition is crucial in computer vision-based applications, such as drone control, gaming, virtual and augmented reality (VR/AR), and security, especially in human–computer interaction (HCI)-based systems. There are two types of gesture recognition systems, i.e., static and dynamic. However, our focus in this paper is on dynamic gesture recognition. In dynamic hand gesture recognition systems, the sequences of frames, i.e., temporal data, pose significant processing challenges and reduce efficiency compared to static gestures. These data become multi-dimensional compared to static images because spatial and temporal data are being processed, which demands complex deep learning (DL) models with increased computational costs. This article presents a novel triple-layer algorithm that efficiently reduces the 3D feature map into 1D row vectors and enhances the overall performance. First, we process the individual images in a given sequence using the MediaPipe framework and extract the regions of interest (ROI). The processed cropped image is then passed to the Inception-v3 for the 2D feature extractor. Finally, a long short-term memory (LSTM) network is used as a temporal feature extractor and classifier. Our proposed method achieves an average accuracy of more than 89.7%. The experimental results also show that the proposed framework outperforms existing state-of-the-art methods.

Implementation and research of gesture recognition in HCI field

Since the first Neural Network computer was made by Marvin Minsky and his schoolmates in 1951, the development of artificial intelligence (AI)has undergone various and huge changes. It generally evolves into a field that has infinite possibilities and a major branch of computer science that can not be ignored. With the trend of computerization and the rapid evolution of the Internet of Things (IoT), gesture recognition emerged. Various prototypes are blossoming in the laboratories, and some of them have become certain products that have practical applications in later days. At the same time, more and more deep learning technology has been applied to gesture recognition systems that greatly improve the quality of the service. The purpose of this review is to summarize and analyze the existing algorithms of Dynamic Gesture Recognition systems, which have several different methods that are based on multiple signal extractors. This study focuses on the 3D Convolutional Neural Network(CNN), which plays an important role in the Dynamic Gesture Recognition optimization algorithm and data analysis algorithm. In this paper, we reviewed past papers in the gesture recognition field, which include the sEMG method, microwave method, and vision recognition method. During the process of data collecting and summarizing the past papers, we mainly focused on the accuracy differences between recognition methods and the efficiency differences in data processing algorithms. (CNN-based, LSTM based, etc.) Then, we analyzed the main difficulties and challenges of these methods, which are briefly listed in the Introduction part. Data processing algorithms are also being studied, and a horizontal comparison between CNN-based, LSTM-based, and transformer-based algorithms is also being made. Besides this, for those problems that already have a reliable solution, we also summarized the possible solutions and listed them out. We found that the gesture recognition system has already been systematically studied and is partly used in some fields. However, the algorithm and modeling methods can still be optimized and it also needs further study to be more widely used.

Dynamic gesture recognition model based on FMCW radar and multidimensional feature fusion with dual-channel CNN and GRU

With the development of computers, gestures have also become a commonly used human-computer interaction, so it becomes important to recognize gestures correctly. The objective of this paper is to propose a deep learning model to improve the accuracy of low performance radar in dynamic gesture recognition.We use a dataset acquired by CAL60S244-AB radar to classify ten gestures. The model consists of a dual-channel convolutional neural network(CNN), Gate Recurrent Unit(GRU) and a classification module. The inputs to the model are feature vectors fused in four dimensions of distance, speed, receiving antenna channel and time, from which the dual-channel CNN can extract shallow and deep spatial information, and GRU can extract temporal information, and the combination of these two parts makes the model's classification accuracy improve. Through experiments, the accuracy of the model on the dataset reaches 95.1%, and the results show that the model proposed in this paper can improve the accuracy of dynamic gesture classification for low-precision radar, which verifies the effectiveness of the model.

Gesture-mmWAVE: Compact and Accurate Millimeter-Wave Radar-Based Dynamic Gesture Recognition for Embedded Devices

Gesture-mmWAVE: Compact and Accurate Millimeter-Wave Radar-Based Dynamic Gesture Recognition for Embedded Devices

Augmented Virtual Mouse System with Enhanced Gesture Recognition

This paper presents the development of a contactless system designed to serve as an alternative to the traditional physical input device, such as a computer mouse or touchpad, utilized in human-computer interaction. The proposed system aims to offer a more convenient and hygienic way of interacting with computer devices by using hand gestures instead of physical contact with the mouse system or machine interface. The system has potential applications in various fields including healthcare, public interfaces, and gaming. Existing gesture-controlled input systems lack a comprehensive touchless and accurate system capable of performing almost every operation of a physical mouse. The proposed approach involves recognising the hand and its gestures. Hand tracking and dynamic gesture recognition are incorporated to ensure a smooth and hassle-free user experience during interaction. The efficient Mediapipe framework is used to detect and track hand movements from video frames captured through a computing device Camera.Computer vision algorithms are employed for gesture recognition within each frame, along with Python libraries for processing gestures. Sufficient gestures are added into the system, enabling it to execute a wide range of mouse functions with substantial precision metrics. This paper provides details regarding the development and overall performance of the system. Key Words: hands, mouse, contactless, gesture, OpenCV, mediapipe

Research on Dynamic Gesture Recognition and Control System based on Machine Vision

Hand gesture recognition and control is a new type of human-computer interaction that can provide a more convenient and efficient operation mode by utilizing non-contact gesture recognition technology. This paper presents a lightweight dynamic gesture recognition method for intelligent office presentation control. First, we introduce the concept of hand gesture recognition and go over key gesture recognition technologies like classification. The structure, process, and evaluation index of the gesture recognition algorithm are described in detail using a convolutional neural network model. During the experiment's algorithm verification phase, we test and analyze the algorithm using the Python language, compilation environment, and data set. In the control experiment, we evaluated the system's ability to control the office application's start, play, next, previous, and exit functions. We achieve 96.3% accuracy on the test set. Experimental results show that the system can recognize a wide range of hand gestures and accurately control the presentation.

ST-TGR: Spatio-Temporal Representation Learning for Skeleton-Based Teaching Gesture Recognition.

Teaching gesture recognition is a technique used to recognize the hand movements of teachers in classroom teaching scenarios. This technology is widely used in education, including for classroom teaching evaluation, enhancing online teaching, and assisting special education. However, current research on gesture recognition in teaching mainly focuses on detecting the static gestures of individual students and analyzing their classroom behavior. To analyze the teacher's gestures and mitigate the difficulty of single-target dynamic gesture recognition in multi-person teaching scenarios, this paper proposes skeleton-based teaching gesture recognition (ST-TGR), which learns through spatio-temporal representation. This method mainly uses the human pose estimation technique RTMPose to extract the coordinates of the keypoints of the teacher's skeleton and then inputs the recognized sequence of the teacher's skeleton into the MoGRU action recognition network for classifying gesture actions. The MoGRU action recognition module mainly learns the spatio-temporal representation of target actions by stacking a multi-scale bidirectional gated recurrent unit (BiGRU) and using improved attention mechanism modules. To validate the generalization of the action recognition network model, we conducted comparative experiments on datasets including NTU RGB+D 60, UT-Kinect Action3D, SBU Kinect Interaction, and Florence 3D. The results indicate that, compared with most existing baseline models, the model proposed in this article exhibits better performance in recognition accuracy and speed.

Bimodal Intelligent Electronic Skin Based on Proximity and Tactile Interaction for Pressure and Configuration Perception.

The flexible bimodal e-skin exhibits significant promise for integration into the next iteration of human-computer interactions, owing to the integration of tactile and proximity perception. However, those challenges, such as low tactile sensitivity, complex fabrication processes, and incompatibility with bimodal interactions, have restricted the widespread adoption of bimodal e-skin. Herein, a bimodal capacitive e-skin capable of simultaneous tactile and proximity sensing has been developed. The entire process eliminates intricate fabrication techniques, employing DLP-3D printing for the electrode layers and sacrificial templating for the dielectric layers, conferring high tactile sensitivity (1.672 kPa-1) and rapid response capability (∼30 ms) to the bimodal e-skin. Moreover, exploiting the "fringing electric field" effect inherent in parallel-plate capacitors has facilitated touchless sensing, thereby enabling static distance recognition and dynamic gesture recognition of varying materials. Interestingly, an e-skin sensing array was created to identify the positions and pressure levels of various objects of different masses. Furthermore, with the aid of machine learning techniques, an artificial neural network has been established to possess intelligent object recognition capabilities, facilitating the identification, classification, and training of various object configurations. The advantages of the bimodal e-skin render it highly promising for extensive applications in the field of next-generation human-machine interaction.

Improved mKLT and low layered HG-CNN based dynamic gesture recognition hardware system

Improved mKLT and low layered HG-CNN based dynamic gesture recognition hardware system

High‐accuracy dynamic gesture recognition: A universal and self‐adaptive deep‐learning‐assisted system leveraging high‐performance ionogels‐based strain sensors

AbstractGesture recognition utilizing flexible strain sensors is a highly valuable technology widely applied in human–machine interfaces. However, achieving rapid detection of subtle motions and timely processing of dynamic signals remain a challenge for sensors. Here, highly resilient and durable ionogels are developed by introducing micro‐scale incompatible phases in macroscopic homogeneous polymeric network. The compatible network disperses in conductive ionic liquid to form highly resilient and stretchable skeleton, while incompatible phase forms hydrogen bonds to dissipate energy thus strengthening the ionogels. The ionogels‐derived strain sensors show highly sensitivity, fast response time (<10 ms), low detection limit (~50 μm), and remarkable durability (>5000 cycles), allowing for precise monitoring of human motions. More importantly, a self‐adaptive recognition program empowered by deep‐learning algorithms is designed to compensate for sensors, creating a comprehensive system capable of dynamic gesture recognition. This system can comprehensively analyze both the temporal and spatial features of sensor data, enabling deeper understanding of the dynamic process underlying gestures. The system accurately classifies 10 hand gestures across five participants with impressive accuracy of 93.66%. Moreover, it maintains robust recognition performance without the need for further training even when different sensors or subjects are involved. This technological breakthrough paves the way for intuitive and seamless interaction between humans and machines, presenting significant opportunities in diverse applications, such as human–robot interaction, virtual reality control, and assistive devices for the disabled individuals.

An Exosuit System With Bidirectional Hand Support for Bilateral Assistance Based on Dynamic Gesture Recognition.

Hand motor impairment has seriously affected the daily life of the elderly. We developed an electromyography (EMG) exosuit system with bidirectional hand support for bilateral coordination assistance based on a dynamic gesture recognition model using graph convolutional network (GCN) and long short-term memory network (LSTM). The system included a hardware subsystem and a software subsystem. The hardware subsystem included an exosuit jacket, a backpack module, an EMG recognition module, and a bidirectional support glove. The software subsystem based on the dynamic gesture recognition model was designed to identify dynamic and static gestures by extracting the spatio-temporal features of the patient's EMG signals and to control glove movement. The offline training experiment built the gesture recognition models for each subject and evaluated the feasibility of the recognition model; the online control experiments verified the effectiveness of the exosuit system. The experimental results showed that the proposed model achieve a gesture recognition rate of 96.42% ± 3.26 %, which is higher than the other three traditional recognition models. All subjects successfully completed two daily tasks within a short time and the success rate of bilateral coordination assistance are 88.75% and 86.88%. The exosuit system can effectively help patients by bidirectional hand support strategy for bilateral coordination assistance in daily tasks, and the proposed method can be applied to various limb assistance scenarios.

SimplyMime: A Dynamic Gesture Recognition and Authentication System for Smart Remote Control

SimplyMime: A Dynamic Gesture Recognition and Authentication System for Smart Remote Control

A Convolutional-Transformer-Based Approach for Dynamic Gesture Recognition of Data Gloves

A Convolutional-Transformer-Based Approach for Dynamic Gesture Recognition of Data Gloves

Traditional Cultural and Creative Product Design Methods Combining Digital Art Elements

Abstract As a mode of perpetuating and revitalizing traditional culture, cultural and creative products have garnered widespread affection and recognition from the public. In the context of evolving societal trends and advancements in science and technology, the digitization of traditional cultural and creative products has emerged as a prominent trend. This study undertakes the digital design of these products, primarily focusing on the innovative application of a style transfer algorithm to traditional motifs, supplemented by their visualization through digital platforms such as augmented reality (AR) and virtual reality (VR). Furthermore, it facilitates intelligent consumer interaction via gesture recognition algorithms, thereby enhancing user engagement and experience. During the implementation phase, this research conducts comparative analyses of style transfer and gesture recognition within digital cultural and creative products. It also employs the Kano questionnaire to categorize and analyze user needs effectively. Notably, while the recall rate of the style transfer algorithm documented in this study remains below 0.9, it consistently achieves high precision, significantly enhances feature extraction capabilities, and improves the quality of the style transfer effects produced. Moreover, the static gesture recognition algorithm achieves an impressive average recognition rate of 98.2%. The dynamic gesture recognition algorithm, meanwhile, maintains an average recognition rate of 94% with a processing time of 3.2 seconds, balancing the demands of real-time interaction and accuracy effectively. This study systematically analyzes the significance of each requirement element across six dimensions, classifying customer needs for digital cultural and creative products into four distinct categories. Additionally, it delineates a viable pathway for the integration of digital art elements into the design of artistic and innovative products, setting a foundation for future innovations in this field.

A real-time basketball referee gesture recognition system based on multi-scale spatio-temporal features

Abstract In this paper, the multi-scale spatio-temporal convolution module and the channel attention mechanism of global information synchronization have excellent performance in the field of sports gesture action recognition; therefore, combining the multi-scale spatio-temporal convolution and the channel attention mechanism to construct the dynamic gesture recognition model. The trained, dynamic gesture recognition model is deployed to cell phones and web pages through Java computational programming language, and the application scenarios of the basketball referee gesture recognition system are explained in detail. The real-time basketball referee gesture recognition system processes are analyzed using simulation analysis after determining the dataset. The basketball referee gesture recognition system’s video cropping frame analysis has an accuracy of 87.64% and 71.25%, with a loss value of 6.24 and 5.59, as shown in the results. When compared to the single-resolution features on the basketball referee gesture recognition system, the fused features exhibit some improvement, and the average recognition rate is 92.20%. This study is applicable to the referee gesture tracking method in complex basketball sports scenarios and lays the foundation for higher semantic level research in the field of sports video analytics, such as action recognition, event detection, and content understanding.