Approach For Gesture Recognition Research Articles

SILK-SVM: An Effective Machine Learning Based Key-Frame Extraction Approach for Dynamic Hand Gesture Recognition

SILK-SVM: An Effective Machine Learning Based Key-Frame Extraction Approach for Dynamic Hand Gesture Recognition

Real-Time Hand Gesture Monitoring Model Based on MediaPipe's Registerable System.

Hand gesture recognition plays a significant role in human-to-human and human-to-machine interactions. Currently, most hand gesture detection methods rely on fixed hand gesture recognition. However, with the diversity and variability of hand gestures in daily life, this paper proposes a registerable hand gesture recognition approach based on Triple Loss. By learning the differences between different hand gestures, it can cluster them and identify newly added gestures. This paper constructs a registerable gesture dataset (RGDS) for training registerable hand gesture recognition models. Additionally, it proposes a normalization method for transforming hand gesture data and a FingerComb block for combining and extracting hand gesture data to enhance features and accelerate model convergence. It also improves ResNet and introduces FingerNet for registerable single-hand gesture recognition. The proposed model performs well on the RGDS dataset. The system is registerable, allowing users to flexibly register their own hand gestures for personalized gesture recognition.

An extended variational autoencoder for cross-subject electromyograph gesture recognition

Surface electromyographic hand gesture recognition has gained significant attention in recent years, especially within the field of human–computer interfaces. However, cross-subject tasks remain challenging due to inherent individual differences. To address this, a novel approach for hand gesture recognition is proposed that leverages a subject-generalized variational autoencoder. This approach involves an extended variational autoencoder designed to disentangle input data into three distinct feature-specific representations. The primary classifier within the variational autoencoder focuses on gesture recognition, while two auxiliary classifiers work together to extract subject-specific and gesture-specific features. The gesture-specific features capture generalized characteristics applicable across all subjects, enabling direct application to new subjects. To enhance accuracy and stability, a competitive voting strategy is implemented. The effectiveness of the proposed method was evaluated using a dataset comprising six representative gestures performed by eight subjects. Comparative analysis with baseline models shows that our approach outperforms others, demonstrating superior generalization with an average accuracy of 90.52% in cross-subject validation.

Elbow Gesture Recognition with an Array of Inductive Sensors and Machine Learning.

This work presents a novel approach for elbow gesture recognition using an array of inductive sensors and a machine learning algorithm (MLA). This paper describes the design of the inductive sensor array integrated into a flexible and wearable sleeve. The sensor array consists of coils sewn onto the sleeve, which form an LC tank circuit along with the externally connected inductors and capacitors. Changes in the elbow position modulate the inductance of these coils, allowing the sensor array to capture a range of elbow movements. The signal processing and random forest MLA to recognize 10 different elbow gestures are described. Rigorous evaluation on 8 subjects and data augmentation, which leveraged the dataset to 1270 trials per gesture, enabled the system to achieve remarkable accuracy of 98.3% and 98.5% using 5-fold cross-validation and leave-one-subject-out cross-validation, respectively. The test performance was then assessed using data collected from five new subjects. The high classification accuracy of 94% demonstrates the generalizability of the designed system. The proposed solution addresses the limitations of existing elbow gesture recognition designs and offers a practical and effective approach for intuitive human-machine interaction.

Statement of Retraction: An Improved Approach for Human Movement Gesture Recognition Based on Grey Wolf Optimization and 3D Reconstruction

Statement of Retraction: An Improved Approach for Human Movement Gesture Recognition Based on Grey Wolf Optimization and 3D Reconstruction

A Review on Sign Language Recognition and Learning System

This paper presents innovative approaches to various aspects of sign language recognition and learning, catering to the diverse needs within this field. It introduces a system utilizing 2D image sampling and concatenation, trained with convolutional neural networks, achieving accurate and robust sign recognition even with low-cost cameras. Another system, SignQuiz, offers a cost-effective web-based solution for learning finger-spelled signs in Indian Sign Language (ISL), outperforming traditional printed mediums. Additionally, a dynamic hand gesture recognition system employing deep learning architectures demonstrates superior performance over existing methods, addressing challenges such as hand segmentation and sequence modeling. Furthermore, an efficient deep convolutional neural network approach for hand gesture recognition, including transfer learning, achieves high recognition rates across various datasets. Finally, a robust model for static sign recognition in ISL, utilizing CNNs and evaluating on a diverse dataset, attains exceptional accuracy and outperforms previous works. These contributions collectively advance the field of sign language recognition and learning, offering solutions that are accurate, cost-effective, and efficient, thereby facilitating better communication and interaction for the hearing-impaired community. Key Words: CNN, Sign language recognition, Hand gesture recognition, Machine learning, Transfer Learning, SVM.

TRANS-CNN-Based Gesture Recognition for mmWave Radar.

In order to improve the real-time performance of gesture recognition by a micro-Doppler map of mmWave radar, the point cloud based gesture recognition for mmWave radar is proposed in this paper. Two steps are carried out for mmWave radar-based gesture recognition. The first step is to estimate the point cloud of the gestures by 3D-FFT and the peak grouping. The second step is to train the TRANS-CNN model by combining the multi-head self-attention and the 1D-convolutional network so as to extract the features in the point cloud data at a deeper level to categorize the gestures. In the experiments, TI mmWave radar sensor IWR1642 is used as a benchmark to evaluate the feasibility of the proposed approach. The results show that the accuracy of the gesture recognition reaches 98.5%. In order to prove the effectiveness of our approach, a simply 2Tx2Rx radar sensor is developed in our lab, and the accuracy of recognition reaches 97.1%. The results show that our proposed gesture recognition approach achieves the best performance in real time with limited training data in comparison with the existing methods.

Frequency and Spatial Domain-Based Approaches for Recognition of Indian Sign Language Gestures

Objectives: The objective of this paper is to introduce and demonstrate an innovative approach for the recognition of Indian sign language gestures, with a focus on bridging communication gap between the deaf and hearing communities. The goal is to contribute to the development of effective tools and technologies that facilitate seamless communication between individuals using sign language and the people with no knowledge about sign language. Methods: The methodology consists of three key steps. First, data pre-processing involves resizing and contours extraction. Next, feature extraction employs Fourier descriptors for frequency domain analysis and gray-level-co-occurrence matrix for spatial domain analysis. Finally, various machine learning models including SVM, Random Forest, Logistic Regression, K-Nearest Neighbor and Naive Bayes are trained on a standard dataset. Findings: In our controlled experimental setup, we applied a diverse set of machine learning classifiers to evaluate the proposed approach for gesture recognition. Among the classifiers tested, K-Nearest Neighbors demonstrated the highest accuracy, achieving 99.82%. To validate the robustness of our approach, we employed k-fold cross-validation with 5 folds. Novelty: This study presents an innovative method for sign language recognition by employing a dual-domain fusion strategy that prominently emphasizes the frequency domain. Through the integration of Fourier descriptors, the research conducts a detailed frequency domain analysis to characterize the contour shapes of sign language gestures. The synergy with gray-level co-occurrence matrix texture features in the spatial domain analysis, contributes to the creation of a comprehensive feature vector. The proposed approach ensures a thorough exploration of gesture features, there by advancing the precision and efficacy of sign language recognition. Keywords: Indian Sign Language (ISL), Sign Language Recognition (SLR), Frequency domain, Spatial domain, Fourier descriptors, Gray level co­occurrence matrix (GLCM), K­ Fold

Point cloud based hand gesture recognition using template matching

Hand gesture recognition is important in human-computer interaction with wide applications in many fields. Different from common hand gesture recognition based on 2D images acquired from RGB camera, the utilization of 3D images provides additional spatial information about the target and attracts more and more attention in hand gesture recognition. However, most 3D images for hand gesture recognition are based on depth maps, which only take the distance information as a channel of 2D images, without taking full use of the 3D information. Besides, greater data volume of 3D images brings challenges to the arithmetic facility of hand gesture recognition. Here, we proposed a point cloud based method for hand gesture recognition. To fully use the 3D information, plane points for template matching were extracted based on their normal distributions, which leads to the average recognition rate over 97%. Pre-classification was implemented to ensure a high-efficient recognition without additional requirements for the computer. The proposed method may provide approach for accurate and efficient hand gesture recognition based on 3D images.

A Convolutional-Transformer-Based Approach for Dynamic Gesture Recognition of Data Gloves

A Convolutional-Transformer-Based Approach for Dynamic Gesture Recognition of Data Gloves

Rejecting Unknown Gestures Based on Surface-Electromyography Using Variational Autoencoder.

The conventional surface electromyography (sEMG)-based gesture recognition systems exhibit impressive performance in controlled laboratory settings. As most systems are trained in a closed-set setting, the systems's performance may see significant deterioration when novel gestures are presented as imposter. In addition, the state-of-the-art generative and discriminative methods have achieved considerable performance on high-density sEMG signals. This can be seen as an unrealistic setting as the real-world muscle computer interface are mainly comprised of sparse multichannel sEMG signals. In this work, we propose a novel variational autoencoder based approach for open-set gesture recognition based on sparse multichannel sEMG signals. Using the predefined corresponding latent conditional distribution of known gestures, the conditional Gaussian distribution of each known gesture is learned. Those samples with low probability density are identified as unknown gestures. The sEMG signals of known gestures are classified using the Kullback-Leibler divergences between the predefined prior distributions and input samples. The proposed approach is evaluated using three benchmark sparse multichannel sEMG databases. The experimental results demonstrate that our approach outperforms the existing open-set sEMG-based gesture recognition approaches and achieves a better trade-off between classifying known gestures and rejecting unknown gestures.

EchoGest: Soft Ultrasonic Waveguides based Sensing Skin for Subject-Independent Hand Gesture Recognition.

Gesture recognition is crucial for enhancing human-computer interaction and is particularly pivotal in rehabilitation contexts, aiding individuals recovering from physical impairments and significantly improving their mobility and interactive capabilities. However, current wearable hand gesture recognition approaches are often limited in detection performance, wearability, and generalization. We thus introduce EchoGest, a novel hand gesture recognition system based on soft, stretchable, transparent artificial skin with integrated ultrasonic waveguides. Our presented system is the first to use soft ultrasonic waveguides for hand gesture recognition. Ecoflex™ 00-31 and Ecoflex™ 00-45 Near Clear™ silicone elastomers were employed to fabricate the artificial skin and ultrasonic waveguides, while 0.1 mm diameter silver-plated copper wires connected the transducers in the waveguides to the electrical system. The wires are enclosed within an additional elastomer layer, achieving a sensing skin with a total thickness of around 500 μm. Ten participants wore the EchoGest system and performed static hand gestures from two gesture sets: 8 daily life gestures and 10 American Sign Language (ASL) digits 0-9. Leave-One-Subject-Out Cross-Validation analysis demonstrated accuracies of 91.13% for daily life gestures and 88.5% for ASL gestures. The EchoGest system has significant potential in rehabilitation, particularly for tracking and evaluating hand mobility, which could substantially reduce the workload of therapists in both clinical and home-based settings. Integrating this technology could revolutionize hand gesture recognition applications, from real-time sign language translation to innovative rehabilitation techniques.

RETRACTED ARTICLE: An Improved Approach for Human Movement Gesture Recognition Based on Grey Wolf Optimization and 3D Reconstruction

Statement of Retraction: We, the Editors and Publisher of International Journal of Human Computer — Interaction, have retracted the following article: Mingyang Han (2023) An Improved Approach for Human Movement Gesture Recognition Based on Grey Wolf Optimization and 3D Reconstruction, International Journal of Human — Computer Interaction, DOI: https://doi.org/10.1080/10447318.2023.2277487 Since publication, significant concerns have been raised about the fact that this article has substantial overlap with the following article: Mustafa Z, Nsour H, Tahir SBud. 2023. Hand gesture recognition via deep data optimization and 3D reconstruction. PeerJ Computer Science 9:e1619 https://doi.org/10.7717/peerj-cs.1619 The author has not responded to our queries about this significant level of overlap. As this is a serious breach of our Editorial Policies, we are retracting the article from the journal. We have been informed in our decision-making by our editorial policies and the COPE guidelines. The retracted article will remain online to maintain the scholarly record, but it will be digitally watermarked on each page as &‘Retracted&’.

Diffusion Convolution Neural Network-Based Multiview Gesture Recognition for Athletes in Dynamic Scenes

This paper focuses on deep vision sensing-assisted gesture recognition for athletes in dynamic scenes. Although many research attention had been devoted to this field in recent years, most of existing works failed to fully take characteristics of dynamic scenes into consideration. To deal with this challenge, this paper proposes a diffusion convolution neural network-based multiview gesture recognition approach in dynamic scenes. For one thing, the dynamic spatiotemporal slice position selection based on the body mask heatmap is adopted to calculate positions of horizontal and vertical slices. Thus, the dynamic selection of slice positions in two directions can be realized, and then the extraction of bi-directional spatiotemporal slice images can be completed. For another, action sequences through the 3D residual neural network are learned, and the spatiotemporal information among frames are mined through recurrent networks. Through their combination, a multi-view gesture recognition approach for athletes is constructed. In the experiments, two standard datasets UCF101 and HMDB51 are utilized to establish simulation environment. The proposed method can reach the accuracy beyond 95% on the two datasets. Compared with several typical recognition methods, the proposed method shows higher accuracy.

A novel sEMG-based dynamic hand gesture recognition approach via residual attention network

A novel sEMG-based dynamic hand gesture recognition approach via residual attention network

SBI-DHGR: Skeleton-based intelligent dynamic hand gestures recognition

Hand gesture recognition (HGR) plays a significant role in interpreting the meaning of sign language, human–computer interaction, and robot control. This paper proposes a real-time skeleton-based intelligent dynamic hand gesture recognition (SBI-DHGR) approach, which comprises three modules: Palm Centroid (PC), Data augmentation with the Tenet Effect, and Deep learning architecture. The palm Centroid (PC) module is introduced to identify the proposed palm center joint. Data augmentation with a tenet effect module is designed to improve the CNN model’s generalizability. Further, a novel deep learning model is proposed for temporal 3D-HGR by exploiting the capabilities of a multi-channel convolutional neural network (CNN) and long short-term memory (LSTM) recurrent network. The multi-channel CNN is introduced to learn the cardinal positions of hand joints by capturing the low, average, and high-level features. LSTM is embedded to learn the temporal characteristics of hand joints. The effectiveness of the SBI-DHGR framework is evaluated over five challenging datasets: SHREC-14, SHREC-28, DHG-14, DHG-28, and FPHA, by adopting person-dependent and person-independent validation setups.

Human Gesture Recognition Based on Millimeter-Wave Radar Using Improved C3D Convolutional Neural Network

<p>In this paper, we propose a time sequential IC3D convolutional neural network approach for hand gesture recognition based on frequency modulated continuous wave (FMCW) radar. Firstly, the FMCW radar is used to collect the echoes of human hand gestures. A two-dimensional fast Fourier transform calculates the range and velocity information of hand gestures in each frame signal to construct the Range-Doppler heat map dataset of hand gestures. Then, we design an IC3D network for feature extraction and classification of the dynamic gesture heat map. Finally, the experiment results show that the gesture recognition system designed in this paper effectively solves the problems of the difficulty of human gesture feature extraction and low utilization of time series information, and the average recognition accuracy rate can reach more than 99.8%.</p> <p> </p>

An Approach for Gesture Recognition Based on a Lightweight Convolutional Neural Network

Gesture recognition, which plays an important role to understand meaningful movements of human bodies, is one of the most effective approaches for humans to interact. Sign language is a fundamental and innate means of communication for hearing-impaired individuals. Though significant progress has been made, the state-of-the-art gesture recognition methods yield week performance for conditions with dynamic gestures in videos. Thus, robust gesture recognition remains a challenging issue because of many barriers of gesture-irrelevant factors. The key to robust gesture recognition is to learn effective and concise spatiotemporal information. Inspired by the great promise of the convolutional neural network (CNN) and its breakthroughs, we introduce an approach for identifying static alphabet gestures in the American Sign Language (ASL). The proposed CNN-based approach has been developed to classify letters of the alphabet from A to Z. It composes of three phases: a preprocessing phase for extracted of the region of interest, a feature extraction, and a classification phase. The performance of proposed gesture recognition approach is evaluated on the common ASL dataset and it achieves 94.83% accuracy, which is good enough to develop a strong translator from gesture-based ASL to spoken language as it is capable to handle a variety of 24 hand gestures.

Multi-Scopic Cognitive Memory System for Continuous Gesture Learning.

With the advancement of artificial intelligence technologies in recent years, research on intelligent robots has progressed. Robots are required to understand human intentions and communicate more smoothly with humans. Since gestures can have a variety of meanings, gesture recognition is one of the essential issues in communication between robots and humans. In addition, robots need to learn new gestures as humans grow. Moreover, individual gestures vary. Because catastrophic forgetting occurs in training new data in traditional gesture recognition approaches, it is necessary to preserve the prepared data and combine it with further data to train the model from scratch. We propose a Multi-scopic Cognitive Memory System (MCMS) that mimics the lifelong learning process of humans and can continuously learn new gestures without forgetting previously learned gestures. The proposed system comprises a two-layer structure consisting of an episode memory layer and a semantic memory layer, with a topological map as its backbone. The system is designed with reference to conventional continuous learning systems in three ways: (i) using a dynamic architecture without setting the network size, (ii) adding regularization terms to constrain learning, and (iii) generating data from the network itself and performing relearning. The episode memory layer clusters the data and learns their spatiotemporal representation. The semantic memory layer generates a topological map based on task-related inputs and stores them as longer-term episode representations in the robot's memory. In addition, to alleviate catastrophic forgetting, the memory replay function can reinforce memories autonomously. The proposed system could mitigate catastrophic forgetting and perform continuous learning by using both machine learning benchmark datasets and real-world data compared to conventional methods.

MEMS based eating and drinking gesture spotting using machine learning techniques

An effective human-computer interaction for collaborative, collective and brilliant computing is provided by the Gesture recognition. Our study uses a single 3-axis accelerometer and 3-axis gyroscope for acquiring the data. Dynamic Time Warping (DTW), time and frequency domain and gesture discrepancy features are extracted from the raw data and these features are subjected to supervised machine learning algorithms such as KNN, SVM, RF and ANN. The system is evaluated depending on the data base of more than 10,000 traces obtained from five subsets. Through this study, we intend to identify eating and drinking gestures and more broadly short activities on real time environment. Many approaches for gesture recognition in specific settings have already been explored and studied using the combined interaction of several sensors. Despite these powerful hypotheses, gesture interpretation is still fragile and often depends on the individual’s positioning relative to the cameras not with the sensors.