Continuous Hand Gesture Recognition Research Articles

Continuous hand gesture segmentation and acknowledgement of hand gesture path for innovative effort interfaces

Human-computer interaction (HCI) has revolutionized the way we interact with computers, making it more intuitive and user-friendly. It is a dynamic field that has found it is applications in various industries, including multimedia and gaming, where hand gestures are at the forefront. The advent of ubiquitous computing has further heightened the interest in using hand gestures as input. However, recognizing continuous hand gestures presents a set of challenges, primarily stemming from the variable duration of gestures and the lack of clear starting and ending points. Our main objective is to propose a solution: the framework for “continuous palm motion analysis and retrieval” based on “Spatial-temporal and path knowledge”. Framework harnesses the power of cognitive deep learning networks (DLN), offering a significant advancement in the continuous hand gesture recognition domain. we conducted rigorous experiments using a diverse video dataset capturing hand gestures for boasting an impressive F-score of up to 0.99. The potential of our framework to significantly enhance the accuracy and reliability of hand gesture recognition in real-world applications.

Hand Gestures Recognition Model Using Adaptive Feature Extraction with Attention-Based Hybrid Deep Learning via Optimization Strategy

Research on continuous hand gesture recognition is motivated by applications of deaf people to provide effective communication between one another. Numerous techniques are adapted for effectively identifying the hand gesture. Moreover, the collection of data becomes a challenging task in the existing models. In some instances, hand gesture in low-resolution images becomes a complicated task. In this research work, a deep learning-based hand gesture detection model is designed using hand data points to uncover patterns that are used to convey information. Initially, hand gesture images are obtained from public sources. The collected inputs are preprocessed through Contrast-Limited Adaptive Histogram Equalization and filtering to eliminate the unnecessary blurs in the images. The preprocessed images undergo hand segmentation using four methods color space transformation, skin color detection, active contour, and morphological operation. The segmented image is subjected to data point feature extraction, where the Adaptive Weighted Scale-Invariant Feature Transform is utilized for further enhancement. The hand data point-extracted features are given to the recognition of hand gestures with the support of an attention-based hybrid network, where the Attention-based Hybrid 1D Convolutional Neural Network with Recurrent Neural Network recognizes the hand gestures from the hand data points. In the hybridization network, the values are optimized with the help of the investigated Adaptive Dandelion White Shark Optimizer to enhance recognition effectiveness. The test results are validated with the existing hand gesture recognition models using diverse evaluation metrics. The findings of the recommended method show 97% and 99% in terms of accuracy and NPV. Thus, the developed model is validated to outperform significant performance rather than the existing models.

Soft Wrist-Worn Multi-Functional Sensor Array for Real-Time Hand Gesture Recognition

Soft electronics have been widely applied to wearable electronics, hand gesture detection and human robot interaction (HRI). Soft wrist-worn sensor system (SWSS) with electromyography (EMG), strain/pressure sensing abilities has been created for continuous hand gesture recognition, which includes multi-source data sensing, data collection and process, and wireless communication. SWSS is conformal contact with skin surface as the softness of SWSS, and it would improve the wearability with the wrist during long-term hand gesture monitoring. Linear discriminant analysis (LDA) and support vector machines (SVM) algorithms are applied to hand gesture recognition with average accuracy 83.67%, 86.8% for Group #1 (wrist flexion (WF), wrist extension (WE), finger flexion (FF), finger extension (FE), radial deviation (RD), and ulnar deviation (UD)) and 84.71%, 88.53% for Group #2 (sign language 0–9 digits), respectively. It is found that the recognition accuracy has just a little decreased during the long-term hand gesture detection at different sessions (at initial time, one day after, three days after and seven days after). This paper demonstrated the feasibility of gesture recognition via SWSS on wrist, and it could be integrated into wrist-worn electronic system for human robot interaction.

A Novel Detection and Recognition Method for Continuous Hand Gesture Using FMCW Radar

In this article, a novel method for continuous hand gesture detection and recognition is proposed based on a frequency modulated continuous wave (FMCW) radar. Firstly, we adopt the 2-Dimensional Fast Fourier Transform (2D-FFT) to estimate the range and Doppler parameters of the hand gesture raw data, and construct the range-time map (RTM) and Doppler-time map (DTM). Meanwhile, we apply the Multiple Signal Classification (MUSIC) algorithm to calculate the angle and construct the angle-time map (ATM). Secondly, a hand gesture detection method is proposed to segment the continuous hand gestures using a decision threshold. Thirdly, the central time-frequency trajectory of each hand gesture spectrogram is clustered using the k-means algorithm, and then the Fusion Dynamic Time Warping (FDTW) algorithm is presented to recognize the hand gestures. Finally, experiments show that the accuracy of the proposed hand gesture detection method can reach 96.17%. The hand gesture average recognition accuracy of the proposed FDTW algorithm is 95.83%, while its time complexity is reduced by more than 50%.

Sensor-Based Continuous Hand Gesture Recognition by Long Short-Term Memory

This article aims to present a novel sensor-based continuous hand gesture recognition algorithm by long short-term memory (LSTM). Only the basic accelerators and/or gyroscopes are required by the algorithm. Given a sequence of input sensory data, a many-to-many LSTM scheme is adopted to produce an output path. A maximum a posteriori estimation is then carried out based on the observed path to obtain the final classification results. A prototype system based on smartphones has been implemented for the performance evaluation. Experimental results show that the proposed algorithm is an effective alternative for robust and accurate hand-gesture recognition.

Latern: Dynamic Continuous Hand Gesture Recognition Using FMCW Radar Sensor

Recently, hand gesture recognition systems have become increasingly interesting to researchers in the field of human–computer interfaces. Real-world systems for human dynamic hand gesture recognition is challenging as: 1) the system must be robust to various conditions; 2) there is a rich diversity in how people perform hand gestures, making hand gesture recognition difficult; and 3) the system must detect and recognize hand gestures continuously using unsegmented input streams in order to avoid noticeable lag between performing a gesture and its classification. In this paper, to address these challenges, we present Latern, a novel system for dynamic continuous hand gesture recognition based on a frequency-modulated continuous wave radar sensor. The radar system does not depend on lighting, noise, or atmospheric conditions. We employ a recurrent 3-D convolutional neural network to perform the classification of dynamic hand gestures. To enhance the processing performance, a connectionist temporal classification algorithm is used to train the network to predict class labels from inprogress gestures in unsegmented input streams. The experimental results show that Latern is able to achieve high recognition rates of 96%, which is higher than state-of-the-art hand gesture recognition systems. In addition, the conclusion in this paper can be used for a real-time hand gesture recognition system design.

Structured dynamic time warping for continuous hand trajectory gesture recognition

Continuous hand gesture recognition is an important area of HCI and challenged by various writing habits and unconstrained hand movement. In this paper, we propose a Structured Dynamic Time Warping (SDTW) approach for continuous hand trajectory recognition. We first propose an automatic continuous trajectory segmentation approach which combines templates and velocity information to spot the beginning and ending points in hand gesture trajectories. Then we assign different weights to feature sequences based on the structured information, from the positions of corner points in the arbitrary trajectories. Finally, we evaluate the SDTW on the Continuous Letter Trajectory (CLT) database. Experimental results show that the proposed approach is robust to the diversity of same handwritten letter, and significantly outperforms state-of-the-art approaches.

Continuous hand gesture recognition based on trajectory shape information

In this paper, we propose a continuous hand gesture recognition method based on trajectory shape information. A key issue in recognizing continuous gestures is that performance of conventional recognition algorithms may be lowered by such factors as, unknown start and end points of a gesture or variations in gesture duration. These issues become particularly difficult for those methods that rely on temporal information. To alleviate the issues of continuous gesture recognition, we propose a framework that simultaneously performs both segmentation and recognition. Each component of the framework applies shape-based information to ensure robust performance for gestures with large temporal variation. A gesture trajectory is divided by a set of key frames by thresholding its tangential angular change. Variable-sized trajectory segments are then generated using the selected key frames. For recognition, these trajectory segments are examined to determine whether the segment belongs to a class among intended gestures or a non-gesture class based on fusion of shape information and temporal features. In order to assess performance, the proposed algorithm was evaluated with a database of digit hand gestures. The experimental results indicate that the proposed algorithm has a high recognition rate while maintaining its performance in the presence of continuous gestures.

Survey on 3D Hand Gesture Recognition

Three-dimensional hand gesture recognition has attracted increasing research interests in computer vision, pattern recognition, and human-computer interaction. The emerging depth sensors greatly inspired various hand gesture recognition approaches and applications, which were severely limited in the 2D domain with conventional cameras. This paper presents a survey of some recent works on hand gesture recognition using 3D depth sensors. We first review the commercial depth sensors and public data sets that are widely used in this field. Then, we review the state-of-the-art research for 3D hand gesture recognition in four aspects: 1) 3D hand modeling; 2) static hand gesture recognition; 3) hand trajectory gesture recognition; and 4) continuous hand gesture recognition. While the emphasis is on 3D hand gesture recognition approaches, the related applications and typical systems are also briefly summarized for practitioners.

A novel set of features for continuous hand gesture recognition

Applications requiring the natural use of the human hand as a human–computer interface motivate research on continuous hand gesture recognition. Gesture recognition depends on gesture segmentation to locate the starting and end points of meaningful gestures while ignoring unintentional movements. Unfortunately, gesture segmentation remains a formidable challenge because of unconstrained spatiotemporal variations in gestures and the coarticulation and movement epenthesis of successive gestures. Furthermore, errors in hand image segmentation cause the estimated hand motion trajectory to deviate from the actual one. This research moves toward addressing these problems. Our approach entails using gesture spotting to distinguish meaningful gestures from unintentional movements. To avoid the effects of variations in a gesture’s motion chain code (MCC), we propose instead to use a novel set of features: the (a) orientation and (b) length of an ellipse least-squares fitted to motion-trajectory points and (c) the position of the hand. The features are designed to support classification using conditional random fields. To evaluate the performance of the system, 10 participants signed 10 gestures several times each, providing a total of 75 instances per gesture. To train the system, 50 instances of each gesture served as training data and 25 as testing data. For isolated gestures, the recognition rate using the MCC as a feature vector was only 69.6 % but rose to 96.0 % using the proposed features, a 26.1 % improvement. For continuous gestures, the recognition rate for the proposed features was 88.9 %. These results show the efficacy of the proposed method.

A novel approach for dynamic hand gesture recognition using contour-based similarity images

A novel approach is proposed for the recognition of moving hand gestures based on the representation of hand motions as contour-based similarity images (CBSIs). The CBSI was constructed by calculating the similarity between hand contours in different frames. The input CBSI was then matched with CBSIs in the database to recognize the hand gesture. The proposed continuous hand gesture recognition algorithm can simultaneously divide the continuous gestures into disjointed gestures and recognize them. No restrictive assumptions were considered for the motion of the hand between the disjointed gestures. The proposed algorithm was tested using hand gestures from American Sign Language and the results showed a recognition rate of 91.3% for disjointed gestures and 90.4% for continuous gestures. The experimental results illustrate the efficiency of the algorithm for noisy videos.

A COLORED FINGER TIP-BASED TRACKING METHOD FOR CONTINUOUS HAND GESTURE RECOGNITION

Hand gesture recognition system can be used for human-computer interaction (HCI). Proper hand segmentation from the background and other body parts of the video is the primary requirement for the design of a hand-gesture based application. These video frames can be captured from a low cost webcam (camera) for use in a vision based gesture recognition technique. This paper discusses about the continuous hand gesture recognition. The aim of this paper is to report a robust and efficient hand segmentation algorithm where a new method, wearing glove on the hand is utilized. After that a new idea called “Finger-Pen”, is developed by segmenting only one finger from the hand for proper tracking. In this technique only a finger tip is segmented in spite of the full hand part. Hence this technique allows the hand (excepting the segmented finger tip) to move freely during the tracking time also. Problems such as skin colour detection, complexity from large numbers of people in front of the camera, complex background removal and variable lighting condition are found to be efficiently handled by the system. Noise present in the segmented image due to dynamic background can be removed with the help of this adaptive technique which is found to be effective for the application conceived.