Robust Hand Gesture Recognition Research Articles

Robust Hand Gesture Recognition Using HOG Features and machine learning

Robust Hand Gesture Recognition Using HOG Features and machine learning

Innovative healthcare solutions: robust hand gesture recognition of daily life routines using 1D CNN.

Hand gestures are an effective communication tool that may convey a wealth of information in a variety of sectors, including medical and education. E-learning has grown significantly in the last several years and is now an essential resource for many businesses. Still, there has not been much research conducted on the use of hand gestures in e-learning. Similar to this, gestures are frequently used by medical professionals to help with diagnosis and treatment. We aim to improve the way instructors, students, and medical professionals receive information by introducing a dynamic method for hand gesture monitoring and recognition. Six modules make up our approach: video-to-frame conversion, preprocessing for quality enhancement, hand skeleton mapping with single shot multibox detector (SSMD) tracking, hand detection using background modeling and convolutional neural network (CNN) bounding box technique, feature extraction using point-based and full-hand coverage techniques, and optimization using a population-based incremental learning algorithm. Next, a 1D CNN classifier is used to identify hand motions. After a lot of trial and error, we were able to obtain a hand tracking accuracy of 83.71% and 85.71% over the Indian Sign Language and WLASL datasets, respectively. Our findings show how well our method works to recognize hand motions. Teachers, students, and medical professionals can all efficiently transmit and comprehend information by utilizing our suggested system. The obtained accuracy rates highlight how our method might improve communication and make information exchange easier in various domains.

A Hybrid Image Augmentation Technique for User- and Environment-Independent Hand Gesture Recognition Based on Deep Learning

This research stems from the increasing use of hand gestures in various applications, such as sign language recognition to electronic device control. The focus is the importance of accuracy and robustness in recognizing hand gestures to avoid misinterpretation and instruction errors. However, many experiments on hand gesture recognition are conducted in limited laboratory environments, which do not fully reflect the everyday use of hand gestures. Therefore, the importance of an ideal background in hand gesture recognition, involving only the signer without any distracting background, is highlighted. In the real world, the use of hand gestures involves various unique environmental conditions, including differences in background colors, varying lighting conditions, and different hand gesture positions. However, the datasets available to train hand gesture recognition models often lack sufficient variability, thereby hindering the development of accurate and adaptable systems. This research aims to develop a robust hand gesture recognition model capable of operating effectively in diverse real-world environments. By leveraging deep learning-based image augmentation techniques, the study seeks to enhance the accuracy of hand gesture recognition by simulating various environmental conditions. Through data duplication and augmentation methods, including background, geometric, and lighting adjustments, the diversity of the primary dataset is expanded to improve the effectiveness of model training. It is important to note that the utilization of the green screen technique, combined with geometric and lighting augmentation, significantly contributes to the model’s ability to recognize hand gestures accurately. The research results show a significant improvement in accuracy, especially with implementing the proposed green screen technique, underscoring its effectiveness in adapting to various environmental contexts. Additionally, the study emphasizes the importance of adjusting augmentation techniques to the dataset’s characteristics for optimal performance. These findings provide valuable insights into the practical application of hand gesture recognition technology and pave the way for further research in tailoring techniques to datasets with varying complexities and environmental variations.

HGR-FYOLO: a robust hand gesture recognition system for the normal and physically impaired person using frozen YOLOv5

HGR-FYOLO: a robust hand gesture recognition system for the normal and physically impaired person using frozen YOLOv5

A novel feature fusion technique for robust hand gesture recognition

A novel feature fusion technique for robust hand gesture recognition

Multi-modal fusion for robust hand gesture recognition based on heterogeneous networks

Multi-modal fusion for robust hand gesture recognition based on heterogeneous networks

Hand Gesture Recognition

Abstract: This work presents a computer-vision-based application for recognizing hand gestures. A live video feed is captured by a camera, and a still image is extracted from that feed with the aid of an interface. At least once per count hand gesture (one, two, three, four, and five), the system is trained. After that, the system is given a test gesture to see if it can identify it. Several algorithms that are capable of distinguishing a hand gesture were studied. It was determined that the highest rate of accuracy was achieved by using the computational neural network known as the Alexnet algorithm. Traditionally, systems have used data gloves or markers as a means of input. We are free to use the system however we like. In this way, the user can make natural hand gestures in front of the camera. The system implemented serves as an extendable basis for future work toward a fully robust hand gesture recognition system, which is still the subject of intensive research and development.

An Efficient and Robust Hand Gesture Recognition System of Sign Language Employing Finetuned Inception-V3 and Efficientnet-B0 Network

Hand Gesture Recognition (HGR) is a promising research area with an extensive range of applications, such as surgery, video game techniques, and sign language translation, where sign language is a complicated structured form of hand gestures. The fundamental building blocks of structured expressions in sign language are the arrangement of the fingers, the orientation of the hand, and the hand’s position concerning the body. The importance of HGR has increased due to the increasing number of touchless applications and the rapid growth of the hearing-impaired population. Therefore, real-time HGR is one of the most effective interaction methods between computers and humans. Developing a user-free interface with good recognition performance should be the goal of real-time HGR systems. Nowadays, Convolutional Neural Network (CNN) shows great recognition rates for different image-level classification tasks. It is challenging to train deep CNN networks like VGG-16, VGG-19, Inception-v3, and Efficientnet-B0 from scratch because only some significant labeled image datasets are available for static hand gesture images. However, an efficient and robust hand gesture recognition system of sign language employing finetuned Inception-v3 and Efficientnet-Bo network is proposed to identify hand gestures using a comparative small HGR dataset. Experiments show that Inception-v3 achieved 90% accuracy and 0.93% precision, 0.91% recall, and 0.90% f1-score, respectively, while EfficientNet-B0 achieved 99% accuracy and 0.98%, 0.97%, 0.98%, precision, recall, and f1-score respectively.

Hand Gesture Detection and Recognition

The design introduces an operation using computer vision for Hand gesture recognition. A camera records a live videotape sluice, from which a shot is taken with the help of interface. The system is trained for each type of count hand gestures (one, two, three, four, and five) at least formerly. After that a test gesture is given to it and the system tries to fete it. A exploration was carried out on a number of algorithms that could best separate a hand gesture. It was set up that the slant sum algorithm gave the loftiest delicacy rate. In the preprocessing phase, a tone- developed algorithm removes the background of each training gesture. After that the image is converted into a double image and the totalities of all slant rudiments of the picture are taken. This sum helps us in secerning and classifying different hand gestures. former systems have used data gloves or labels for input in the system. I've no similar constraints for using the system. The stoner can give hand gestures in view of the camera naturally. A fully robust hand gesture recognition system is still under heavy exploration and development; the enforced system serves as an extendible foundation for unborn work.

Robust Dynamic Hand Gesture Recognition Based on Millimeter Wave Radar Using Atten-TsNN

The millimeter-wave radar can sense the subtle movement of the hand. However, the traditional hand gesture recognition methods do not work well in dynamic interference scenarios. To address this issue, a robust hand gesture recognition method is proposed based on the self-attention time-series neural networks. Firstly, the original radar echo is constructed in terms of frame, sequence, and channel at the input terminal of the network. A one-dimensional time-series neural network is built, and the time-distributed layer is used as the wrapper to extract the feature from each frame sequence independently. Then the self-attention mechanism is employed to assign the adequate weights to the sequence of frames entered in parallel to obtain the inter-frame correlation and suppress the random interference. Finally, the Global AvgPooling layer is used to reduce the number of channels, and the fully connected layer outputs the label of the gesture. The experimental results show that the proposed method can achieve a high recognition rate in the presence of random dynamic interference.

Automatic and robust hand gesture recognition by SDD features based model matching

Automatic and robust hand gesture recognition by SDD features based model matching

A Robust Hand Gesture Sensing and Recognition Based on Dual-Flow Fusion With FMCW Radar

For hand gesture recognition (HGR) sensing based on frequency modulated continuous wave (FMCW) radar, it is challenging to cope with the ambiguity and variability of gesture motions performed by different peoples due to the private habits or position, which inevitably reduce the robustness of HGR sensing. To solve this problem, we propose a novel dual-flow fusion deformable residual network (DFDRN). More detailed gesture information, time frequency maps and spectrum map videos, are fed into DFDRN, which improves the generalization and motion modeling ability of HGR through the extensive feature space. To further improve the robustness of HGR sensing, the deformable block is exploited to replace the convolution. To focus on the important features of dual-flow fusion, the convolutional block attention module (CBAM) is embedded in the backbone of DFDRN. Experimental results show that the accuracy of DFDRN, which can reach 97.5%, is obviously better than the state-of-the-art HGR methods.

Robust Hand Gesture Recognition Based on RGB-D Data for Natural Human–Computer Interaction

To naturally interact with virtual environment by hand gesture, this paper presents a robust RGB-D data based recognition method of static and dynamic hand gesture. Firstly, for static hand gesture recognition, starting from the hand gesture contour extraction, the palm center is identified by Distance Transform (DT) algorithm. The fingertips are localized by employing the K-Curvature-Convex Defects Detection algorithm (K-CCD). On the basis, the distances of the pixels on hand gesture contour to palm center and the angle between fingertips are considered as the auxiliary features to construct a multimodal feature vector, and then recognition algorithm is presented to robustly recognize the static hand gestures. Secondly, combining Euclidean distance between hand joints and shoulder center joint with the modulus ratios of skeleton features, this paper generates a unifying feature descriptor for each dynamic hand gesture and proposes an improved dynamic time warping (IDTW) algorithm to obtain recognition results of dynamic hand gestures. Finally, we conduct extensive experiments to test and verify the static and dynamic hand gesture recognition algorithm and realize a low-cost real-time application of natural interaction with virtual environment by hand gestures.

Robust Hand Gestures Recognition Using a Deep CNN and Thermal Images

Medical systems and assistive technologies, human-computer interaction, human-robot interaction, industrial automation, virtual environment control, sign language translation, crisis and disaster management, entertainment and computer games, and so on all use RGB cameras for hand gesture recognition. However, their performance is limited especially in low-light conditions. In this paper, we propose a robust hand gesture recognition system based on high-resolution thermal imaging that is light-independent. A dataset of 14,400 thermal hand gestures is constructed, separated into two color tones. We also propose using a deep CNN to classify high-resolution hand gestures accurately. The proposed models were also tested on Raspberry Pi 4 and Nvidia AGX edge computing devices, and the results were compared to the benchmark models. The model also achieves an accuracy of 98.81% and an inference time of 75.138 ms on Nvidia Jetson AGX. In contrast to hand gesture recognition systems based on RGB cameras, which have limited performance in the dark-light conditions, the proposed system based on reliable high resolution thermal images is well-suited to a wide range of applications.

Human hand gesture recognition with convolutional neural networks for K-12 double-teachers instruction mode classroom

Infrared hand gesture images can capture the human accurate gesture information in the weak illumination environment. In this article, we propose a novel hand gesture recognition model with convolutional neural networks for the human behavior analysis in “double teachers” classroom instruction and learning scenario. The recognized instructors’ hand gestures can be utilized to analyze teachers’ non-verbal behaviors which attract learners’ attention as well as enhance their learning outcomes. To extract the features of the infrared hand gesture images, a non-linear neural networks is constructed with four convolution layers. It aims to learn teachers’ pointing gestures and provides with the accurate gestures information. To achieve robust hand gesture recognition, the supervised convolutional neural network is constructed with three convolution layers to extract the infrared hand images features. Experiment results demonstrate that the proposed method outperforms the state-of-the-art methods in the term of the recognition accuracy and classification error. From the comparing experiments, the results prove that the developed approach can estimate the hand gesture with more than 92% recognition ratio.

Multidimensional Feature Representation and Learning for Robust Hand-Gesture Recognition on Commercial Millimeter-Wave Radar

This article presents a robust hand-gesture recognition method via multidimensional feature representation and learning specifically designed for commercial frequency-modulated continuous wave (FMCW) multi-input multi-output (MIMO) millimeter-wave radar. First, the optimal configuration of the radar system parameters for the hand-gesture recognition scenario is investigated and a standard procedure to determine the system configuration is given. Then a moving scattering center model is proposed to represent the 3-D point cloud in the range–Doppler (RD)-angular multidimensional feature space. A scattering point detection and tracking algorithm is presented based on a set of motion constraints in terms of position, velocity, and acceleration. It is derived from the space-time continuity of a nonrigid target. Finally, a lightweight multichannel convolutional neural network (CNN) is designed to learn and classify multidimensional gesture features including radial RD and tangential azimuth–elevation. Extensive experiments are carried out with the developed system and a large data set is obtained to train and test the classifier. The results show that the proposed gesture recognition method can effectively distinguish gestures that are easily confused in the RD domain and achieve robust performances under various conditions.

Robust hand gesture recognition system based on a new set of quaternion Tchebichef moment invariants

Hand gesture recognition is a challenging task due to the complexity of hand movements and to the variety among the same gesture performed by distinct subjects. Recent technologies, such as Kinect sensor, provide new opportunities, allowing to capture both RGB and depth (RGB-D) images, which offer high discriminant information for efficient hand gesture recognition. In the aspect of feature extraction, the traditional methods process the RGB and depth information independently. In this paper, we propose a robust hand gesture recognition system based on a new feature extraction method, fusing RGB images and depth information simultaneously, by using the quaternion algebra that provide a more robust and holistical representation. In fact, we introduce, for the first time, a novel type of feature extraction method, named quaternion Tchebichef moment invariants. The novelty of the proposed method in this paper lies in the direct derivation of invariants from their orthogonal moments, based on the algebraic properties of the discrete Tchebichef polynomials. The proposed approach based on quaternion algebra is suggested to process the four components holistically, for a robust and efficient hand gesture recognition system. The obtained experimental and theoretical results demonstrate that the present approach is very effective for addressing the problem of hand gesture recognition and have proved its robustness against geometrical distortion, noisy conditions and complex background compared to the state of the art, indicating that it could be highly useful for many computer vision applications.

Hand Gesture Recognition for Sign Language Using 3DCNN

Recently, automatic hand gesture recognition has gained increasing importance for two principal reasons: the growth of the deaf and hearing-impaired population, and the development of vision-based applications and touchless control on ubiquitous devices. As hand gesture recognition is at the core of sign language analysis a robust hand gesture recognition system should consider both spatial and temporal features. Unfortunately, finding discriminative spatiotemporal descriptors for a hand gesture sequence is not a trivial task. In this study, we proposed an efficient deep convolutional neural networks approach for hand gesture recognition. The proposed approach employed transfer learning to beat the scarcity of a large labeled hand gesture dataset. We evaluated it using three gesture datasets from color videos: 40, 23, and 10 classes were used from these datasets. The approach obtained recognition rates of 98.12%, 100%, and 76.67% on the three datasets, respectively for the signer-dependent mode. For the signer-independent mode, it obtained recognition rates of 84.38%, 34.9%, and 70% on the three datasets, respectively.

Fast and robust dynamic hand gesture recognition via key frames extraction and feature fusion

Gesture recognition is a hot topic in computer vision and pattern recognition, which plays a vitally important role in natural human-computer interface. Although great progress has been made recently, fast and robust hand gesture recognition remains an open problem, since the existing methods have not well balanced the performance and the efficiency simultaneously. To bridge it, this work combines image entropy and density clustering to exploit the key frames from hand gesture video for further feature extraction, which can improve the efficiency of recognition. Moreover, a feature fusion strategy is also proposed to further improve feature representation, which elevates the performance of recognition. To validate our approach in a “wild” environment, we also introduce two new datasets called HandGesture and Action3D datasets. Experiments consistently demonstrate that our strategy achieves competitive results on Northwestern University, Cambridge, HandGesture and Action3D hand gesture datasets. Our code and datasets will release at https://github.com/Ha0Tang/HandGestureRecognition.

Towards Zero Re-Training for Long-Term Hand Gesture Recognition via Ultrasound Sensing.

While myoelectric pattern recognition is a prevailing way for gesture recognition, the inherent nonstationarity of electromyography signals hinders its long-term application. This study aims to prove a hypothesis that morphological information of muscle contraction detected by ultrasound image is potentially suitable for long-term use. A set of ultrasound-based algorithms are proposed to realize robust hand gesture recognition over multiple days, with user training only at the first day. A markerless calibration algorithm is first presented to position the ultrasound probe during donning and doffing; an algorithm combining speeded-up robust features and bag-of-features model being immune to ultrasound probe shift and rotation is then introduced; a self-enhancing classification method is next adopted to update classification model automatically by incorporating useful knowledge from testing data; finally the performance of long-term hand gesture recognition with zero re-training is validated by a six-day experiment of six healthy subjects, whose outcomes strongly support the hypothesis with about 94% of gesture recognition accuracy for each testing day. This study confirms the feasibility of adoption of ultrasound sensing for long-term musculature related applications.