Static Hand Gesture Research Articles

Sign Language Recognition (SLR)

Abstract— Sign Language is mainly used by deaf (hard hearing) and dumb people to exchange information between their own community and with other people. It is a language where people use their hand gestures to communicate as they can’t speak or hear. Sign Language Recognition (SLR) deals with recognizing the hand gestures acquisition and continues till text or speech is generated for corresponding hand gestures. Here hand gestures for sign language can be classified as static and dynamic. However, static hand gesture recognition is simpler than dynamic hand gesture recognition, but both recognition is important to the human community. We can use Deep Learning Computer Vision to recognize the hand gestures by building Deep Neural Network architectures (Convolution Neural Network Architectures. Index Terms – HTML, CSS, Python, Pip, Mediapip, opencv, NumPy

Biomimetic learning of hand gestures in a humanoid robot.

Hand gestures are a natural and intuitive form of communication, and integrating this communication method into robotic systems presents significant potential to improve human-robot collaboration. Recent advances in motor neuroscience have focused on replicating human hand movements from synergies also known as movement primitives. Synergies, fundamental building blocks of movement, serve as a potential strategy adapted by the central nervous system to generate and control movements. Identifying how synergies contribute to movement can help in dexterous control of robotics, exoskeletons, prosthetics and extend its applications to rehabilitation. In this paper, 33 static hand gestures were recorded through a single RGB camera and identified in real-time through the MediaPipe framework as participants made various postures with their dominant hand. Assuming an open palm as initial posture, uniform joint angular velocities were obtained from all these gestures. By applying a dimensionality reduction method, kinematic synergies were obtained from these joint angular velocities. Kinematic synergies that explain 98% of variance of movements were utilized to reconstruct new hand gestures using convex optimization. Reconstructed hand gestures and selected kinematic synergies were translated onto a humanoid robot, Mitra, in real-time, as the participants demonstrated various hand gestures. The results showed that by using only few kinematic synergies it is possible to generate various hand gestures, with 95.7% accuracy. Furthermore, utilizing low-dimensional synergies in control of high dimensional end effectors holds promise to enable near-natural human-robot collaboration.

EchoGest: Soft Ultrasonic Waveguides based Sensing Skin for Subject-Independent Hand Gesture Recognition.

Gesture recognition is crucial for enhancing human-computer interaction and is particularly pivotal in rehabilitation contexts, aiding individuals recovering from physical impairments and significantly improving their mobility and interactive capabilities. However, current wearable hand gesture recognition approaches are often limited in detection performance, wearability, and generalization. We thus introduce EchoGest, a novel hand gesture recognition system based on soft, stretchable, transparent artificial skin with integrated ultrasonic waveguides. Our presented system is the first to use soft ultrasonic waveguides for hand gesture recognition. Ecoflex™ 00-31 and Ecoflex™ 00-45 Near Clear™ silicone elastomers were employed to fabricate the artificial skin and ultrasonic waveguides, while 0.1 mm diameter silver-plated copper wires connected the transducers in the waveguides to the electrical system. The wires are enclosed within an additional elastomer layer, achieving a sensing skin with a total thickness of around 500 μm. Ten participants wore the EchoGest system and performed static hand gestures from two gesture sets: 8 daily life gestures and 10 American Sign Language (ASL) digits 0-9. Leave-One-Subject-Out Cross-Validation analysis demonstrated accuracies of 91.13% for daily life gestures and 88.5% for ASL gestures. The EchoGest system has significant potential in rehabilitation, particularly for tracking and evaluating hand mobility, which could substantially reduce the workload of therapists in both clinical and home-based settings. Integrating this technology could revolutionize hand gesture recognition applications, from real-time sign language translation to innovative rehabilitation techniques.

Robust static hand gesture recognition: harnessing sparsity of deeply learned features

Robust static hand gesture recognition: harnessing sparsity of deeply learned features

Glove-Based Hand Gesture Recognition for Diver Communication.

We have developed a smart dive glove that recognizes 13 static hand gestures used in diving communication. The smart glove employs five dielectric elastomer sensors to capture finger motion and implements a machine learning classifier in the onboard electronics to recognize gestures. Five basic classification algorithms are trained and assessed: the decision tree, support vector machine (SVM), logistic regression, Gaussian naïve Bayes, and multilayer perceptron. These basic classifiers were selected as they perform well in multiclass classification problems, can be trained using supervised learning, and are model-based algorithms that can be implemented on a microprocessor. The training dataset was collected from 24 participants providing for a range of different hand sizes. After training, the algorithms were evaluated in a dry environment using data collected from ten new participants to test how well they cope with new information. Furthermore, an underwater experiment was conducted to assess any impact of the underwater environment on each algorithm's classification. The results show all classifiers performed well in a dry environment. The accuracies and F1-scores range between 0.95 and 0.98, where the logistic regressor and SVM have the highest scores for both the accuracy and F1-score (0.98). The underwater results showed that all algorithms work underwater; however, the performance drops when divers must focus on buoyancy control, breathing, and diver trim.

Smart-Data-Glove-Based Gesture Recognition for Amphibious Communication.

This study has designed and developed a smart data glove based on five-channel flexible capacitive stretch sensors and a six-axis inertial measurement unit (IMU) to recognize 25 static hand gestures and ten dynamic hand gestures for amphibious communication. The five-channel flexible capacitive sensors are fabricated on a glove to capture finger motion data in order to recognize static hand gestures and integrated with six-axis IMU data to recognize dynamic gestures. This study also proposes a novel amphibious hierarchical gesture recognition (AHGR) model. This model can adaptively switch between large complex and lightweight gesture recognition models based on environmental changes to ensure gesture recognition accuracy and effectiveness. The large complex model is based on the proposed SqueezeNet-BiLSTM algorithm, specially designed for the land environment, which will use all the sensory data captured from the smart data glove to recognize dynamic gestures, achieving a recognition accuracy of 98.21%. The lightweight stochastic singular value decomposition (SVD)-optimized spectral clustering gesture recognition algorithm for underwater environments that will perform direct inference on the glove-end side can reach an accuracy of 98.35%. This study also proposes a domain separation network (DSN)-based gesture recognition transfer model that ensures a 94% recognition accuracy for new users and new glove devices.

A Survey of Sign Language Recognition

Sign Language is mainly used by deaf (hard hearing) and dumb people to exchange information between their own community and with other people. It is a language where people use their hand gestures to communicate as they can't speak or hear. The goal of sign language recognition (SLR) is to identify acquired hand motions and to continue until related hand gestures are translated into text and speech. Here, static and dynamic hand gestures for sign language can be distinguished. The human community values both types of recognition, even if static hand gesture recognition is easier than dynamic hand gesture recognition. By creating Deep Neural Network designs (Convolution Neural Network designs), where the model will learn to detect the hand motions images throughout an epoch, we are using Deep Learning Computer Vision to recognize the hand gestures. After the model successfully recognizes the motion, an English text file is created that can subsequently be translated to speech. The user can choose from a variety of translations for this paragraph. This application can be used without an internet connection and is entirely offline. With this model's improved efficiency, communication will be easier for the deaf (hard of hearing) and disabled people. We shall discuss the use of deep learning for sign language recognition in this paper. Key Words: sign language, convolutional neural network, computer vision.

Static Hand Gesture Recognition Based on Millimeter-Wave Near-Field FMCW-SAR Imaging

To address the limitations of wireless sensing in static gesture recognition and the issues of Computer Vision’s dependence on lighting conditions, we propose a method that utilizes millimeter-wave near-field SAR (Synthetic Aperture Radar) imaging for static gesture recognition. First, a millimeter-wave near-field SAR imaging system is used to scan the defined static gestures to obtain data. Then, based on the distance plane, the three-dimensional gesture is divided into multiple two-dimensional planes, constructing an imaging dataset. Finally, an HOG (Histogram of Oriented Gradients) is used to extract features from the imaging results, PCA (Principal Component Analysis) is applied for feature dimensionality reduction, and RF (Random Forest) performs classification. Experimental verification shows that the proposed method achieves an average recognition precision of 97% in unobstructed situations and 93% in obstructed situations, providing an effective means for wireless-sensing-based static gesture recognition.

Visual Static Hand Gesture Recognition Using Convolutional Neural Network

Hand gestures are an essential part of human-to-human communication and interaction and, therefore, of technical applications. The aim is increasingly to achieve interaction between humans and computers that is as natural as possible, for example, by means of natural language or hand gestures. In the context of human-machine interaction research, these methods are consequently being explored more and more. However, the realization of natural communication between humans and computers is a major challenge. In the field of hand gesture recognition, research approaches are being pursued that use additional hardware, such as special gloves, to classify gestures with high accuracy. Recently, deep learning techniques using artificial neural networks have been increasingly proposed for the problem of gesture recognition without using such tools. In this context, we explore the approach of convolutional neural network (CNN) in detail for the task of hand gesture recognition. CNN is a deep neural network that can be used in the fields of visual object processing and classification. The goal of this work is to recognize ten types of static hand gestures in front of complex backgrounds and different hand sizes based on raw images without the use of extra hardware. We achieved good results with a CNN network architecture consisting of seven layers. Through data augmentation and skin segmentation, a significant increase in the model’s accuracy was achieved. On public benchmarks, two challenging datasets have been classified almost perfectly, with testing accuracies of 96.5% and 96.57%.

Feature Extraction Based on Cellular Particle Swarm Optimization Algorithm for American Sign Language Images

Image Recognition is becoming a critical task and many problem-solving systems and approaches for image detection, analysis and classification are introduced by many modern researcher. The techniques should be user friendly and ease to interpret for both normal people and special people, it they should be analysed. The sign language act as a mode to transfer and exchange the message, information, knowledge and ideas from deaf to common people. The gaining information and responses to the pattern or gesture is called as sign. Sign Language is only mode to communication between the hearing-impaired people and common people. In this method each individual gesture is called as sign. In this system the American Sign Language recognition system which is used for visually impaired people to access and educate the special children’s. American Sign Language (ASL) images of a special people is collected with some constraints are taken as the dataset. The objective of the research work is done with three different Phase. Phase1 is pre-processing, Phase 2 is segmentation and Phase 3 is feature extraction for static hand gestures with the maximum possible accuracy rate. A novel approach for the proposed system is analysed with its structural feature extraction and compared with its parameters. The novel feature extraction is made with 8 different structural features they are Bounding Box, Area, Perimeter, Centroid, Roundness, EquiDiameter, Number of Boundaries and Angle. F-Measure, Recall and Precision based on this Accuracy, Sensitivity and Specificity are measured for features are extracted images. The accuracy of the new Phase has been found significantly higher.

Static Hand Gesture Recognition Using Capacitive Sensing and Machine Learning

Automated hand gesture recognition is a key enabler of Human-to-Machine Interfaces (HMIs) and smart living. This paper reports the development and testing of a static hand gesture recognition system using capacitive sensing. Our system consists of a 6×18 array of capacitive sensors that captured five gestures-Palm, Fist, Middle, OK, and Index-of five participants to create a dataset of gesture images. The dataset was used to train Decision Tree, Naïve Bayes, Multi-Layer Perceptron (MLP) neural network, and Convolutional Neural Network (CNN) classifiers. Each classifier was trained five times; each time, the classifier was trained using four different participants' gestures and tested with one different participant's gestures. The MLP classifier performed the best, achieving an average accuracy of 96.87% and an average F1 score of 92.16%. This demonstrates that the proposed system can accurately recognize hand gestures and that capacitive sensing is a viable method for implementing a non-contact, static hand gesture recognition system.

Static hand gesture recognition method based on the Vision Transformer

Static hand gesture recognition method based on the Vision Transformer

Static hand gesture recognition using multi-dilated DenseNet-based deep learning architecture

ABSTRACT Hand Gesture Recognition (HGR), a Human-Robot Interaction (HRI) create user interfaces to enhance safety and control. Several approaches are introduced to recognize static hand gestures effectively, however the classification efficiency suffered due to various lightings like natural light, artificial light and dark rooms increases the difficulty of accurate HGR. To tackle these problems, a Multi-Dilated Convolution-based DenseNet (MDCDN) architecture, a combination of multi-dilated convolution and DenseNet is proposed which extracts the features automatically. Finally, the benefits of high-level deep learning techniques are leveraged for the gesture recognition from hand. Python is used for architecture evaluation. The outcome of proposed is estimated in terms of accuracy, recall, F-measure, precision, etc, using ASL, ISL, Massey and HSR real datasets. Each dataset contains huge amount of gesture classes and their pictures have an equal amount of uniform and complex backgrounds. The proposed results are promising and provide better performance than existing methods.

Using VGG Models with Intermediate Layer Feature Maps for Static Hand Gesture Recognition

A hand gesture recognition system provides a robust and innovative solution to nonverbal communication through human–computer interaction. Deep learning models have excellent potential for usage in recognition applications. To overcome related issues, most previous studies have proposed new model architectures or have fine-tuned pre-trained models. Furthermore, these studies relied on one standard dataset for both training and testing. Thus, the accuracy of these studies is reasonable. Unlike these works, the current study investigates two deep learning models with intermediate layers to recognize static hand gesture images. Both models were tested on different datasets, adjusted to suit the dataset, and then trained under different methods. First, the models were initialized with random weights and trained from scratch. Afterward, the pre-trained models were examined as feature extractors. Finally, the pre-trained models were fine-tuned with intermediate layers. Fine-tuning was conducted on three levels: the fifth, fourth, and third blocks, respectively. The models were evaluated through recognition experiments using hand gesture images in the Arabic sign language acquired under different conditions. This study also provides a new hand gesture image dataset used in these experiments, plus two other datasets. The experimental results indicated that the proposed models can be used with intermediate layers to recognize hand gesture images. Furthermore, the analysis of the results showed that fine-tuning the fifth and fourth blocks of these two models achieved the best accuracy results. In particular, the testing accuracies on the three datasets were 96.51%, 72.65%, and 55.62% when fine-tuning the fourth block and 96.50%, 67.03%, and 61.09% when fine-tuning the fifth block for the first model. The testing accuracy for the second model showed approximately similar results.

GHOSM: Graph-based Hybrid Outline and Skeleton Modelling for Shape Recognition

An efficient and accurate shape detection model plays a major role in many research areas. With the emergence of more complex shapes in real-life applications, shape recognition models need to capture the structure with more effective features to achieve high accuracy rates for shape recognition. This article presents a new method for 2D/3D shape recognition based on graph spectral domain handcrafted features, which are formulated by exploiting both an outline and a skeleton shape through the global outline and internal details. A fully connected graph is generated over the shape outline to capture the global outline representation while a hierarchically clustered graph with adaptive connectivity is formed on the skeleton to capture the structural descriptions of the shape. We demonstrate the ability of the Fiedler vector to provide the graph partitioning of the skeleton graph. The performance evaluation demonstrates the efficiency of the proposed method compared to state-of-the-art studies with increments of 4.09%, 2.2%, and 14.02% for 2D static hand gestures, 2D shapes, and 3D shapes, respectively.

DeReFNet: Dual-stream Dense Residual Fusion Network for static hand gesture recognition

Vision-based hand gesture recognition (HGR) system provides the most effective and natural way of interaction between humans and machines. However, the recognition performance of such an HGR system is challenging due to the variations in illumination, complex backgrounds, the shape of the user’s hand, and inter-class similarity. This work proposes a compact dual-stream dense residual fusion network (DeReFNet) to address the above challenges. The proposed convolutional neural network architecture mainly utilizes the strength of global features from each residual block of the residual stream and spatial information from the other stream using dense connectivity. Both the streams are fused to gather enriched information using the feature concatenation module. The efficacy of the DeReFNet is validated using a subject-independent cross-validation technique on four publicly available benchmark datasets. Furthermore, the qualitative and quantitative analysis of the benchmarked datasets illustrates that the DeReFNet outperforms state-of-the-art methods in terms of accuracy and computational time.

An Efficient and Robust Hand Gesture Recognition System of Sign Language Employing Finetuned Inception-V3 and Efficientnet-B0 Network

Hand Gesture Recognition (HGR) is a promising research area with an extensive range of applications, such as surgery, video game techniques, and sign language translation, where sign language is a complicated structured form of hand gestures. The fundamental building blocks of structured expressions in sign language are the arrangement of the fingers, the orientation of the hand, and the hand’s position concerning the body. The importance of HGR has increased due to the increasing number of touchless applications and the rapid growth of the hearing-impaired population. Therefore, real-time HGR is one of the most effective interaction methods between computers and humans. Developing a user-free interface with good recognition performance should be the goal of real-time HGR systems. Nowadays, Convolutional Neural Network (CNN) shows great recognition rates for different image-level classification tasks. It is challenging to train deep CNN networks like VGG-16, VGG-19, Inception-v3, and Efficientnet-B0 from scratch because only some significant labeled image datasets are available for static hand gesture images. However, an efficient and robust hand gesture recognition system of sign language employing finetuned Inception-v3 and Efficientnet-Bo network is proposed to identify hand gestures using a comparative small HGR dataset. Experiments show that Inception-v3 achieved 90% accuracy and 0.93% precision, 0.91% recall, and 0.90% f1-score, respectively, while EfficientNet-B0 achieved 99% accuracy and 0.98%, 0.97%, 0.98%, precision, recall, and f1-score respectively.

Hand Gesture Recognition Pad Using an Array of Inductive Sensors

Gesture recognition is a field of study within human-computer interaction technology and is used in an increasing number of applications. So far, the systems designed for gesture recognition have been able to sense multiple static and dynamic gestures. However, they suffer from some limitations. This study introduces a novel sensing pad system consisting of an array of inductive sensors which can recognize and differentiate specific static hand gestures through machine learning algorithms. It is designed to be a non-contact apparatus where the gestures made by a user can be perceived by the system. It uses five coils, one for each finger, and can sense the fingers that are unfolded while making a particular gesture. Ten volunteer users participated in this study. Ten gestures, numbers 1-10 of the American Sign Language are chosen to be tested upon, 10 times each for every user. The responses from the sensing coils were measured via a data acquisition board and sent to the PC for processing. A total of 1000 responses were recorded and processed using machine learning algorithms which provided an accuracy of 98.7% using 5-fold cross-validation (5F-CV) and 97.3% using leave-one-subject-out CV (LOSO-CV) proving that the system can successfully distinguish hand gestures instantly.

SSEC: Semantic Segmentation and Ensemble Classification Framework for Static Hand Gesture Recognition using RGB-D Data

Hand Gesture Recognition (HGR) refers to identifying various hand postures used in Sign Language Recognition (SLR) and Human Computer Interaction (HCI) applications. Complex background in uncontrolled environmental condition is the major challenging issue which impacts the recognition accuracy of HGR system. This can be effectively addressed by discarding the background using suitable semantic segmentation method, where it predicts the hand region pixels into foreground and rest of the pixels into background. In this paper, we have analyzed and evaluated well known semantic segmentation architectures for hand region segmentation using both RGB and depth data. Further, ensemble of segmented RGB and depth stream is used for hand gesture classification through probability score fusion. Experimental results shows that the proposed novel framework of Semantic Segmentation and Ensemble Classification (SSEC) is suitable for static hand gesture recognition and achieved F1-score of 88.91% on OUHANDS test dataset.

Noise-Against Skeleton Extraction Framework and Application on Hand Gesture Recognition

Extracting stable skeletons from noisy images is a challenging problem since the skeletonization method is prone to be affected by inner and border noise. Although many methods have been proposed in the past for increasing the antinoise ability of skeletonization methods, most of them either only overcome border noise or, at the cost of lost topology, degrade the effects of two noises. In this paper, we propose a skeleton extraction framework to enhance the robustness of the existing skeletonization method against both inner and border noise. In our approach, we first use the different scales of Gaussian filters to smooth the input image and obtain multiple representations. Then, binarization and skeletonization were performed to produce a series of binary images and a series of skeletal images. Next, we use our measure on these binary and skeletal images to find the most suitable skeleton. Since our measure considers both the skeleton image changes and binary image changes caused by using a filter, the selected skeleton is sufficiently robust and has all the necessary skeletal branches. The inner noise experiment and border noise experiment are conducted for comparison. From the perspective of the measure of the rate of variation in the skeleton, the proposed framework can reduce the inner noise by approximately 92% and the border noise by approximately 40%. In addition, the experiment on static hand gesture recognition has demonstrated that the introduction of our framework can increase up to 11% mean recognition accuracy.