Hand Gesture Recognition System Research Articles

U-DeepHand: FMCW Radar-Based Unsupervised Hand Gesture Feature Learning Using Deep Convolutional Auto-Encoder Network

Recently, although radar sensors have been widely applied for hand gesture recognition (HGR) tasks, conventional radar-based HGR systems still have two major challenges. First, these systems rely on supervised learning approaches to learn gesture features, which normally require a large-scale labeled dataset to address the overfitting problem. However, the acquisition of such dataset is time-consuming. Second, the radar signature of hand movement is often influenced by micromotion caused by other body parts, which leads to distorted motion features, resulting in poor identification accuracy. To overcome these problems, we propose an unsupervised hand gesture feature learning method using the deep convolutional auto-encoder network to analyze hand gesture signal collected by a frequency modulated continuous wave (FMCW) radar sensor. First, via a convolutional encoder sub-network, input radar range profiles are transformed into lower dimensional representations. Then, the representations are expanded to reconstruct the corresponding input profiles by a deconvolutional decoder sub-network. In addition, to investigate the mechanisms of the proposed network and evaluate its performance, we conduct an in-depth study of the feature maps learned from various hand gesture experimental data and evaluate the corresponding classification performance. The results demonstrate that the proposed convolutional auto-encoder network is able to achieve high recognition accuracy with low training sample cost, which outperforms the state-of-the-art hand gesture recognition systems based on transfer learning VGGNet and fully connected-based auto-encoder network.

Hand Gesture Recognition Enhancement Based on Spatial Fuzzy Matching in Leap Motion

Gesture recognition is an important human–computer interaction interface. This article introduces a novel hand gesture recognition system based on Leap Motion gen.2. In this system, a spatial fuzzy matching (SFM) algorithm is first presented by matching and fusing spatial information to construct a fused gesture dataset. For dynamic hand recognition, an initial frame correction strategy based on SFM is proposed to fast initialize the trajectory of test gesture with respect to the gesture dataset. A notable feature of this system is that it can run on ordinary laptops due to the small size of the fused dataset, which accelerates the calculation of recognition rate. Experimental results show that the system recognizes static hand gestures at recognition rates of 94%–100% and over 90% of dynamic gestures using our collected dataset. This can greatly enhance the usability of Leap Motion.

Implementation and Performance Analysis of Different Hand Gesture Recognition Methods

In recent few years, hand gesture recognition is one of the advanced grooming technologies in the era of human computer interaction and computer vision due to a wide area of application in the real world. But it is a very complicated task to recognize hand gesture easily due to gesture orientation, light condition, complex background, translation and scaling of gesture images. To remove this limitation, several research works have developed which is successfully decrease this complexity. However, the intention of this paper is proposed and compared four different hand gesture recognition system and apply some optimization technique on it which ridiculously increased the existing model accuracy and model running time. After employed the optimization tricks, the adjusted gesture recognition model accuracy was 93.21% and the run time was 224 seconds which was 2.14% and 248 seconds faster than an existing similar hand gesture recognition model. The overall achievement of this paper could be applied for smart home control, camera control, robot control, medical system, natural talk, and many other fields in computer vision and human-computer interaction.

Cooperative Sensing and Wearable Computing for Sequential Hand Gesture Recognition

Hand gestures recognition (HGR) has been considered as one of the crucial research fields of human–computer interaction (HCI). Computer vision is a very active research field in the HGR, traditional vision-based methods, which used camera and ultrasonic/optical sensor to collect the videos or images of the hand gestures shown by participants, have some limitations, such as fixed in-lab location, complex lighting conditions, and cluttered backgrounds. In order to provide new approaches, we described the development of a novel hand gesture recognition system that combined wearable armband and smart glove made by customizable pressure sensor arrays to detect sequential hand gestures. A deep learning technique long short-term memory (LSTM) algorithm had been computed to build an effective model to classify hand gestures by training and testing the collected inertial measurement unit (IMU), electromyographic (EMG), and finger and palm’s pressure data. Furthermore, we built a relatively large database of ten sequential hand gestures consisted by five dynamic gestures and five air gestures collected from ten participants. Our experimental results showed an outstanding classification performance of the proposed LSTM algorithm. These findings have promising implications for sequential hand gesture recognition and the HCI research status.

Efficient representation of size functions based on moments theory

Nowadays, there is a need to develop efficient and intuitive solutions such as hand gestures recognition for the Human-machine interaction. This paper presents a hand gestures recognition system based on salient geometric features extracted using size functions theory. We propose a new representation of the reduced size function based on Tchebichef moments providing more details and information for their graphs descriptions compared to existing representations. In addition, a methodical algorithm of fast Tchebichef moments computation for grey scale images is well adapted to the encoded graph of size function. Furthermore, a contour discretization based on a convexity approach is proposed for an optimal computation of the measuring functions, and new measuring functions for hand gestures classification and retrieval are proposed. The comparison with existing systems indicates that our method competes with the best ranked method for the dynamic case and surpasses the state of the art in static case; in addition it presents the advantage to be applied in both static and dynamic cases.

Recognition of static hand gesture with using ANN and SVM

Hand gesture recognition is a relevant study topic for a reason that sometimes we may not be in position to communicate verbally. There is need to design Hand gesture recognition systems in order to help people adopt to nonverbal communication mainly sign language. However, there is no clue to understand the meaning of gesture through the computers directly. So this calls for definitions that generalize models in a computer. That is why the machine-learning approaches are implemented in recognition systems. There are generally two types of hand gestures recognition systems which researches have concentrated on. These include static and dynamic Hand gesture recognition systems. However, in building Hand gesture recognition systems, various machine learning approaches have been used. For implementing the proposed system, MS Kinect depth sensor was used as a hardware. The Kinect depth sensor is composed of an infrared camera. This is an advantage to the systems that are designed basing on the depth sensing because factors like color, clothing and background have less effect on the performance. So Kinect based depth sensor systems have a high accuracy and performance making them relevant and applicable in our daily lives. In this paper, we propose a static hand gesture recognition system in real time using two machine learning methods namely Support Vector Machine and Artificial Neural Networks. We use of the newly launched Microsoft Kinect sensor for image extraction. The sensor helps us to extract the hand images. We implement the system on a Matlab platform for reasons that Matlab is widely used by researchers in different fields and that can handle complex computations. In the training of the model, we collect a hundred depth-based Histogram of Oriented Gradient features per alphabet from the hand gesture images which we trained, tested and validated using Artificial Neural Networks (ANN) and Support Vector Machine (SVM). From this dataset, we can generate the generalized gesture model for each alphabet image. For the proposed system, the classification with ANN proves a higher performance then SVM.

Adaptive Global Time Sequence Averaging Method Using Dynamic Time Warping

Time sequence averaging under dynamic time warping (DTW) is a non-trivial problem, and its exact solution requires unrealistic computational complexity in practice. The DTW barycenter averaging (DBA) method is one of the most effective iterative approximation solutions to date. However, there are still a few drawbacks in the DBA method. First, the length of the resulting average sequence depends on the selected initial average sequence; second, the discrepancy distance between the resulting average sequence and target sequence set is highly sensitive to the initialization, as we have demonstrated through the experiments described here. In this study, we propose an adaptive DBA (ADBA) algorithm to address these drawbacks. We define a scaling coefficient based on the DTW alignments such that the temporal aberrations between the average sequence and target sequence set can be qualitatively captured. The algorithm is realized by an iterative process. For each iteration, the temporary average sequence and target sequence set are partitioned into several aligned subsequence sets according to the variation in the signs of the scaling coefficients. Then, these partitioned average subsequences are adaptively compressed or stretched such that the average discrepancy distance and overall temporal aberration can be locally minimized. The comparison experiments carried out on the standard datasets illustrate that the proposed algorithm achieves lower average discrepancy distance, overall temporal aberration, and higher robustness than the available methods. Additionally, the proposed algorithm is verified by an accelerometer-based hand gesture recognition system, where ADBA produces more effective gesture templates.

Rapid speedup segment analysis based feature extraction for hand gesture recognition

The dependency on computers and machines have progressively increased in the past few years and hence human interaction with computers is one of the most actively researched area in science. Many Hand Gesture Recognition (HGR) systems have been developed and they continue to evolve, where in which the gesture interaction becomes smoother and smarter. Gesture recognition is usually implemented in three phases- gesture segmentation, feature extraction and gesture classification, and it’s important that the processes involved in these stages are chosen appropriately such that the misrecognition rate will be kept to a minimum. This paper has proposed a novel algorithm that combines 2D-Discrete Wavelet Transform along with Speed Up Robust Feature (SURF) extraction technique to achieve a robust HGR system that is rotation and scale invariant. The proposed method has achieved an overall classification accuracy of 96.9% with the Radial Basis Function Neural Network (RBFNN) classifier.

Surface EMG hand gesture recognition system based on PCA and GRNN

The principal component analysis method and GRNN neural network are used to construct the gesture recognition system, so as to reduce the redundant information of EMG signals, reduce the signal dimension, improve the recognition efficiency and accuracy, and enhance the feasibility of real-time recognition. Using the means of extracting key information of human motion, the specific action mode is identified. In this paper, nine static gestures are taken as samples, and the surface EMG signal of the arm is collected by the electromyography instrument to extract four kinds of characteristics of the signal. After dimension reduction and neural network learning, the overall recognition rate of the system reached 95.1%, and the average recognition time was 0.19 s.

Hand Gesture Recognition Using PCA, KNN and SVM

To interact with world using expressions or body movements is comparatively effective than just speaking. Gesture recognition can be a better way to convey meaningful information. Communication through gestures has been widely used by humans to express their thoughts and feelings. Gestures can be performed with any body part like head, face, hands and arms but most predominantly hand is use to perform gestures, Hand Gesture Recognition have been widely accepted for numerous applications such as human computer interactions, robotics, sign language recognition, etc. This paper focuses on bare hand gesture recognition system by proposing a scheme using a database-driven hand gesture recognition based upon skin color model approach and thresholding approach along with an effective template matching with can be effectively used for human robotics applications and similar other applications .Initially, hand region is segmented by applying skin color model in YCbCr color space. Y represents the luminance and Cb and Cr represents chrominance. In the next stage Otsu thresholding is applied to separate foreground and background. Finally, template based matching technique is developed using Principal Component Analysis (PCA), k-nearest neighbour (KNN) and Support Vector Machine (SVM) for recognition. KNN is used for statistical estimation and pattern recognition. SVM can be used for classification or regression problems.

Developing a Hand Gesture Recognition System for Mapping Symbolic Hand Gestures to Analogous Emojis in Computer-Mediated Communication

Recent trends in computer-mediated communication (CMC) have not only led to expanded instant messaging through the use of images and videos but have also expanded traditional text messaging with richer content in the form of visual communication markers (VCMs) such as emoticons, emojis, and stickers. VCMs could prevent a potential loss of subtle emotional conversation in CMC, which is delivered by nonverbal cues that convey affective and emotional information. However, as the number of VCMs grows in the selection set, the problem of VCM entry needs to be addressed. Furthermore, conventional means of accessing VCMs continue to rely on input entry methods that are not directly and intimately tied to expressive nonverbal cues. In this work, we aim to address this issue by facilitating the use of an alternative form of VCM entry: hand gestures. To that end, we propose a user-defined hand gesture set that is highly representative of a number of VCMs and a two-stage hand gesture recognition system (trajectory-based, shape-based) that can identify these user-defined hand gestures with an accuracy of 82%. By developing such a system, we aim to allow people using low-bandwidth forms of CMCs to still enjoy their convenient and discreet properties while also allowing them to experience more of the intimacy and expressiveness of higher-bandwidth online communication.

Electromyography-Based Hand Gesture Recognition System for Upper Limb Amputees

This article presents a low-power embedded platform that recognizes arm gestures by decoding surface electromyography (EMG) signals of amputees. The system consists of ADS1298 as analog front end (AFE), which can acquire 8-channel EMG signals simultaneously. For the validation of this system in real-time environment, four subjects have been recruited, including one transradial amputee. Six activities have been performed by each subject for the EMG pattern recognition experiment. Two cases have been created for testing data analysis of real-time classification in a DSP processor. Case one (feature extraction in DSP processor and testing off-line) shows the highest classification accuracy of 97.60% and the mean classification accuracy of 95.40%. Case two (training and testing within DSP processor) shows the highest classification accuracy of 97.75% and the mean classification accuracy of 92%. This shows that this system is capable of recognizing user intention in the real-time environment, with above 91% accuracy.

Shape Based Continuous Real Time Hand Gesture Recognition System of American Sign Language Using KNN Classifier

The sign language is a language of communication for the speech and hearing impaired people used for their daily conversations and needs. This research paper give away, a creative framework, aims to accomplish the transfiguration of 24 continuous real time alphabet gestures of American Sign Language into human and / or device recognizable English text. The real time alphabets gestures take into consideration for testing are absolutely invariant to location, background, illumination, skin color, gender, and distance in nature. The real time hand gesture recognition process is conceded using the regular web camera. Histogram of Oriented Gradients (HOG) is used for hand feature extraction after the preprocessing operations. As a result of this research work, experimentation is carried out using the K-Nearest Neighbor (KNN) classifier for classification and recognition of gestures with k=3, 5, and 7. Finally, yields an overall average recognition rate of continuous real time alphabet gestures is 98.44%using KNN Classifier (k=3) with the least recognition time of 0.38 seconds per gesture, which is excellent comparing with the proposed technique k=5 (accuracy 93.75%, time elapsed 0.42 seconds), and k=7 (accuracy 90.10%, time elapsed 0.45 seconds), and also with the state of art techniques in real time environment.

Intelligent Human-Computer Interaction Based on Surface EMG Gesture Recognition

Urban intelligence is an emerging concept which guides a series of infrastructure developments in modern smart cities. Human-computer interaction (HCI) is the interface between residents and the smart cities, it plays a key role in bridging the gap in applicating information technologies in modern cities. Hand gestures have been widely acknowledged as a promising HCI method, recognition human hand gestures using surface electromyogram (sEMG) is an important research topic in the application of sEMG. However, state-of-the-art signal processing technologies are not robust in feature extraction and pattern recognition with sEMG signals, several technical problems are still yet to be solved. For example, how to maintain the availability of myoelectric control in intermittent use, since pattern recognition qualities are greatly affected by time variability, but it is unavoidable during daily use. How to ensure the reliability and effectiveness of myoelectric control system also important in developing a good human-machine interface. In this paper, linear discriminant analysis (LDA) and extreme learning machine (ELM) are implemented in hand gesture recognition system, which is able to reduce the redundant information in sEMG signals and improve recognition efficiency and accuracy. The characteristic map slope (CMS) is extracted by using the feature re-extraction method because CMS can strengthen the relationship of features cross time domain and enhance the feasibility of cross-time identification. This study is focusing on optimizing the time differences in sEMG pattern recognition, the experimental results are beneficial to reducing the time differences in gesture recognition based on sEMG. The recognition framework proposed in this paper can enhance the generalization ability of HCI in the long term use and it also simplifies the data collection stage before training the device ready for daily use, which is of great significance to improve the time generalization performance of an HCI system.

Hand Gesture Recognition Based on Active Ultrasonic Sensing of Smartphone: A Survey

With the rapid development of Internet of Things, hand gesture recognition has drawn wide attention in the field of ubiquitous computing because it provides us with simple and natural human-computer interaction mode. Among these various implementations, hand gesture recognition using ultrasonic signals of smartphone has become a hot research topic due to its various advantages. In this paper, we consider the smartphone as an active sonar sensing system to identify hand movements. Specifically, the speakers emit ultrasonic signal and the microphone on the same phone receives the changed echo affected by hand movements. This paper investigates the state-of-the-art hand gesture applications and presents a comprehensive survey on the characteristics of studies using the active sonar sensing system. Firstly, we review the existing research of hand gesture recognition based on acoustic signals. After that, we introduce the characteristics of ultrasonic signal and describe the fundamental principle of hand gesture recognition. Then, we focus on the typical methods used in these studies and present a detailed analysis on signal generation, feature extraction, preprocessing, and recognition methods. Next, we investigate the state-of-the-art ultrasonic-based applications of hand gesture recognition using smartphone and analyze them in detail from dynamic gesture recognition and hand tracking. Afterwards, we make a discussion about these systems from signal acquisition, signal processing, and performance evaluation to obtain some insight into development of the ultrasonic hand gesture recognition system. Finally, we conclude by discussing the challenges, insight, and open issues involved in hand gesture recognition based on ultrasonic signal of the smartphone.

Short-Range Radar Based Real-Time Hand Gesture Recognition Using LSTM Encoder

Due to the development of short-range radar with high-resolution, the radar sensor has a high potential to be used in real human-computer interaction (HCI) applications. The radar sensor has advantages over optical cameras in that it is unaffected by illumination and it is able to detect the objects in an occluded environment. This paper proposes a hand gesture recognition system for a real-time application of HCI using 60 GHz frequency-modulated continuous wave (FMCW) radar, Soli, developed by Google. The overall system includes signal processing part that generates range-Doppler map (RDM) sequences without clutter and machine learning part including a long short-term memory (LSTM) encoder to learn the temporal characteristics of the RDM sequences. A set of data is collected from 10 participants for the experiment. The proposed hand gesture recognition system successfully distinguishes 10 gestures with a high classification accuracy of 99.10%. It also recognizes the gestures of a new participant with an accuracy of 98.48%.

Recognition of Symbolic Gestures Using Depth Information

Symbolic gestures are the hand postures with some conventionalized meanings. They are static gestures that one can perform in a very complex environment containing variations in rotation and scale without using voice. The gestures may be produced in different illumination conditions or occluding background scenarios. Any hand gesture recognition system should find enough discriminative features, such as hand-finger contextual information. However, in existing approaches, depth information of hand fingers that represents finger shapes is utilized in limited capacity to extract discriminative features of fingers. Nevertheless, if we consider finger bending information (i.e., a finger that overlaps palm), extracted from depth map, and use them as local features, static gestures varying ever so slightly can become distinguishable. Our work here corroborated this idea and we have generated depth silhouettes with variation in contrast to achieve more discriminative keypoints. This approach, in turn, improved the recognition accuracy up to 96.84%. We have applied Scale-Invariant Feature Transform (SIFT) algorithm which takes the generated depth silhouettes as input and produces robust feature descriptors as output. These features (after converting into unified dimensional feature vectors) are fed into a multiclass Support Vector Machine (SVM) classifier to measure the accuracy. We have tested our results with a standard dataset containing 10 symbolic gesture representing 10 numeric symbols (0-9). After that we have verified and compared our results among depth images, binary images, and images consisting of the hand-finger edge information generated from the same dataset. Our results show higher accuracy while applying SIFT features on depth images. Recognizing numeric symbols accurately performed through hand gestures has a huge impact on different Human-Computer Interaction (HCI) applications including augmented reality, virtual reality, and other fields.

Hand gesture recognition using DWT and F ‐ratio based feature descriptor

This study demonstrates the development of vision based static hand gesture recognition system using web camera in real-time applications. The vision based static hand gesture recognition system is developed using the following steps: preprocessing, feature extraction and classification. The preprocessing stage consists of illumination compensation, segmentation, filtering, hand region detection and image resize. This study proposes a discrete wavelet transform (DWT) and Fisher ratio ( F -ratio) based feature extraction technique to classify the hand gestures in an uncontrolled environment. This method is not only robust towards distortion and gesture vocabulary, but also invariant to translation and rotation of hand gestures. A linear support vector machine is used as a classifier to recognise the hand gestures. The performance of the proposed method is evaluated on two standard public datasets and one indigenously developed complex background dataset for recognition of hand gestures. All above three datasets are developed based on American Sign Language (ASL) hand alphabets. The experimental result is evaluated in terms of mean accuracy. Two possible real-time applications are conducted, one is for interpretation of ASL sign alphabets and another is for image browsing.

Safe Driving using Vision-based Hand Gesture Recognition System in Non-uniform Illumination Conditions

Nowadays, there is tremendous growth in in-car interfaces for driver safety and comfort, but controlling these devices while driving requires the driver’s attention. One of the solutions to reduce the number of glances at these interfaces is to design an advanced driver assistance system (ADAS). A vision-based touch-less hand gesture recognition system is proposed here for in-car human-machine interfaces (HMI). The performance of such systems is unreliable under ambient illumination conditions, which change during the course of the day. Thus, the main focus of this work was to design a system that is robust towards changing lighting conditions. For this purpose, a homomorphic filter with adaptive thresholding binarization is used. Also, gray-level edge-based segmentation ensures that it is generalized for users of different skin tones and background colors. This work was validated on selected gestures from the Cambridge Hand Gesture Database captured in five sets of non-uniform illumination conditions that closely resemble in-car illumination conditions, yielding an overall system accuracy of 91%, an average frame-by-frame accuracy of 81.38%, and a latency of 3.78 milliseconds. A prototype of the proposed system was implemented on a Raspberry Pi 3 interface together with an Android application, which demonstrated its suitability for non-critical in-car interfaces like infotainment systems.

Analisis Penerapan Metode Convex Hull Dan Convexity Defects Untuk Pengenalan Isyarat Tangan

Research of Human Computer Interaction (HCI) and Computer Vision (CV) is increasingly focused on advanced interface for interacting with humans and creating system models for various purposes. Especially for input device problem to interact with computer. Humans are accustomed to communicate with fellow human beings using voice communication and accompanied by body pose and hand gesture. The main purpose of this research is to applying the Convex Hull and Convexity Defects methods for Hand Gesture Recognition system.
 In this research, the Hand Gesture Recognition system designed with the OpenCV library and then receives input from the user's hand gesture using an integrated webcam on the computer and system generates a language output from the hand-recognizable gestures.
 Testing involves several variables which affect success in recognizing user's hand gestures, such as hand distance with webcam, corner of the finger, light conditions and background conditions. As a result, the user's hand gestures can be recognized with a stable and accurate when at a distance of 50cm-70cm, corner of the finger 25o–70o, light conditions 150lux-460lux and plain background conditions.