Hand Sign Recognition Research Articles

Deep learning-based sign language recognition system for static signs

Sign language for communication is efficacious for humans, and vital research is in progress in computer vision systems. The earliest work in Indian Sign Language (ISL) recognition considers the recognition of significant differentiable hand signs and therefore often selecting a few signs from the ISL for recognition. This paper deals with robust modeling of static signs in the context of sign language recognition using deep learning-based convolutional neural networks (CNN). In this research, total 35,000 sign images of 100 static signs are collected from different users. The efficiency of the proposed system is evaluated on approximately 50 CNN models. The results are also evaluated on the basis of different optimizers, and it has been observed that the proposed approach has achieved the highest training accuracy of 99.72% and 99.90% on colored and grayscale images, respectively. The performance of the proposed system has also been evaluated on the basis of precision, recall and F-score. The system also demonstrates its effectiveness over the earlier works in which only a few hand signs are considered for recognition.

Intelligent metasurface imager and recognizer

There is an increasing need to remotely monitor people in daily life using radio-frequency probe signals. However, conventional systems can hardly be deployed in real-world settings since they typically require objects to either deliberately cooperate or carry a wireless active device or identification tag. To accomplish complicated successive tasks using a single device in real time, we propose the simultaneous use of a smart metasurface imager and recognizer, empowered by a network of artificial neural networks (ANNs) for adaptively controlling data flow. Here, three ANNs are employed in an integrated hierarchy, transforming measured microwave data into images of the whole human body, classifying specifically designated spots (hand and chest) within the whole image, and recognizing human hand signs instantly at a Wi-Fi frequency of 2.4 GHz. Instantaneous in situ full-scene imaging and adaptive recognition of hand signs and vital signs of multiple non-cooperative people were experimentally demonstrated. We also show that the proposed intelligent metasurface system works well even when it is passively excited by stray Wi-Fi signals that ubiquitously exist in our daily lives. The reported strategy could open up a new avenue for future smart cities, smart homes, human-device interaction interfaces, health monitoring, and safety screening free of visual privacy issues.

Multi-Modal Deep Hand Sign Language Recognition in Still Images Using Restricted Boltzmann Machine.

In this paper, a deep learning approach, Restricted Boltzmann Machine (RBM), is used to perform automatic hand sign language recognition from visual data. We evaluate how RBM, as a deep generative model, is capable of generating the distribution of the input data for an enhanced recognition of unseen data. Two modalities, RGB and Depth, are considered in the model input in three forms: original image, cropped image, and noisy cropped image. Five crops of the input image are used and the hand of these cropped images are detected using Convolutional Neural Network (CNN). After that, three types of the detected hand images are generated for each modality and input to RBMs. The outputs of the RBMs for two modalities are fused in another RBM in order to recognize the output sign label of the input image. The proposed multi-modal model is trained on all and part of the American alphabet and digits of four publicly available datasets. We also evaluate the robustness of the proposal against noise. Experimental results show that the proposed multi-modal model, using crops and the RBM fusing methodology, achieves state-of-the-art results on Massey University Gesture Dataset 2012, American Sign Language (ASL). and Fingerspelling Dataset from the University of Surrey’s Center for Vision, Speech and Signal Processing, NYU, and ASL Fingerspelling A datasets.

Kurdish Sign Language Recognition System

Deaf people all around the world face difficulty to communicate with the others. Hence, they use their own language to communicate with each other. On the other hand, it is difficult for deaf people to get used to technological services such as websites, television, mobile applications, and so on. This project aims to design a prototype system for deaf people to help them to communicate with other people and computers without relying on human interpreters. The proposed system is for letter-based Kurdish Sign Language (KuSL) which has not been introduced before. The system would be a real-time system that takes actions immediately after detecting hand gestures. Three algorithms for detecting KuSL have been implemented and tested, two of them are well-known methods that have been implemented and tested by other researchers, and the third one has been introduced in this paper for the 1st time. The new algorithm is named Gridbased gesture descriptor. It turned out to be the best method for the recognition of Kurdish hand signs. Furthermore, the result of the algorithm was 67% accuracy of detecting hand gestures. Finally, the other well-known algorithms are named scale invariant feature transform and speeded-up robust features, and they responded with 42% of accuracy.

Off-Chip Training with Additive Perturbation for FPGA-Based Hand Sign Recognition System

Off-Chip Training with Additive Perturbation for FPGA-Based Hand Sign Recognition System

Implementation of Hand Sign Recognition for Non-Linear Dimensionality Reduction based on PCA

Implementation of Hand Sign Recognition for Non-Linear Dimensionality Reduction based on PCA

Depth Sensor Based Automatic Hand Region Extraction by Using Time-series Curve and Its Application to Japanese Finger-spelled Sign Language Recognition

Abstract Hand sign recognition is one of most challenging issues in computer vision and human computer interaction, and many researchers tackle this issue. In this research, we focus on JFSL (Japanese Finger-spelled Sign Language) which is one of hand signs. The tasks for achieving high performance of JFSL recognition as well as other hand signs are how to extract hand region precisely and how to recognize hand signs accurately. To deal with the former task, in this paper, we propose an automatic hand region extraction method with a depth sensor. The characteristic points of our proposed method are to utilize Time-Series Curve, which is one of contour features, and to extract hand region accurately without wearing landmark object such as a color wristband. On the other hand, to tackle the latter task, in this research, we focus on a deep neural network based recognition method since such a method is reported that it allows us to achieve high performance for various recognition tasks. Therefore, in this paper, we investigate JFSL recognition performance with a deep neural network approach compared to that with the conventional image recognition method (HOG+SVM). From the experimental results with 8 subjects, we have confirmed that our proposed method allows us to extract hand region accurately regardless of subjects and JFSL signs. In addition, from the experimental results with a deep neural network based recognition method for JFSL recognition, we have achieved at least average recognition rate over 88%.

3축 모션 센서 기반 SWAT 수신호 모션 인식 시스템 설계 및 구현

Hand-signal is an effective communication means in the situation where voice cannot be used for expression especially for soldiers. Vision-based approaches using cameras as input devices are widely suggested in the literature. However, these approaches are not suitable for soldiers that have unseen visions in many cases. in addition, existing special-glove approaches utilize the information of fingers only. Thus, they are still lack for soldiers' hand-sign al recognition that involves not only finger motions, but also additional information such as the rotation of a hand. In this paper, we have designed and implemented a new recognition system for six military hand-signal motions, i. e., 'ready', 'move', quick move', 'crawl', 'stop', and 'lying-down'. For this purpose, we have proposed a finger-recognition method and motion-recognition methods. The finger-recognition method discriminate how much each finger is bended, i. e., 'completely flattened', 'slightly flattened', 'slightly bended', and 'completely bended'. The motion-recognition algorithms are based on the characterization of each hand-signal motion in terms of the three axes. Through repetitive experiments, our system have shown 91.2% of correct recognition.

Vision based Traffic Police Hand Signal Recognition in Surveillance Video - A Survey

Human gesture recognition has become a very important topic in computer vision. The purpose of this survey is to provide a detailed overview and categories of current issues and trends. The recognition of human hand gesture movement can be performed at various level of abstraction. This survey concentrate on approaches that aim on recognizing traffic police hand signals. Many application and algorithms were discussed with the recognition framework. General overview of an traffic control gestures and its various applications where discussed in this paper. Most of the recognition system uses the benchmark datasets like KTH, Weizmann. some other datasets were used by the action recognition system. In this paper image representation,action representation, human action detection,feature extraction and human action recognition were also discussed.

A biometric authentication model using hand gesture images

A novel hand biometric authentication method based on measurements of the user’s stationary hand gesture of hand sign language is proposed. The measurement of hand gestures could be sequentially acquired by a low-cost video camera. There could possibly be another level of contextual information, associated with these hand signs to be used in biometric authentication. As an analogue, instead of typing a password ‘iloveu’ in text which is relatively vulnerable over a communication network, a signer can encode a biometric password using a sequence of hand signs, ‘i’ , ‘l’ , ‘o’ , ‘v’ , ‘e’ , and ‘u’. Subsequently the features from the hand gesture images are extracted which are integrally fuzzy in nature, to be recognized by a classification model for telling if this signer is who he claimed himself to be, by examining over his hand shape and the postures in doing those signs. It is believed that everybody has certain slight but unique behavioral characteristics in sign language, so are the different hand shape compositions. Simple and efficient image processing algorithms are used in hand sign recognition, including intensity profiling, color histogram and dimensionality analysis, coupled with several popular machine learning algorithms. Computer simulation is conducted for investigating the efficacy of this novel biometric authentication model which shows up to 93.75% recognition accuracy.

Combined Gesture-Speech Recognition and Synthesis Using Neural Networks

Sign languages such as Spanish Sign Language (LSE) are the primary communication way among members of the Deaf community. However this language is not widely known outside of this community. The techniques for automatic recognizing hand signs proposed in this paper allow creating systems which can help deaf people to communicate with others, by providing them with computer tools for assisted communication or potentially with a fully automatic portable sign-language-to-speech translation system. The aim of this research it to implement a self-organizing neural network based technique for hand sign recognition, using self organizing map (SOM) and learning vector quantization (LVQ) based algorithms. The two classifiers are then combined to make the final decision. SOM and LVQ classifiers are training for the 26 signs of the manual alphabet and the speech synthesizer concatenate the different phones, diphones and triphones stored in the database to generate the right words in artificial speech. Finally, the system is to contribute to the implementation of meaningful human machine interactions in a workstation, mainly for welfare applications.

What Comes Down Must Go Up: Training Shipboard Helicopter Signaling Skills

This paper discusses the findings from a project to develop a low-cost, effective training solution for shipboard helicopter signaling skills. We identified training objectives, explored training/simulation concepts, identified signal task training requirements, and demonstrated a prototype concept. The training solution includes an innovative prototype Hand Signal Recognition System that utilizes computer vision, a personal computer, a video camera, image acquisition and processing software, and a decision-making algorithm to recognize hand signals in support of signal task training for the Navy Landing Signal Enlisted (LSE) position. The training solution is practical for LSE signal training at the schoolhouse, capable of being deployed aboard ship with some constraints, and cost-effective. The training techniques and hand signal recognition technology developed in this project, and the lessons learned about training shipboard signaling skills, are useful to a variety of training and human factors professionals. The training and technology are relevant to a variety of applications such as emergency medical evacuation, search and rescue, oil platform operations, and shipboard control of unmanned aerial vehicles.

Cooperation of different neuronal systems during hand sign recognition

Hand signs with symbolic meaning can often be utilized more successfully than words to communicate an intention; however, the underlying brain mechanisms are undefined. The present study using magnetoencephalography (MEG) demonstrates that the primary visual, mirror neuron, social recognition and object recognition systems are involved in hand sign recognition. MEG detected well-orchestrated multiple brain regional electrical activity among these neuronal systems. During the assessment of the meaning of hand signs, the inferior parietal, superior temporal sulcus (STS) and inferior occipitotemporal regions were simultaneously activated. These three regions showed similar time courses in their electrical activity, suggesting that they work together during hand sign recognition by integrating information in the ventral and dorsal pathways through the STS. The results also demonstrated marked right hemispheric predominance, suggesting that hand expression is processed in a manner similar to that in which social signs, such as facial expressions, are processed.

EMG pattern classification using SOFMs for hand signal recognition

We propose a method of pattern classification of electromyographic (EMG) signals using a set of self- organizing feature maps (SOFMs). The proposed method is simple to apply in that the EMG signals are directly input to the SOFMs without preprocessing. Experimental results are presented that show the effectiveness of the SOFM based classifier for the recognition of the hand signal version of the Korean alphabet from EMG signal patterns.

Appearance-Based Hand Sign Recognition from Intensity Image Sequences

In this paper, we present a new approach to recognizing hand signs. In this approach, motion recognition (the hand movement) is tightly coupled with spatial recognition (hand shape). The system uses multiclass, multidimensional discriminant analysis to automatically select the most discriminating linear features for gesture classification. A recursive partition tree approximator is proposed to do classification. This approach combined with our previous work on hand segmentation forms a new framework which addresses the three key aspects of hand sign interpretation: hand shape, location, and movement. The framework has been tested to recognize 28 different hand signs. The experimental results show that the system achieved a 93.2% recognition rate for test sequences that had not been used in the training phase. It is shown that our approach provide performance better than that of nearest neighbor classification in the eigensubspace.

Using The 3D Hand Model To Recognize Stationary Hand Signs

Sign language is the primary communication means for the hearing impaired because typewriting or handwriting cannot provide equivalent power of expression, naturalness, or convenience. The capability to computer-process sign language has consequently received much research attention. This paper presents the 3D hand model for performing hand sign recognition based on 3D “structural features” of the hand, that is, flexion of joints and contactness of fingers. This technique mitigates the weaknesses in conventional approaches, as the 3D structural features of the hand extracted using the hand model are stable against perturbations and provide more clues to distinguish hand signs. The effectiveness and capability of our approach is successfully demonstrated by testing it on the American Sign Language manual alphabet.