Gesture Recognition Algorithm Research Articles

Algorithm for Gesture Recognition Using an IR-UWB Radar Sensor

This paper introduces a human gesture recognition algorithm using an impulse radio ultra-wide- band (IR-UWB) radar sensor. Human gesture recognition has been one of the hottest research topics for quite a long time. Many gesture recognition algorithms or systems using other sensors have been proposed such as using cameras, RFID tags and so on. Among which gesture recognition systems using cameras have been extensively studied in past years and widely used in practical. While it might show some deficiencies in some cases. For example, the users might not like to be filmed by cameras considering their privacies. Besides, it might not work well in very dark environments. While RFID tags could be inconvenient to many people and are likely to be lost. Our gesture recognition algorithm uses IR-UWB radar sensor which has pretty high resolution in ranging and adjustable gesture recognition range, meanwhile, does not have problems in privacy issues or darkness. In this paper, the gesture recognition algorithm is based on the moving direction and distance change of the human hand and the change of the frontal surface area of hand towards radar sensor. By combining these changes while doing gestures, the algorithm may recognize basically 6 kinds of hand gestures. The experimental results show that these gestures are of quite good performance. The performance analysis from experiments is also given.

Smartwatch-Enhanced Interaction with an Advanced Troubleshooting System for Industrial Machines

Smartwatches are unobtrusive everyday devices which can be also exploited for effective gesture-based human-machine interaction. In this paper, we propose the use of a smartwatch to interact with an advanced troubleshooting application to be used in industrial environment. The application is a hypermedia information system aiming at assisting workers in performing preventive and corrective machine maintenance. The smartwatch allows a handsfree interaction, thus facilitating the use of the whole system when wearing personal protective equipment such as gloves or having fingers greased with oil or dust, which prevent operating touch screens. The algorithm for gesture recognition we have devised, which is based on template matching, is described in the paper, together with its experimental validation. Finally, we present a preliminary usability assessment of the overall system, meant as integration of the smartwatch with the hypermedia system.

Sign recognition using key frame selection

This paper deals with static and dynamic hand gesture (digits) recognition. The method provides a threefold novel contribution: (1) segmentation algorithm gives better results on any skin colour and any size of hand on complex and non-uniform background; (2) key frame finding algorithm and (3) the recognition technique of signs (static digits, alphabets and dynamic digits). We separate out key frames from a sequence of static gestures, which include correct gestures from a video sequence. The recognition efficiency of key frame detection is 93% using the proposed algorithm. The segmentation efficiency is almost 95%. Features are extracted using the proposed feature extraction algorithm, and gestures are recognised. We propose a novel algorithm for static and dynamic gesture recognition. The proposed algorithm shows recognition efficiency of 94.8% for static gestures and 94% for dynamic gestures.

Recent methods and databases in vision-based hand gesture recognition: A review

Successful efforts in hand gesture recognition research within the last two decades paved the path for natural human–computer interaction systems. Unresolved challenges such as reliable identification of gesturing phase, sensitivity to size, shape, and speed variations, and issues due to occlusion keep hand gesture recognition research still very active. We provide a review of vision-based hand gesture recognition algorithms reported in the last 16 years. The methods using RGB and RGB-D cameras are reviewed with quantitative and qualitative comparisons of algorithms. Quantitative comparison of algorithms is done using a set of 13 measures chosen from different attributes of the algorithm and the experimental methodology adopted in algorithm evaluation. We point out the need for considering these measures together with the recognition accuracy of the algorithm to predict its success in real-world applications. The paper also reviews 26 publicly available hand gesture databases and provides the web-links for their download.

Gesture recognition using Markov Systems and wearable wireless inertial sensors

Wearable wireless devices and ubiquitous computing are expected to grow significantly in the coming years. Standard inputs such as a mouse and keyboard are not well suited for such mobile systems and gestures are seen as an effective alternative to these classic input styles. This paper examines gesture recognition algorithms that use an inertial sensor worn on the forearm. The recognition algorithms use the sensor's quaternion orientation in either a Hidden Markov Model or Markov Chain based approach. A set of six gestures were selected to fit within the context of an active video game. Despite the fact that the Hidden Markov Model is one of the most commonly used methods for gesture recognition, the experiments showed that the Markov Chain based algorithms outperformed the Hidden Markov Model. The Markov Chain algorithm obtained an average accuracy of 95%, while also having a much faster computation time, making it better suited for real time applications.

Distributed Spatiotemporal Gesture Recognition in Sensor Arrays

We present algorithms for gesture recognition using in-network processing in distributed sensor arrays embedded within systems such as tactile input devices, sensing skins for robotic applications, and smart walls. We describe three distributed gesture-recognition algorithms that are designed to function on sensor arrays with minimal computational power, limited memory, limited bandwidth, and possibly unreliable communication. These constraints cause storage of gesture templates within the system and distributed consensus algorithms for recognizing gestures to be difficult. Building up on a chain vector encoding algorithm commonly used for gesture recognition on a central computer, we approach this problem by dividing the gesture dataset between nodes such that each node has access to the complete dataset via its neighbors. Nodes share gesture information among each other, then each node tries to identify the gesture. In order to distribute the computational load among all nodes, we also investigate an alternative algorithm, in which each node that detects a motion will apply a recognition algorithm to part of the input gesture, then share its data with all other motion nodes. Next, we show that a hybrid algorithm that distributes both computation and template storage can address trade-offs between memory and computational efficiency.

Case Study: Hand Gesture Based Digital Pen

In this paper an MEMS accelerometer mostly based on gesture recognition algorithm and its application are presented. The hardware module consists of a triaxial MEMS accelerometer, microcontroller, and RF wireless transmission module for sensing and collecting accelerations of handwriting and hand gesture trajectories. Users will use this digital pen to write down digits, alphabets& some hand gestures. The accelerations of hand motions deliberate by the accelerometer are transmit wirelessly to a personal computer for trajectory recognition. The trajectory algorithm collected of information variety assortment, signal pre-processing for reconstruct the trajectory to satisfy the collective errors caused by drift of sensors. So, by altering the location of MEMS (micro electro mechanical systems) we can proficient to demonstrate the alphabetical characters and numerical on PC.

Expansion of Smartwatch Touch Interface from Touchscreen to Around Device Interface Using Infrared Line Image Sensors.

Touchscreen interaction has become a fundamental means of controlling mobile phones and smartwatches. However, the small form factor of a smartwatch limits the available interactive surface area. To overcome this limitation, we propose the expansion of the touch region of the screen to the back of the user’s hand. We developed a touch module for sensing the touched finger position on the back of the hand using infrared (IR) line image sensors, based on the calibrated IR intensity and the maximum intensity region of an IR array. For complete touch-sensing solution, a gyroscope installed in the smartwatch is used to read the wrist gestures. The gyroscope incorporates a dynamic time warping gesture recognition algorithm for eliminating unintended touch inputs during the free motion of the wrist while wearing the smartwatch. The prototype of the developed sensing module was implemented in a commercial smartwatch, and it was confirmed that the sensed positional information of the finger when it was used to touch the back of the hand could be used to control the smartwatch graphical user interface. Our system not only affords a novel experience for smartwatch users, but also provides a basis for developing other useful interfaces.

On an algorithm for Vision-based hand gesture recognition

A vision-based static hand gesture recognition method which consists of preprocessing, feature extraction, feature selection and classification stages is presented in this work. The preprocessing stage involves image enhancement, segmentation, rotation and filtering. This work proposes an image rotation technique that makes segmented image rotation invariant and explores a combined feature set, using localized contour sequences and block-based features for better representation of static hand gesture. Genetic algorithm is used here to select optimized feature subset from the combined feature set. This work also proposes an improved version of radial basis function (RBF) neural network to classify hand gesture images using selected combined features. In the proposed RBF neural network, the centers are automatically selected using k-means algorithm and estimated weight matrix is recursively updated, utilizing least-mean-square algorithm for better recognition of hand gesture images. The comparative performances are tested on two indigenously developed databases of 24 American sign language hand alphabet.

A Dynamic Gesture Trajectory Recognition Based on Key Frame Extraction and HMM

Aiming at changing high computational complexity, underdeveloped real time, low recognition rate of dynamic gesture recognition algorithms, this paper present a real-time dynamic gesture trajectory recognition method based on key frame extraction and HMM. Key frames are selected without keeping track of all the details of one dynamic gesture, which is based on difference degree between frames. The trajectory data stream, sorted by the time-warping algorithm, is used to construct the Hidden Markov Method model of dynamic gesture. Finally, optimal transition probabilities are employed to implement dynamic gesture recognition. The result of this experiment implies that this method has high robustness and real time. The average recognition rate of dynamic gesture (0~9) is up to 87.67%, and average time efficiency is 0.46s.

Similarity Matching-Based Extensible Hand Gesture Recognition

This paper presents an accelerometer-based smart ring and a similarity matching-based extensible hand gesture recognition algorithm. Users can wear the ring to perform gestures in 2-D space. The accelerations of hand motions are collected by the three-axis accelerometer, which is integrated in the ring. We divide the gestures into two types, i.e., the basic gesture and the complex gesture, which can be decomposed into a basic gesture sequence. A segmentation algorithm is developed to identify individual gestures in a sequence automatically. To recognize the basic gesture, a simple but effective feature based on the average jerk is extracted. The recognized basic gesture is then encoded by a Johnson code. Finally, the complex gesture is recognized by comparing the similarity between the obtained basic gesture sequence and the stored templates. A library of eight basic gestures and 12 complex gestures is created, and the users can easily define and add their own gestures without pretraining. The model discussed in this paper achieves a basic gesture recognition rate of 98.9% and a complex gesture recognition rate of 97.2%. Compared with complete matching, the proposed algorithm based on similarity matching improves the complex gesture recognition rate ~12%. Experimental results have successfully validated the feasibility and effectiveness of the gesture decomposition and similarity matching-based gesture recognition algorithm.

Real-time human pose estimation and gesture recognition from depth images using superpixels and SVM classifier.

In this paper, we present human pose estimation and gesture recognition algorithms that use only depth information. The proposed methods are designed to be operated with only a CPU (central processing unit), so that the algorithm can be operated on a low-cost platform, such as an embedded board. The human pose estimation method is based on an SVM (support vector machine) and superpixels without prior knowledge of a human body model. In the gesture recognition method, gestures are recognized from the pose information of a human body. To recognize gestures regardless of motion speed, the proposed method utilizes the keyframe extraction method. Gesture recognition is performed by comparing input keyframes with keyframes in registered gestures. The gesture yielding the smallest comparison error is chosen as a recognized gesture. To prevent recognition of gestures when a person performs a gesture that is not registered, we derive the maximum allowable comparison errors by comparing each registered gesture with the other gestures. We evaluated our method using a dataset that we generated. The experiment results show that our method performs fairly well and is applicable in real environments.

Adaptive and Personalized Gesture Recognition Using Textile Capacitive Sensor Arrays

Upper extremity mobility impairment is a common sequel of Spinal Cord Injury (SCI), brain injury, strokes, and degenerative diseases such as Guillain-Barre and ALS. Existing assistive technology solutions that provide access as user input devices are intrusive and expensive, and require physical contact that can have deleterious effects such as skin friction injury for paralyzed users who have reduced skin sensitivity. To address this problem, in this paper, we present the design, implementation, and evaluation of a non-contact proximity gesture recognition system using fabric capacitive sensor arrays. The fabric sensors are lightweight, flexible, and can be easily integrated into items of quotidian use such as clothing, bed sheets, and pillow covers. Our gesture recognition algorithm builds on two known classification techniques, Hidden Markov Model and Dynamic Time Warping to convert raw capacitance values to alphanumeric gestures. Our system is personalized to the user, allowing personalized selection of gesture sets and definition of gesture patterns in accordance with their capabilities. Our system adapts to changes in sensor configuration and orientation with minimal user training and intervention. We have evaluated our system in the context of a gesture-driven home automation system on six subjects that includes an individual who has a C6 Spinal Cord injury. We show that our system can recognize gestures of varying complexity with an average accuracy of 99 percent with minimal training.

A Track-Based Gesture Recognition Algorithm for Kinect

This paper presents a new gesture track recognition method based on the depth image information received from the Kinect sensor. First, a Kinect sensor is used to obtain the coordinates of a moving arm. Then, the gesture tracks corresponding to these coordinates are analyzed. Matching and recognition of gesture tracks are implemented by performing golden section search. The results show that this track-based method is highly effective in gesture recognition.

Remote Gesture Control Using HMM Classifier Based on Electric Potential Sensors

In this paper, HMM based non-contact gesture recognition using EPS(Electric Potential Sensor) is implemented, First, EMI(Electro-Magnetic Interference) in ELF(Extremely Low Frequency) band from surrounding environments is minimized through EMI removing process as well as preprocessing. Hand gesture recognition algorithms applied for smart TV remote control are presented and evaluated. Our proposed cumulative electric potential based “left to right model” is implemented and its performance is compared to that of an ergodic model based directional vectors. Also, we implement the HMM by finding the optimal numbers of states and quantization, and then compare its performance with other typical algorithms.

Depth camera-based 3D hand gesture controls with immersive tactile feedback for natural mid-air gesture interactions.

Vision-based hand gesture interactions are natural and intuitive when interacting with computers, since we naturally exploit gestures to communicate with other people. However, it is agreed that users suffer from discomfort and fatigue when using gesture-controlled interfaces, due to the lack of physical feedback. To solve the problem, we propose a novel complete solution of a hand gesture control system employing immersive tactile feedback to the user's hand. For this goal, we first developed a fast and accurate hand-tracking algorithm with a Kinect sensor using the proposed MLBP (modified local binary pattern) that can efficiently analyze 3D shapes in depth images. The superiority of our tracking method was verified in terms of tracking accuracy and speed by comparing with existing methods, Natural Interaction Technology for End-user (NITE), 3D Hand Tracker and CamShift. As the second step, a new tactile feedback technology with a piezoelectric actuator has been developed and integrated into the developed hand tracking algorithm, including the DTW (dynamic time warping) gesture recognition algorithm for a complete solution of an immersive gesture control system. The quantitative and qualitative evaluations of the integrated system were conducted with human subjects, and the results demonstrate that our gesture control with tactile feedback is a promising technology compared to a vision-based gesture control system that has typically no feedback for the user's gesture inputs. Our study provides researchers and designers with informative guidelines to develop more natural gesture control systems or immersive user interfaces with haptic feedback.

An Inertial Pen With Dynamic Time Warping Recognizer for Handwriting and Gesture Recognition

This paper presents an inertial-sensor-based digital pen (inertial pen) and its associated dynamic time warping (DTW)-based recognition algorithm for handwriting and gesture recognition. Users hold the inertial pen to write numerals or English lowercase letters and make hand gestures with their preferred handheld style and speed. The inertial signals generated by hand motions are wirelessly transmitted to a computer for online recognition. The proposed DTW-based recognition algorithm includes the procedures of inertial signal acquisition, signal preprocessing, motion detection, template selection, and recognition. We integrate signals collected from an accelerometer, a gyroscope, and a magnetometer into a quaternion-based complementary filter for reducing the integral errors caused by the signal drift or intrinsic noise of the gyroscope, which might reduce the accuracy of the orientation estimation. Furthermore, we have developed a minimal intra-class to maximal inter-class based template selection method (min-max template selection method) for a DTW recognizer to obtain a superior class separation for improved recognition. Experimental results have successfully validated the effectiveness of the DTW-based recognition algorithm for online handwriting and gesture recognition using the inertial pen.

Motion-Based Gesture Recognition Algorithms for Robot Manipulation

The prevailing trend of integrating inertial sensors in consumer electronics devices has inspired research on new forms of human-computer interaction utilizing hand gestures, which may be set-up on mobile devices themselves. At present, motion gesture recognition is intensely studied, with various recognition techniques being employed and tested. This paper provides an in-depth, unbiased comparison of different algorithms used to recognize gestures based primarily on the single 3D accelerometer recordings. The study takes two of the most popular and arguably the best recognition methods currently in use - dynamic time warping and hidden Markov model - and sets them against a relatively novel approach founded on distance metric learning. The three selected algorithms are evaluated in terms of their overall performance, accuracy, training time, execution time and storage efficacy. The optimal algorithm is further implemented in a prototype user application, aimed to serve as an interface for controlling the motion of a toy robot via gestures made with a smartphone.

A Gesture Recognition Algorithm Using Hausdorff-Like Distance Template Matching Based on the Main Direction of Gesture

Hand gesture of rotation, scaling and translation is the key problem of gesture recognition. This paper proposes a gesture recognition algorithm based on Hausdorff-like distance template matching of gesture main direction. Firstly, we segment hand gesture from video stream. Secondly, we calculate the main direction of gesture in the image, and build a 2D rectangular coordinate system. Then, we clockwise divide the gesture into eight sub-image area along the main direction of gesture and calculate the coordinates of target pixel points in each sub-image area in the 2D rectangular coordinate system. Finally, the algorithm of Hausdorff-like distance template matching is used to recognize the final gesture. Experimental results show that this algorithm can achieve real-time correct recognition of gestures in relatively stable light conditions. The overall recognition rate can reach 95%.

Regression Analysis for Gesture Recognition Using Passive RFID Technology in Smart Home Environments

Gesture recognition is a field of research that consists in recognizing patterns of movement performed by a human using his body parts with or without the help of a comprehensive device (a mouse, a laser, etc.). This particular area as attracted a number of researchers over the years that applied such algorithms in a broad range of disciplines. In particular, it was exploited on early research initiatives with pervasive environments to enable simple communication with automation systems. Nowadays, those environments are used for more than automation. Many researchers, in fact, believe it is one of the most promising solutions to the problems related to ageing of the population. Smart homes are seen as an alternative to the full-time support of a semi-autonomous person by healthcare professional and thus also a potentially economically viable solution to the rising cost of such support. However, researchers are still facing many challenges in that regards, such as the comprehension of the context and of the ongoing activity of daily living. In that equation, gesture recognition could help extract more information from the collected data and thus reinforce the context modeling. The knowledge extracted could even help with monitoring of more fine-grained activities and with the understanding of normal or abnormal behaviors. Gesture recognition is often considered as a solved problem since the techniques to perform it work well as soon as we can track accurately. However, in smart environment, tracking is very imprecise and hard to achieve with limited technology (i.e. noninvasive sensors). In this paper, we present a novel gesture recognition algorithm that works under uncertainty, and that is scalable to the precision of the tracking system. The algorithm is based on the tracking of passive RFID tags installed on all everyday life objects in a smart environment. A set of experimentation in simulation and in a real smart home environment is presented. The results are very encouraging despite the very low accuracy of the passive RFID tracking system.