Gesture Recognition Applications Research Articles

Compiling a gesture recognition application for a low-power spatial architecture

Energy efficiency is one of the main performance goals when designing processors for embedded systems. Typically, the simpler the processor, the less energy it consumes. Thus, an ultra-low power multicore processor will, likely have very small distributed memory with a simple interconnect. To compile for such an architecture, a partitioning strategy that can tune between space and communication minimization is crucial to fit a program in its limited resources and achieve good performance. A careful program layout design is also critical. Aside fulfilling the space constraint, a compiler needs to be able to optimize for program latency to satisfy a certain timing requirement as well. To satisfy all aforementioned constraints, we present a flexible code partitioning strategy and light-weight mechanisms to express parallelism and program layout. First, we compare two strategies for partitioning program structures and introduce a language construct to let programmers choose which strategies to use and when. The compiler then partitions program structures with a mix of both strategies. Second, we add supports for programmer-specified parallelism and program layout through imposing additional spatial constraints to the compiler. We evaluate our compiler by implementing an accelerometer-based gesture recognition application on GA144, a recent low-power minimalistic multicore architecture. When compared to MSP430, GA144 is overall 19x more energy-efficient and 23x faster when running this application. Without these inventions, this application would not be able to fit on GA144.

A Gesture Recognition System for Detecting Behavioral Patterns of ADHD.

We present an application of gesture recognition using an extension of dynamic time warping (DTW) to recognize behavioral patterns of attention deficit hyperactivity disorder (ADHD). We propose an extension of DTW using one-class classifiers in order to be able to encode the variability of a gesture category, and thus, perform an alignment between a gesture sample and a gesture class. We model the set of gesture samples of a certain gesture category using either Gaussian mixture models or an approximation of convex hulls. Thus, we add a theoretical contribution to classical warping path in DTW by including local modeling of intraclass gesture variability. This methodology is applied in a clinical context, detecting a group of ADHD behavioral patterns defined by experts in psychology/psychiatry, to provide support to clinicians in the diagnose procedure. The proposed methodology is tested on a novel multimodal dataset (RGB plus depth) of ADHD children recordings with behavioral patterns. We obtain satisfying results when compared to standard state-of-the-art approaches in the DTW context.

Handwriting Recognition in Free Space Using WIMU-Based Hand Motion Analysis

We present a wireless-inertial-measurement-unit- (WIMU-) based hand motion analysis technique for handwriting recognition in three-dimensional (3D) space. The proposed handwriting recognition system is not bounded by any limitations or constraints; users have the freedom and flexibility to write characters in free space. It uses hand motion analysis to segment hand motion data from a WIMU device that incorporates magnetic, angular rate, and gravity sensors (MARG) and a sensor fusion algorithm to automatically distinguish segments that represent handwriting from nonhandwriting data in continuous hand motion data. Dynamic time warping (DTW) recognition algorithm is used to recognize handwriting in real-time. We demonstrate that a user can freely write in air using an intuitive WIMU as an input and hand motion analysis device to recognize the handwriting in 3D space. The experimental results for recognizing handwriting in free space show that the proposed method is effective and efficient for other natural interaction techniques, such as in computer games and real-time hand gesture recognition applications.

Explore Efficient Local Features from RGB-D Data for One-Shot Learning Gesture Recognition.

Availability of handy RGB-D sensors has brought about a surge of gesture recognition research and applications. Among various approaches, one shot learning approach is advantageous because it requires minimum amount of data. Here, we provide a thorough review about one-shot learning gesture recognition from RGB-D data and propose a novel spatiotemporal feature extracted from RGB-D data, namely mixed features around sparse keypoints (MFSK). In the review, we analyze the challenges that we are facing, and point out some future research directions which may enlighten researchers in this field. The proposed MFSK feature is robust and invariant to scale, rotation and partial occlusions. To alleviate the insufficiency of one shot training samples, we augment the training samples by artificially synthesizing versions of various temporal scales, which is beneficial for coping with gestures performed at varying speed. We evaluate the proposed method on the Chalearn gesture dataset (CGD). The results show that our approach outperforms all currently published approaches on the challenging data of CGD, such as translated, scaled and occluded subsets. When applied to the RGB-D datasets that are not one-shot (e.g., the Cornell Activity Dataset-60 and MSR Daily Activity 3D dataset), the proposed feature also produces very promising results under leave-one-out cross validation or one-shot learning.

Layered compression for high-precision depth data.

With the development of depth data acquisition technologies, access to high-precision depth with more than 8-b depths has become much easier and determining how to efficiently represent and compress high-precision depth is essential for practical depth storage and transmission systems. In this paper, we propose a layered high-precision depth compression framework based on an 8-b image/video encoder to achieve efficient compression with low complexity. Within this framework, considering the characteristics of the high-precision depth, a depth map is partitioned into two layers: 1) the most significant bits (MSBs) layer and 2) the least significant bits (LSBs) layer. The MSBs layer provides rough depth value distribution, while the LSBs layer records the details of the depth value variation. For the MSBs layer, an error-controllable pixel domain encoding scheme is proposed to exploit the data correlation of the general depth information with sharp edges and to guarantee the data format of LSBs layer is 8 b after taking the quantization error from MSBs layer. For the LSBs layer, standard 8-b image/video codec is leveraged to perform the compression. The experimental results demonstrate that the proposed coding scheme can achieve real-time depth compression with satisfactory reconstruction quality. Moreover, the compressed depth data generated from this scheme can achieve better performance in view synthesis and gesture recognition applications compared with the conventional coding schemes because of the error control algorithm.

Statistical based Neural Network in Human Activity Recognition

This research proposed an efficient method in classification of Human Activity Recognition tasks. The evaluated tuned models show higher than 99 percent mean accuracy and gain more training and testing accuracy in comparison to previous studies. Dimensionally reduction have been introduced based on P-value evaluation in feature space. Finally a hybrid model that compressed statistically in feature space alongside with Neural Network architecture have been proposed. The final model could be used as best architecture of hardware implementation in gesture recognition applications.

Nonlinear multiscale Maximal Lyapunov Exponent for accurate myoelectric signal classification

Surface Electromyography (sEMG) is a non-invasive, easy to record signal of superficial muscles from the skin surface. The sEMG is widely used in evaluating the functional status of the hand to assist in hand gesture recognition, prosthetics and rehabilitation applications. Considering the nonlinear and non-stationary characteristics of sEMG, hand gesture recognition using sEMG signals necessitate designers to use Maximal Lyapunov Exponent (MLE) or ensemble Empirical Mode Decomposition (EMD) based MLEs. In this research, we propose a hand gesture recognition method of sEMG based on nonlinear multiscale MLE. The aim is to increase the classification accuracy of sEMG features while reducing the complexity of EMD. The nonlinear MLE features are classified using Flexible Neural Tree (FNT), which can solve highly structured dependent problems of the Artificial Neural Network (ANN). The testing has been conducted using several experiments with five participants. The classification performance of nonlinear multiscale MLE method is compared with MLE and EMD-based MLE through simulations. Experimental results demonstrate that the former algorithm outperforms the two latter algorithms and can classify six different hand gestures up to 97.6% accuracy.

Similarity invariant partial shape matching using coarse-to-fine strategy

The matching between an open contour and a closed contour is a basis for the alignment of their common part and a similarity measure. We propose a coarse-to-fine method for partial shape matching, which does not need to scan the target shape, construct a codebook of model contour fragments, or depend on background support domains. For this purpose, a linearization algorithm for partial shapes is introduced to extract the initial shape segments from the closed contour those possibly match with the open contour. The next refining procedure of the coarse matching eliminates significantly dissimilar shape segments to reduce the further processing of fine matching. We propose a shape similarity description to finely describe the similarity between the open contour and the remaining shape segments. Finally, an order-preserving point injection between the open contour and the closed contour is established. Valuations of the proposed method on a benchmark dataset and real images demonstrate that the overall and component performances are excellent and robust to various disturbances and similarity transformations. Last, a gesture recognition application is implemented.

Face and Hand Gesture Recognition using Principle Component Analysis and kNN Classifier

Gesture recognition pertains to recognizing meaningful expressions of motion by a human, involving the hands, arms, face, head, and/or body. It is of utmost importance in designing an intelligent and efficient human–computer interface. The applications of gesture recognition are manifold, ranging from sign language through medical rehabilitation to virtual reality. In this paper, we provide a survey on gesture recognition with particular emphasis on hand gestures and facial expressions. Applications involving wavelet transform and principal component analysis for face and hand gesture recognition on digital images. General Terms Face & hand gesture recognition

Nonnegative matrix factorization for the identification of EMG finger movements: evaluation using matrix analysis.

Surface electromyography (sEMG) is widely used in evaluating the functional status of the hand to assist in hand gesture recognition, prosthetics and rehabilitation applications. The sEMG is a noninvasive, easy to record signal of superficial muscles from the skin surface. Considering the nonstationary characteristics of sEMG, recent feature selection of hand gesture recognition using sEMG signals necessitate designers to use nonnegative matrix factorization (NMF)-based methods. This method exploits both the additive and sparse nature of signals by extracting accurate and reliable measurements of sEMG features using a minimum number of sensors. The testing has been conducted for simple and complex finger flexions using several experiments with artificial neural network classification scheme. It is shown, both by simulation and experimental studies, that the proposed algorithm is able to classify ten finger flexions (five simple and five complex finger flexions) recorded from two sEMG sensors up to 92% (95% for simple and 87% for complex flexions) accuracy. The recognition performances of simple and complex finger flexions are also validated with NMF permutation matrix analysis.

Fusion of Inertial and Depth Sensor Data for Robust Hand Gesture Recognition

This paper presents the first attempt at fusing data from inertial and vision depth sensors within the framework of a hidden Markov model for the application of hand gesture recognition. The data fusion approach introduced in this paper is general purpose in the sense that it can be used for recognition of various body movements. It is shown that the fusion of data from the vision depth and inertial sensors act in a complementary manner leading to a more robust recognition outcome compared with the situations when each sensor is used individually on its own. The obtained recognition rates for the single hand gestures in the Microsoft MSR data set indicate that our fusion approach provides improved recognition in real-time and under realistic conditions.

English

Moving beyond mouse and keyboard, the evolution of human-computer interaction (HCI) has been an interesting research field in recent years which changes from text-based like using a keyboard to graphic user interface (GUI) based on a mouse, from cumbersome data gloves and tracking devices to visual-based computer application. One of the interesting fields is by using hand gestures to interact with computer. Compared to many other methods, hand gestures have the advantages of being easy to use, natural and intuitive. Successful applications of hand gesture recognition include computer games control, human-robot interaction, and sign language recognition etc. In this paper, a real time vision based hand gesture interaction method is proposed .The basis of our approach is a fast segmentation process with CIELab colour space which separate hand from the image which is given as input by camera and then apply morphological operation for the removal of noise .

KinectFlix: A Hand Gesture Recognition Application for Kinect to Watch NetFlix

KinectFlix is a hand gesture recognition application which uses a Xbox 360 Kinect to interact with NetFlix while watching online movies and TV shows. This project is designed and developed with the intention of providing an easily adaptable interface for the Xbox 360 Kinect to be used with Windows Presentation Foundation (WPF) applications to provide a controller-less interactive user interface.

Natural Hand Gestures Recognition System for Intelligent HCI: A Survey

Gesture recognition is to recognizing meaningful expressions of motion by a human, involving the hands, arms, face, head, and/or body. Hand Gestures have greater importance in designing an intelligent and efficient human–computer interface. The applications of gesture recognition are manifold, ranging from sign language through medical rehabilitation to virtual reality. In this paper a survey on various recent gesture recognition approaches is provided with particular emphasis on hand gestures. A review of static hand posture methods are explained with different tools and algorithms applied on gesture recognition system, including connectionist models, hidden Markov model, and fuzzy clustering. Challenges and future research directions are also highlighted.

Online gesture recognition from pose kernel learning and decision forests

The recent popularization of real time depth sensors has diversified the potential applications of online gesture recognition to end-user natural user interface (NUI). This requires significant robustness of the gesture recognition to cope with the noisy data from the popular depth sensor, while the quality of the final NUI heavily depends on the recognition execution speed. This work introduces a method for real-time gesture recognition from a noisy skeleton stream, such as those extracted from Kinect depth sensors. Each pose is described using an angular representation of the skeleton joints. Those descriptors serve to identify key poses through a Support Vector Machine multi-class classifier, with a tailored pose kernel. The gesture is labeled on-the-fly from the key pose sequence with a decision forest, which naturally performs the gesture time control/warping and avoids the requirement for an initial or neutral pose. The proposed method runs in real time and its robustness is evaluated in several experiments.

A 94 GHz mm-Wave-to-Baseband Pulsed-Radar Transceiver with Applications in Imaging and Gesture Recognition

High-resolution mm-wave array beamformers have applications in medical imaging, gesture recognition, and navigation. A scalable array architecture for 3D imaging is proposed in which single-element phase coherent transceiver (TRX) chips, with programmable TX pulse delay capability, are mounted on a common board to realize the array. This paper presents the design of the enabling TRX chip: a highly integrated 94 GHz phase-coherent pulsed-radar with on-chip antennas. The TRX achieves 10 GHz of frequency tuning range and 300 ps of contiguous pulse position control, enabling its usage in the large-array imager with time-domain TX beamforming. The TRX is capable of transmitting and receiving pulses down to 36 ps, translating to 30 GHz of bandwidth. Interferometric measurements show the TRX can obtain single-target range resolution better than 375 μm (limited by equipment). Based on delay measurements, the time of arrival rms error would be less than 1.3 ps which, if used in a 3D imaging array, leads to less than 0.36 mm of RMS error in voxel size and position.

Real-time computation of disparity for hand-pair gesture recognition using a stereo webcam

This paper presents an algorithm for the real-time computation of disparity using video stereo images captured by a stereo webcam. This algorithm is designed to provide both real-time throughput and robust disparity estimation for real-world applications where computation is limited to a pre-defined region-of-interest (ROI). More specifically, this algorithm is used as part of a hand-pair gesture recognition application where the disparity is computed for two ROI around a hand-pair identified by the segmentation component of the recognition application. The developed algorithm provides the required relative difference in disparity with background at high frame rates for the hand-pair gesture recognition application. The results obtained with an inexpensive commercial VGA stereo webcam show a robust disparity computation of 20 ms/frame enabling real-time hand-pair gesture recognition at 25 fps with >90% recognition rate for a maximum hand speed of 40 cm/s and for hand distances between 30 and 150 cm away from the camera.

GeoPointer – approaching tangible augmentation of the real world

PurposeThe aim of this paper is to present an architecture and prototypical implementation of a context‐sensitive software system which combines the tangible user interface approach with a mobile augmented reality (AR) application.Design/methodology/approachThe work which is described within this paper is based on a creational approach, which means that a prototypical implementation is used to gather further research results. The prototypical approach allows performing ongoing tests concerning the accuracy and different context‐sensitive threshold functions.FindingsWithin this paper, the implementation and practical use of tangible user interfaces for outdoor selection of geographical objects is reported and discussed in detail.Research limitations/implicationsFurther research is necessary within the area of context‐sensitive dynamically changing threshold functions, which would allow improving the accuracy of the selected tangible user interface approach.Practical implicationsThe practical implication of using tangible user interfaces within outdoor applications should improve the usability of AR applications.Originality/valueDespite the fact that there exist a multitude of research results within the area of gesture recognition and AR applications, this research work focuses on the pointing gesture to select outdoor geographical objects.

Silhouette-based gesture and action recognition via modeling trajectories on Riemannian shape manifolds

This paper addresses the problem of recognizing human gestures from videos using models that are built from the Riemannian geometry of shape spaces. We represent a human gesture as a temporal sequence of human poses, each characterized by a contour of the associated human silhouette. The shape of a contour is viewed as a point on the shape space of closed curves and, hence, each gesture is characterized and modeled as a trajectory on this shape space. We propose two approaches for modeling these trajectories. In the first template-based approach, we use dynamic time warping (DTW) to align the different trajectories using elastic geodesic distances on the shape space. The gesture templates are then calculated by averaging the aligned trajectories. In the second approach, we use a graphical model approach similar to an exemplar-based hidden Markov model, where we cluster the gesture shapes on the shape space, and build non-parametric statistical models to capture the variations within each cluster. We model each gesture as a Markov model of transitions between these clusters. To evaluate the proposed approaches, an extensive set of experiments was performed using two different data sets representing gesture and action recognition applications. The proposed approaches not only are successfully able to represent the shape and dynamics of the different classes for recognition, but are also robust against some errors resulting from segmentation and background subtraction.

Improvement in Recognition Techniques for Human Computer Interaction

Hand gestures are an important modality for human computer interaction (HCI) [1]. Compared to many existing interfaces, hand gestures have the advantages of being easy to use, natural, and intuitive. Successful applications of hand gesture recognition include computer games control [2], human-robot interaction [3], and sign language recognition [4], to name a few. Vision-based recognition systems can give computers the capability of understanding and responding to hand gestures. The usability of such systems greatly depends on their ability to function reliably in common real-world environments, without requiring the user to wear special clothes or cumbersome devices such as colored markers or gloves [4]. The aim of this technique is the proposal of a real time vision system for its application within visual interaction environments through hand gesture recognition, using general-purpose hardware and low cost sensors, like a simple personal computer and an USB web cam, so any user could make use of it in his office or home. The basis of our approach is a fast segmentation process to obtain the moving hand from the whole image, which is able to deal with a large number of hand shapes against different backgrounds and lighting conditions, and a recognition process that identifies the hand posture from the temporal sequence of segmented hands. The use of a visual memory (Stored database) allows the system to handle variations within a gesture and speed up the recognition process through the storage of different variables related to each gesture. A hierarchical gesture recognition algorithm is introduced to recognize a large number of gestures. Three stages of the proposed algorithm are based on a new hand tracking technique to recognize the actual beginning of a gesture using a Kalman filtering process, hidden Markov models and graph matching. Processing time is important in working with large databases. Therefore, special cares are taken to deal with the large number of gestures.