Gesture Tracking Research Articles

Computer Methods for Automatic Locomotion and Gesture Tracking in Mice and Small Animals for Neuroscience Applications: A Survey.

Neuroscience has traditionally relied on manually observing laboratory animals in controlled environments. Researchers usually record animals behaving freely or in a restrained manner and then annotate the data manually. The manual annotation is not desirable for three reasons; (i) it is time-consuming, (ii) it is prone to human errors, and (iii) no two human annotators will 100% agree on annotation, therefore, it is not reproducible. Consequently, automated annotation for such data has gained traction because it is efficient and replicable. Usually, the automatic annotation of neuroscience data relies on computer vision and machine learning techniques. In this article, we have covered most of the approaches taken by researchers for locomotion and gesture tracking of specific laboratory animals, i.e. rodents. We have divided these papers into categories based upon the hardware they use and the software approach they take. We have also summarized their strengths and weaknesses.

Plasma treated MXene/Ag-based humidity sensor with ultrahigh sensitivity for gesture tracking

A novel humidity sensor based on MXene (Ti3C2) combined with Ag nanoparticles was prepared in this paper. Owing to high surface-to-volume ratio and outstanding hydrophilicity of sensing film, the humidity sensor shows ultrahigh sensitivity. Moreover, this kind of sensor exhibits fast response speed which can detect human breath. Meanwhile, with a 7×7 sensor array, hand gesture can be tracked without direct contact of these sensors by hand which provides an unique approach for human–machine interaction without using machine vision.

Toward Device-Free Micro-Gesture Tracking via Accurate Acoustic Doppler-Shift Detection

As a universal interaction method, hand and finger gestures can express people’s intention directly and clearly in daily life, and has been one of the hotspots in human–computer interaction community. In this paper, we present <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DMT</i> , a device-free finger gesture tracking system that can track and recognize the finger motion accurately. To achieve this, we transform the mobile device, such as a smart phone, into an active sonar system by establishing inaudible audio links between the built-in speakers and microphone. The finger motion will have an effect (e.g., Doppler-shift) on the audio signals, which makes it possible to track the finger motion according to the received signal characteristics at the microphone side. Due to the small reflection energy and slow moving speed of the finger, the existing methods cannot detect the Doppler-shift accurately. To this end, a Fourier fitting-based method is proposed in the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DMT</i> to accurately detect the Doppler-shift. With the detected Doppler-shift, the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DMT</i> can track the finger motion with high accuracy. The <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DMT</i> supports all kinds of finger gestures interaction, including characters and shapes. Extensive experiments demonstrate the high accuracy and robustness of the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DMT</i> in dynamic environments.

Large-Screen Interactive Imaging System with Switching Federated Filter Method Based on 3D Sensor

Large-screen human-computer interaction technology is reflected in all aspects of daily life. The dynamic gesture tracking algorithm commonly used in recent large-screen interactive technologies demonstrates compelling results but suffers from accuracy and real-time problems. This paper systematically addresses these issues by a switching federated filter method that combines particle filtering and Mean Shifting algorithms based on a 3D sensor. Compared with several algorithms, the results show that the one-hand and two-hand large-screen gesture tracking based on the switched federated filtering algorithm work well, and there is no tracking failure and loss of target. Therefore, the switching federated tracking positioning algorithm can be well applied to the design of human-computer interaction system and provide new ideas for future human-computer interaction.

Gesture Elicitation Studies for Mid-Air Interaction: A Review

Mid-air interaction involves touchless manipulations of digital content or remote devices, based on sensor tracking of body movements and gestures. There are no established, universal gesture vocabularies for mid-air interactions with digital content or remote devices based on sensor tracking of body movements and gestures. On the contrary, it is widely acknowledged that the identification of appropriate gestures depends on the context of use, thus the identification of mid-air gestures is an important design decision. The method of gesture elicitation is increasingly applied by designers to help them identify appropriate gesture sets for mid-air applications. This paper presents a review of elicitation studies in mid-air interaction based on a selected set of 47 papers published within 2011–2018. It reports on: (1) the application domains of mid-air interactions examined; (2) the level of technological maturity of systems at hand; (3) the gesture elicitation procedure and its variations; (4) the appropriateness criteria for a gesture; (5) participants number and profile; (6) user evaluation methods (of the gesture vocabulary); (7) data analysis and related metrics. This paper confirms that the elicitation method has been applied extensively but with variability and some ambiguity and discusses under-explored research questions and potential improvements of related research.

Robust RGB-D Hand Tracking Using Deep Learning Priors

With the irruption of inexpensive depth sensor devices, hand gesture tracking has become a topic of great interest. Two main problems to face respect other tracking algorithms are the high complexity of the hand structure, which translate in a very large amount of possible gestures, and the rapidness of the movements we are able to make when moving the hand or just the fingers. Recent approaches try to fit a 3D hand model to the observed RGB-D data by an optimization function that minimizes the error between the model and the data. However, these algorithms are very dependent on the initialization point, which are impractical to run in a natural environment. To solve these kinds of problems, it is common to use an offline data set with prelearned gestures that will serve as a first rough estimate. In concrete, we present an algorithm that uses an articulated ICP minimization function that is initialized by the parameters obtained from a data set of hand gestures trained through a deep learning framework. This setup has two strong points. First, deep learning provides a very fast and accurate estimate of performed hand gestures. Second, the articulated ICP algorithm allows capturing the possible variability of a gesture performed by different persons or slightly different gestures. Our proposed algorithm is evaluated and validated in several ways. Independent evaluations for the deep learning framework and articulated ICP are performed. Moreover, different real sequences are recorded to validate our approach and, finally, quantitative and qualitative comparisons are conducted with state-of-the-art algorithms.

Dynamic Hand Gesture Tracking Method Based on Key Frame and Video Perceptual Hashing

Aiming at the common problems such as low tracking accuracy, robustness and poor real-time ability for the traditional tracking algorithm, a novel dynamic hand gesture tracking method based on key frame and video perceptual hashing has been proposed in this paper. Firstly, the proposed method is used to frame the video sequence and extract the key frames, in order to obtain a sequence of key frames. Secondly, the video perceptual hashing is used to extract the perceptual feature of the key frame and target template, thus the hash sequence is generated. Finally, the hamming distance is used to compare the similarities between the key frame and target template, therefore, the real-time tracking is carried out by using an online adaptive template update method to ensure the continuity of the hand gesture tracking. Experimental results show that the proposed method can realize the accurate and real-time tracking of the dynamic deformation gestures, and effectively reduce the time complexity of the algorithm, as well as ensure the stability and continuity of the moving target tracking.

Methods on Real Time Gesture Recognition System

Gesture recognition deals with discussion of various methods, techniques and concerned algorithms related to it. Gesture recognition uses a simple &amp; basic sign languages like movement of hand, position of lips &amp; eye ball as well as eye lids positions. The various methods for image capturing, gesture recognition, gesture tracking, gesture segmentation and smoothing methods compared, and by the overweighing advantage of different gesture recognitions and their applications. In recent days gesture recognition is widely utilized in gaming industries, biomedical applications, and medical diagnostics for dumb and deaf people. Due to their wide applications, high efficiency, high accuracy and low expenditure gestures are using in many applications including robotics. By using gestures to develop human computer interaction (HCI) method it is necessary to identify the proper and meaning full gesture from different gesture images. The Gesture recognition avoids use of costly hardware devices for understanding the activities and recognition example lots of I/O devices like keyboard mouse etc. Can be Limited.

Investigating large curved interaction devices

Large interactive surfaces enable novel forms of interaction for their users, particularly in terms of collaborative interaction. During longer interactions, the ergonomic factors of interaction systems have to be taken into consideration. Using the full interaction space may require considerable motion of the arms and upper body over a prolonged period of time, potentially causing fatigue. In this work, we present Curved, a large-surface interaction device, whose shape is designed based on the natural movement of an outstretched arm. It is able to track one or two hands above or on its surface by using 32 capacitive proximity sensors. Supporting both touch and mid-air interaction can enable more versatile modes of use. We use image processing methods for tracking the user’s hands and classify gestures based on their motion. Virtual reality is a potential use case for such interaction systems and was chosen for our demonstration application. We conducted a study with ten users to test the gesture tracking performance, as well as user experience and user preference for the adjustable system parameters.

基于Vibe和时空上下文的运动手势跟踪算法

基于Vibe和时空上下文的运动手势跟踪算法

Adaptive Kalman Filter Applied to Vision Based Head Gesture Tracking for Playing Video Games

This paper proposes an adaptive Kalman filter (AKF) to improve the performance of a vision-based human machine interface (HMI) applied to a video game. The HMI identifies head gestures and decodes them into corresponding commands. Face detection and feature tracking algorithms are used to detect optical flow produced by head gestures. Such approaches often fail due to changes in head posture, occlusion and varying illumination. The adaptive Kalman filter is applied to estimate motion information and reduce the effect of missing frames in a real-time application. Failure in head gesture tracking eventually leads to malfunctioning game control, reducing the scores achieved, so the performance of the proposed vision-based HMI is examined using a game scoring mechanism. The experimental results show that the proposed interface has a good response time, and the adaptive Kalman filter improves the game scores by ten percent.

Dynamic Gesture Recognition based on LeapMotion and HMM-CART Model

This paper focuses on improving the recognition accuracy of dynamic gesture learning, and proposes dynamic gesture track recognition strategy based on HMM-CART model structure. It integrates the modeling ability of the HMM (Hidden-Markov-Model) for time series data and the interpretability of CART (Classification-And-Regression-Tree) for its ability of fast classification and regression, and use it for dynamic gesture recognition. Time series data of movement gestures collected by LeapMotion sensor will be first divided into four channels: finger shape, palm normal vector, palm ball radius and palm displacement vector, and then HMM in HMM Layer will be built for each channel, finally the likelihood probability of model calculated for each sub-sequence of observation should be classified as the input of the CART model in CART Layer to identify the gesture. Experiments show that the accuracy of dynamic gesture recognition method using HMM-CART model is higher than that of traditional single channel HMM.

3D finger tracking and recognition image processing for real-time music playing with depth sensors

In this research, we propose a state-of-the-art 3D finger gesture tracking and recognition method. We use the depth sensors for both hands in real time music playing. In line with the development of 3D depth cameras, we implemented a set of 3D gesture-based instruments, such as Virtual Cello and Virtual Piano, which need precise finger tracking in 3D space. For hands tracking, model-based tracking for left hand and appearance-based tracking for right hand techniques are proposed. To detect finger gestures, our approaches consist number of systematic steps as reducing noise in depth map and geometrical processing for Virtual Cello. For Virtual Piano, we introduce the Neural Network (NN) method to detect special hand gestures. It has Multilayer Perceptron (MLP) structure with back propagation training. Literature has few examples using touch screen as medium, with fixed-coordinates, and 2D–gestures to control MIDI input. The end users should no longer carry anything on their hands. We use Senz3D and Leap Motion due to a few technical benefits. Senz3D and Leap Motion use a closer distance to hands, thus detailed finger gestures can be precisely identified. In the past years, we announced a set of virtual musical instruments and the MINE Virtual Band. Our research work is tested on lab environment and professional theatrical stage. More information and demonstrations of the proposed method can be accessed at: http://video.minelab.tw/DETS/VMIB/.

Track-based Gesture Recognition Method Based on Kinect

This paper presents a new gesture track recognition method based on the depth image information received from the Kinect sensor. First, a Kinect sensor is used to obtain the coordinates of a moving arm. Then, the gesture tracks corresponding to these coordinates are analyzed. Matching and recognition of gesture tracks are implemented by performing golden section search. The results show that this track-based method is highly effective in gesture recognition.

Gesture recognition for human-robot collaboration: A review

Recently, the concept of human-robot collaboration has raised many research interests. Instead of robots replacing human workers in workplaces, human-robot collaboration allows human workers and robots working together in a shared manufacturing environment. Human-robot collaboration can release human workers from heavy tasks with assistive robots if effective communication channels between humans and robots are established. Although the communication channels between human workers and robots are still limited, gesture recognition has been effectively applied as the interface between humans and computers for long time. Covering some of the most important technologies and algorithms of gesture recognition, this paper is intended to provide an overview of the gesture recognition research and explore the possibility to apply gesture recognition in human-robot collaborative manufacturing. In this paper, an overall model of gesture recognition for human-robot collaboration is also proposed. There are four essential technical components in the model of gesture recognition for human-robot collaboration: sensor technologies, gesture identification, gesture tracking and gesture classification. Reviewed approaches are classified according to the four essential technical components. Statistical analysis is also presented after technical analysis. Towards the end of this paper, future research trends are outlined.

Kinect Gesture Recognition Method Using Track Sequence

In order to recognize the trajectory of hand gesture based on image sequence, a method of gesture track recognition based on location relation was presented. First, use the Kinect sensor to extract track sequence, then math samples of hand's coordinate sequences to recognize track shape through the gold segmentation. Experimental results show that this method can effectively for identifying the gesture trajectory.

Tracking and Classification of In-Air Hand Gesture Based on Thermal Guided Joint Filter.

The research on hand gestures has attracted many image processing-related studies, as it intuitively conveys the intention of a human as it pertains to motional meaning. Various sensors have been used to exploit the advantages of different modalities for the extraction of important information conveyed by the hand gesture of a user. Although many works have focused on learning the benefits of thermal information from thermal cameras, most have focused on face recognition or human body detection, rather than hand gesture recognition. Additionally, the majority of the works that take advantage of multiple modalities (e.g., the combination of a thermal sensor and a visual sensor), usually adopting simple fusion approaches between the two modalities. As both thermal sensors and visual sensors have their own shortcomings and strengths, we propose a novel joint filter-based hand gesture recognition method to simultaneously exploit the strengths and compensate the shortcomings of each. Our study is motivated by the investigation of the mutual supplementation between thermal and visual information in low feature level for the consistent representation of a hand in the presence of varying lighting conditions. Accordingly, our proposed method leverages the thermal sensor’s stability against luminance and the visual sensors textural detail, while complementing the low resolution and halo effect of thermal sensors and the weakness against illumination of visual sensors. A conventional region tracking method and a deep convolutional neural network have been leveraged to track the trajectory of a hand gesture and to recognize the hand gesture, respectively. Our experimental results show stability in recognizing a hand gesture against varying lighting conditions based on the contribution of the joint kernels of spatial adjacency and thermal range similarity.

基于OpenCV动态手势检测识别方法的研究

针对动态手势识别实时性不足和识别效率低的问题，本文提出一种基于OpenCV的动态手势检测和数字识别方法。该方法采用肤色特征提取手势轮廓，以预测和重定位的形式采用HMM来对手势进行跟踪，手势的识别采用先对指尖轮廓进行定位的方式。经过实验，本文方法能够稳定地跟踪手掌区域，识别准确度高，并能够在保证识别率的基础上实现对手指数目的准确识别跟踪。 In view of the problem of lack of real-time recognition and low efficiency of dynamic gesture recognition, a dynamic gesture tracking and digital recognition algorithm based on OpenCV is proposed. The method uses the skin color feature to extract the gesture profile, and HMM is used to track gestures in the form of predictions and relocations. The identification of gestures is done by first positioning the fingertip profile. After the experiment, the framework can track the palm area stably, and the recognition accuracy is high, and there is no error and can achieve the accuracy of the target tracking based on the recognition rate.

Centroid tracking based dynamic hand gesture recognition using discrete Hidden Markov Models

In many dynamic hand gesture recognition contexts, time information is not adequately used. The extracted features of dynamic gestures usually do not carry explicit information about time in gesture classification. This results in under-utilized data for more important accurate classification. Another disadvantage is that the gesture classification is then confined to only simple gestures. We have overcome these limitations by introducing centroid tracking of hand gestures that captures and retains the time sequence information for feature extraction. This simplifies the classification of dynamic gestures as movement in time helps efficient classification without burdensome processing.

Soli

This paper presents Soli , a new, robust, high-resolution, low-power, miniature gesture sensing technology for human-computer interaction based on millimeter-wave radar. We describe a new approach to developing a radar-based sensor optimized for human-computer interaction, building the sensor architecture from the ground up with the inclusion of radar design principles, high temporal resolution gesture tracking, a hardware abstraction layer (HAL), a solid-state radar chip and system architecture, interaction models and gesture vocabularies, and gesture recognition. We demonstrate that Soli can be used for robust gesture recognition and can track gestures with sub-millimeter accuracy, running at over 10,000 frames per second on embedded hardware.