Hand Motion Tracking Research Articles

Analysis of attribute domain for geometrical gesture performed by arm movements

<span>Hand gesture recognition commonly uses a camera to track hand movements and transformed into gesture database by using various computational approaches. Motion tracking utilized to map coordinate point of the subject movement, either in skeletal model or marker tracing. Data from motion trackers usually contains massive coordinate sequences of marker movement. A reliable method is required to select best features and analyze these data. However, the current issue whether the selected features and data presentation are significant for the research or not. This research brings the concept of ontology design for arm gesture recognition systems by utilizing the motion capture system. Ontology is the conceptual structure mainly used to retrieve information by establishing relation in complex data model. The proposed ontology framework is divided into three domains which are knowledge domain, attribute domain and process domain. Knowledge domain holds pre-processed gestural data from motion capture. The attribute domain is that the level where all the attribute elements were presented. This paper shows the analysis of the datasets in attribute domain. The analysis is divided into two parts which is precision measure and ANOVA test. Both analyses are to prove the reliability of datasets in attribute domain. The precision measure is used to remove all the common data for all gesture. A statistical analysis of p-value is lower than 0.01 which means the gestural data are statistically significant to be used for the similarity measure.</span>

SAS4P: Providing automatic smoking detection for a persuasive smoking cessation application

Smoking is the biggest avoidable health risk, causing millions of deaths per year worldwide. Persuasive applications are those designed to change a person’s behavior, usually in a specific way. Several mobile phone applications and messaging systems have been used to promote smoking cessation. However, most interventions use participants’ self-reports to track cigarette consumption and avoidance, which may not be accurate or objective. Previous proposals have used sensors to track hand movements and other contextual data to detect smoking or have used devices to detect smoke or breath carbon monoxide. This article proposes a low-cost wearable device that may be worn in a front shirt pocket or clipped to clothing to detect smoke and secondhand smoke. Furthermore, the device is integrated into a persuasive application to promote smoking cessation. The device was evaluated through an experiment to detect whether it may detect direct, passive, and no smoking conditions. The results are promising and may help improve tracking of cigarettes in persuasive applications.

PutEMG-A Surface Electromyography Hand Gesture Recognition Dataset.

In this paper, we present a putEMG dataset intended for the evaluation of hand gesture recognition methods based on sEMG signal. The dataset was acquired for 44 able-bodied subjects and include 8 gestures (3 full hand gestures, 4 pinches and idle). It consists of uninterrupted recordings of 24 sEMG channels from the subject’s forearm, RGB video stream and depth camera images used for hand motion tracking. Moreover, exemplary processing scripts are also published. The putEMG dataset is available under a Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0). The dataset was validated regarding sEMG amplitudes and gesture recognition performance. The classification was performed using state-of-the-art classifiers and feature sets. An accuracy of 90% was achieved for SVM classifier utilising RMS feature and for LDA classifier using Hudgin’s and Du’s feature sets. Analysis of performance for particular gestures showed that LDA/Du combination has significantly higher accuracy for full hand gestures, while SVM/RMS performs better for pinch gestures. The presented dataset can be used as a benchmark for various classification methods, the evaluation of electrode localisation concepts, or the development of classification methods invariant to user-specific features or electrode displacement.

PENGEMBANGAN GAME 3D “EVERPLANE” DENGAN LEAP MOTION

Motion Leap (Hand Motion tracking) is a term for recording hand movements used as digital models and is an additional device that can be connected to a computer and can then be used to replace both mouse and keyboard functions. In the 3D game "Everplane", Leap Motion is used as a game controller and is the main component of the game. The Everplane game is an endless game / endless runner type game. Endless runner is a game where the player's character continues to move forward through the endless world of games. The Everplane game has the concept of exploring space. The process of making 3D games "Everplane" through various data research is needed to meet the needs of game production through three stages, namely Preproduction (Game Design, Character Design and Layout Design), Production (Modeling, Texturing, Design graphic user interface, Music, Leap Setup Motion, Assembly Animating, Programming, and Problem Solfing) and Postproduction (Deploying, Mastering and Merchandise).Keywords: Leap Motion, Everplane game, manufacturing process

Video-Based Driver’s Hand Tracking using Fast Normalized Cross Coefficient with Improved Computational Efficiency

Driver distraction is one of the major causes for fatal car accidents. In a distracting activity, manual distraction is a triggered response of other types of distraction, such as cognitive and visual distraction. Therefore, recognition of manual distraction can contribute to the monitoring of overall drivers’ distraction. In this study, a computer vision-based method to track hand movement from the recorded driving behavior is proposed. This method integrates a low computational cost template matching algorithm using fast normalized cross coefficient (NCC) and a novel searching strategy. The proposed method was evaluated by the VIVA hand tracking data set. It achieves 50.83% of marginal accuracy percentage (mAP), 42.18% of multiple object tracking accuracy (MOTA), 31.56% of mostly tracked (MT), and 19.29% of mostly lost (ML), and it outperformed a state of the art algorithm in MOTA and MT. Additionally, the computational cost of the proposed method is greatly improved, and it can run at around 11.1 frames per second. The outcome of this research will further assist driving distraction recognition and mitigation, and improve driving safety.

A Tangible Solution for Hand Motion Tracking in Clinical Applications.

Objective real-time assessment of hand motion is crucial in many clinical applications including technically-assisted physical rehabilitation of the upper extremity. We propose an inertial-sensor-based hand motion tracking system and a set of dual-quaternion-based methods for estimation of finger segment orientations and fingertip positions. The proposed system addresses the specific requirements of clinical applications in two ways: (1) In contrast to glove-based approaches, the proposed solution maintains the sense of touch. (2) In contrast to previous work, the proposed methods avoid the use of complex calibration procedures, which means that they are suitable for patients with severe motor impairment of the hand. To overcome the limited significance of validation in lab environments with homogeneous magnetic fields, we validate the proposed system using functional hand motions in the presence of severe magnetic disturbances as they appear in realistic clinical settings. We show that standard sensor fusion methods that rely on magnetometer readings may perform well in perfect laboratory environments but can lead to more than 15 cm root-mean-square error for the fingertip distances in realistic environments, while our advanced method yields root-mean-square errors below 2 cm for all performed motions.

Feature Evaluation of Upper Limb Exercise Rehabilitation Interactive System Based on Kinect

The virtual rehabilitation system combining virtual rehabilitation environment and upper limb rehabilitation technology is interactive and interesting, which can improve the enthusiasm and initiative of patients for rehabilitation training, improve the efficiency of rehabilitation training and improve the effect of rehabilitation treatment. This paper firstly conducts in-depth research and analysis on the research progress of the upper limb rehabilitation robot system, and deeply studies the principle and rehabilitation principle of the stroke caused by hemiplegic dyskinesia, and summarizes the goals and methods of the upper limb rehabilitation system design. Secondly, the hand motion tracking is realized by Kinect’s bone tracking, and the optimal tracking distance is determined experimentally, which verifies the stability and robustness of the tracking. Static gesture recognition adopts two gesture recognition schemes based on Kinect depth image and color space model respectively. Finally, using the rehabilitation robot and Kinect sensor as the hardware platform, the virtual rehabilitation training system experimental platform is constructed, and the horizontal rehabilitation exercise and the three-dimensional space rehabilitation exercise are respectively studied experimentally, and the exercise data obtained by using healthy subjects as the experimental object is analyzed. Based on this, the validity and feasibility of the Kinect-based upper limb exercise rehabilitation interactive system were verified.

The Co-design of Hand Rehabilitation Exercises for Multiple Sclerosis Using Hand Tracking System.

Multiple sclerosis (MS) often affects motor function, leading to an adverse effect on daily living. Rehabilitation is important in terms of improving mobility and activities of daily living. Virtual environments (VE) are increasing in popularity within this research area, but research in terms of VE is still rare, for both the upper and lower limb, in people with MS. Leap Motion (LM), a hand motion tracking system, has demonstrated success in stroke research but has yet to be investigated within MS. Following a co-design approach, five participants with MS discussed in a focus group (FG) their hand mobility issues, their thoughts about this technology-based rehabilitation and motivational factors. Findings were incorporated into the design of a series of gamified upper limb rehabilitation exercises, using LM, on Unity Game Engine. Three participants returned and engaged in user testing session and a FG in order to evaluate and discuss their experience. Overall participants found the proposed technology-based exercises to be engaging, immersive and a desirable approach to rehabilitation. Participant feedback underlined the usefulness of co-creation, especially in accommodating the range of motivators and user preferences. However, the study highlighted the loss of tracking of hand movements with LM as one of the limitations. Participants stated they would be likely to use this approach at home if there was a definite rehabilitation benefit and related more to visualising which muscle groups or actions they were aiming to improve.

Interactive Mobile Augmented Reality System for Image and Hand Motion Tracking

In recent years, augmented reality (AR) is considered a promising technology that combines virtual information such as videos, images, and three-dimensional objects with a real camera view in mobile platforms. Interactive AR further provides human–computer interaction to allow the user to interact with virtual objects on the mobile display. In this paper, we proposed a cloud-based mobile augmented reality interactive system (MARIS), which includes MARIS-I for image target tracking and MARIS-H for hand motion tracking. MARIS-I estimates the position of the image target by adopting a feature-based mean-shift algorithm, which is feasible for real-time applications with its small region feature detection. MARIS-H provides two tracking modes for fingertip and back of hand tracking to enhance user experiences (UX) for interaction. Either the center position for the back of hand or fingertip is first estimated by particle filtering technique, which calculates the weighting of each particle according to hand or fingertip model. Afterward, the contour of the fingertip is estimated by level-set-based contour evolution in the fingertip tracking mode. Furthermore, we implement a device/cloud architecture for the proposed MARIS to decrease memory requirement and computational complexity on the device side. Experimental results show that MARIS including MARIS-I and MARIS-H can outperform other existing methods for image and hand motion tracking, respectively. The proposed MARIS is demonstrated in a picture book to provide fruitful interactive UX for digital learning systems.

Multi-Finger Haptics Enhanced Virtual Instrument Interfaces

Measurement interfaces are progressing, and the influence of new technologies in virtual reality and telerobotics made it more advancing. This paper presents a haptics enhanced virtual measurement interface model and demonstrates its usefulness for touchless and virtual instrumentation. Users can interact, locally or remotely, with the instrumentation by using motion tracking and haptics technologies. Attention has been paid to do the integration of multi-finger haptics and hand motion tracking systems for the effective interaction with the virtual instrument interfaces.

A gesture-based design tool: Assessing 2DOF vs. 4DOF steerable instrument control.

Iterative prototyping is costly and time-consuming. Particularly when designing medical instruments, human factors related design choices significantly impact performance and safety. A tool is presented that allows for the evaluation of steerable instrument controls before the onset of the prototyping stage. The design tool couples gestural input to virtually simulated instrument motions using hand motion tracking. We performed a human-subject evaluation of two manual control strategies that differed in their degrees of freedom (DOF). 2DOF thumb control was compared to 4DOF thumb-index finger control. Results identified regions within the instrument workspace that are difficult to reach and showed participants to favor using the thumb for gross and fine-tuning motions at both control strategies. Index finger ab/adduction was found to be least functional. A strong learning effect was observed at 4DOF control. Based on the results, gesture-based instrument design is a viable design tool.

Tracking hand movements captures the response dynamics of the evaluative priming effect

ABSTRACTWe tested the response dynamics of the evaluative priming effect (i.e. facilitation of target responses following evaluatively congruent compared with evaluatively incongruent primes) using a mouse tracking procedure that records hand movements during the execution of categorisation tasks. In Experiment 1, when participants performed the evaluative categorisation task but not the non-evaluative semantic categorisation task, their mouse trajectories for evaluatively incongruent trials curved more toward the opposite response than those for evaluatively congruent trials, indicating the emergence of evaluative priming effects based on response competition. In Experiment 2, implementing a task-switching procedure in which evaluative and non-evaluative categorisation tasks were intermixed, we obtained reliable evaluative priming effects in the non-evaluative semantic categorisation task as well as in the evaluative categorisation task when participants assigned attention to the evaluative stimulus dimension. Analyses of hand movements revealed that the evaluative priming effects in the evaluative categorisation task were reflected in the mouse trajectories, while evaluative priming effects in the non-evaluative categorisation tasks were reflected in initiation times (i.e. the time elapsed between target onset and first mouse movement). Based on these findings, we discuss the methodological benefits of the mouse tracking procedure and the underlying processes of evaluative priming effects.

Hand Gesture Interface for Smart Operation Theatre Lighting

Operation theatre lighting system is a critical component in the operation theatre. Focusing the light to the area of surgery and varying the light intensity is manually done in currently available systems. Disruption of surgeon’s attention and possibility of infection due to non sterilized hands is a major setback of the manual controlling method. This usually requires assistants to adjust the position and intensity of the light system by commands from the surgeon. So, designing an intelligent lighting system for operating theatre which can maintain the intensity values to the required level without shadow effect to the operation area is an important requirement. Proposed system makes use of accelerators in the gloves worn by surgeon so as to track hand movements. Pre defined hand movements can be configured to move the lighting system up, down, left or right. Accelerator input is interpreted for its corresponding movement by controller at transmitter side and it is wirelessly given to the receiver side using an nRF module. At the receiver side the incoming command from the nRF is taken and suitable response generated at the motor driver by the controller to move the lighting device. In the same way predefined hand signal can be used to vary the intensity of the light by using microcontroller to drive an LED driver.

Smartwatch-based Early Gesture Detection 8 Trajectory Tracking for Interactive Gesture-Driven Applications

The paper explores the possibility of using wrist-worn devices (specifically, a smartwatch) to accurately track the hand movement and gestures for a new class of immersive, interactive gesture-driven applications. These interactive applications need two special features: (a) the ability to identify gestures from a continuous stream of sensor data early--i.e., even before the gesture is complete, and (b) the ability to precisely track the hand's trajectory, even though the underlying inertial sensor data is noisy. We develop a new approach that tackles these requirements by first building a HMM-based gesture recognition framework that does not need an explicit segmentation step, and then using a per-gesture trajectory tracking solution that tracks the hand movement only during these predefined gestures. Using an elaborate setup that allows us to realistically study the table-tennis related hand movements of users, we show that our approach works: (a) it can achieve 95% stroke recognition accuracy. Within 50% of gesture, it can achieve a recall value of 92% for 10 novice users and 93% for 15 experienced users from a continuous sensor stream; (b) it can track hand movement during such strokeplay with a median accuracy of 6.2 cm.

Design and Evaluation of a 3-D Printed Optical Sensor for Monitoring Finger Flexion

The development of techniques for monitoring finger movement is becoming increasingly important in areas, such as robotics, virtual reality, and rehabilitation. To date, various techniques have been proposed for tracking hand movements, but the majority suffer from poor accuracy and repeatability. Inspired by the articulated structure of finger joints, we propose a novel 3-D printed optical sensor with a compact hinged configuration for tracking finger flexion. This sensor exploits Malus’ law using the attenuation of light transmitted through crossed polarizers. The sensor consists of a single LED, two pieces of linear polarizing film, and a photodetector that detects the changes in polarized light intensity proportional to the angle of finger flexion. This paper presents the characterization of the proposed optical sensor and compares it with a commonly used commercial bend sensor. Results show that the bend sensor exhibits hysteresis error, low sensitivity at small angles, and significant temporal drift. In contrast, the optical sensor is more accurate (±0.5°) in the measuring range from 0° to 90°, and exhibits high repeatability and stability, as well as a fast dynamic response. Overall, the optical sensor outperforms the commercial bend sensor, and shows excellent potential for monitoring hand movements in real time.

Human stochastic closed-loop behavior for master-slave teleoperation using multi-leap-motion sensor

Teleoperation has a wide range of applications that have been under development over the past two decades. Previous researches have focused on the control design of teleoperation machine systems to deal with obstacles such as time-delayed stability and transparency. Recent researches have shown that the inclusion of human closed-loop dynamics in control design can improve the performance of robot telemanipulation. The complexity of human behavior arises from the uncertainty of both human physiology and psychology; hence, the investigation can benefit from empirical studies. This study develops a type of statistical learning method to model and evaluate human stochastic closed-loop behavior, which is considered as a hand motion during the direct incremental control process of master-slave teleoperation. The hand trajectory is empirically considered as having a binary linear regression relationship with the error and error rate between the demanded and simulated teleoperator trajectories, while random movements with zero error and error rate are discovered. Hand movement tracking is achieved using a multi-leap-motion sensor (MLM), which is a markerless and natural infrared vision-based manner for motion capture. The established behavior model and statistical learning results reveal certain human properties of operational activities including visual perception, decision making, and robot telemanipulation. The properties indicate some probable system enhancements for future work.

Teleoperation of the SCHUNK S5FH under-actuated anthropomorphic hand using human hand motion tracking

This paper describes the development of a remote handling control of an anthropomorphic robotic hand, the SCHUNK S5FH, using the human hand as master by measuring its motion with OptiTrack Technology. The goal of this work is to enhance manipulation studies on the human hand and to instantly transfer those studies on robotic hands. A preliminary study on methods and devices used for fingers tracking led to the choice of a simplified kinematic model of the human hand on the basis of the available motion tracking system. Using the same criteria, the analysis of protocols for markers allocation led to define the number and a method for their arrangement on the fingers and palm. In order to overcome the limitation of the Motion Capture System, a method for identification and labeling has been developed according to their anatomical arrangement. Afterwards, the tracking is performed using the constraints between marker positions on the kinematic chain of the hand and a dynamic labeling algorithm robust with respect to noise, outliers and loss of markers. The validation is performed using the right hand of different subjects and considering different tasks involving flexion/extension and abduction/adduction of fingers and thumb opposition. For testing and validation, preliminary studies on synergies for manipulation tasks such as screwing a cup, has been conducted on the human hand and transferred on the robotic hand.

A Sign Language to Text Converter Using Leap Motion

This paper presents a prototype that can convert sign language into text. A Leap Motion controller was utilised as an interface for hand motion tracking without the need of wearing any external instruments. Three recognition techniques were employed to measure the performance of the prototype, namely the Geometric Template Matching, Artificial Neural Network and Cross Correlation. 26 alphabets from American Sign Language were chosen for training and testing the proposed prototype. The experimental results showed that Geometric Template Matching achieved the highest recognition accuracy compared to the other recognition techniques.

Hand tracking with physiological constraints

Articulated hand tracking systems have been commonly used in virtual reality applications, including systems with human–computer interaction or interaction with game consoles; hand pose estimation has various other applications comprising sign language recognition and animation synthesis. The advanced technological achievements in motion capture over the last decade allow data acquisition with high accuracy and low cost. However, due to the high complexity of the human hand, it is still challenging to animate a hand model able to deal in details with the anatomical and physiological constraints of the hand. In this paper, we present a simple and efficient methodology for tracking and reconstructing 3D hand poses. Using an optical motion capture system, where markers are positioned at strategic points, we manage to acquire the movement of the hand and establish its orientation using a minimum number of markers. An Inverse Kinematics solver was then employed to control the postures of the hand, subject to physiological constraints that restrict the allowed movements to a feasible and natural set. The proposed methodology produces smooth and biomechanically correct movements, while the required processing time remains low, enabling an effective real-time hand motion tracking and reconstruction system.

Robust Hand Motion Tracking through Data Fusion of 5DT Data Glove and Nimble VR Kinect Camera Measurements.

Vision based interfaces for human computer interaction have gained increasing attention over the past decade. This study presents a data fusion approach of the Nimble VR vision based system, using the Kinect camera, with the contact based 5DT Data Glove. Data fusion was achieved through a Kalman filter. The Nimble VR and filter output were compared using measurements performed on (1) a wooden hand model placed in various static postures and orientations; and (2) three differently sized human hands during active finger flexions. Precision and accuracy of joint angle estimates as a function of hand posture and orientation were determined. Moreover, in light of possible self-occlusions of the fingers in the Kinect camera images, data completeness was assessed. Results showed that the integration of the Data Glove through the Kalman filter provided for the proximal interphalangeal (PIP) joints of the fingers a substantial improvement of 79% in precision, from 2.2 deg to 0.9 deg. Moreover, a moderate improvement of 31% in accuracy (being the mean angular deviation from the true joint angle) was established, from 24 deg to 17 deg. The metacarpophalangeal (MCP) joint was relatively unaffected by the Kalman filter. Moreover, the Data Glove increased data completeness, thus providing a substantial advantage over the sole use of the Nimble VR system.