Head Gestures Research Articles

Comparative Analysis of a Vision Based HGR System Used for Handicapped People

The object tracking is critical to visual / video surveillance, analysis of the activity and gesture recognition. The major difficulties to be occurred in the visual tracking are different environmental conditions, illumination changes, occlusion and appearance. In this paper, the comparative analysis of the different systems which are used to recognize the head gestures under different environmental conditions is discussed. The existing algorithm used to recognize the head gestures has some limitations. The existing algorithm cannot work under outdoor environmental conditions. The traditional camshift algorithm and unscented kalman filter are integrated and used to recognize the head gestures under outdoor environmental conditions. The unscented kalman filter is a tracking algorithm used to remove the limitations of the traditional camshift algorithm. The simulation result shows the better performance of the improved algorithm than the traditional camshift algorithm.

The Comparative Analysis of a Vision Based HGR System Used for Handicapped People

The object tracking is critical to visual / video surveillance, analysis of the activity and gesture recognition. The major difficulties to be occurred in the visual tracking are different environmental conditions, illumination changes, occlusion and appearance. In this paper, the comparative analysis of the different systems which are used to recognize the head gestures under different environmental conditions is discussed. The existing algorithm used to recognize the head gestures has some limitations. The existing algorithm cannot work under outdoor environmental conditions. The traditional camshift algorithm and unscented kalman filter are integrated and used to recognize the head gestures under outdoor environmental conditions. The unscented kalman filter is a tracking algorithm used to remove the limitations of the traditional camshift algorithm. The simulation result shows the better performance of the improved algorithm than the traditional camshift algorithm.

GUI Control with Head Gesture

Face detection based mouse movement is one of the important works for the physically challenged people. The movement of head can notify the computer specific task. By understanding, the disabled people computer can pass message to other person and that can help disabled to notify others the urgent need of them. In this project, an application has been built which will help the people who do not have any control of their hands but they can only move their head. In this work, OpenCV based Haar-like feature extraction has been used to detect face. By using face detection algorithm, faces as well as non-faces are classified as faces. So, to reduce the false positive results, face detection algorithm has been enhanced. After face detection, mouse movement has been controlled with the coordinate of face movement.

Headspeak: Morse code Based Head Gesture To Speech Conversion Using Intel Realsense™ Technology

Advancement in technology has ensured better and more intuitive Human Computer Interaction (HCI). However, technological leap has not been able to bridge the gap for individuals with disability, particularly people with different impairments. Work has been done towards translating various sign languages and lip reading, but not much work is identified towards head gesture recognition and head gesture to speech conversion. Paralyzed patients, with ability to only move their head and eye find it rather difficult to communicate and interact with machines. This work presents a novel and efficient technique to map restricted head movements to text format using face and emotion detection capabilities of Intel RealSense SDK using Morse Code Mapping. The work maps UP and DOWN position of the head to DASH and DOT symbols of Morse code for an EYE_BLINK event trigger and SMILE emotion to convert the sequence of Morse symbols to an English character. The new language called HeadSpeak effectively reduces the pain for generating head gestures through 3D face position based gesture detection.The system can produce an average of 19 characters par minute, gesture error rate of .03% and average 4.5 words per minute.

Gesture in contexts of scopal ambiguity: Negation and quantification in English

AbstractGestures can play a facilitative role in the interpretation of structural ambiguities (Guellaiï, Langus, & Nespor, 2014; Prieto, Borràs-Comes, Tubau, & Espinal, 2013; Tubau, González-Fuente, Prieto, & Espinal, 2015) and are associated with spoken expression of negation (Calbris, 2011; Harrison, 2014a; Kendon, 2002, 2004). This study examines gestural forms and timing patterns with specific interpretations intended by speakers in a context of negation in English where the presence of quantification (all/most/many) yields scope ambiguities, for example,All the students didn’t go= (1). Some number of the students went, but all is not the correct number (negation takes wide scope over the quantifier;not>all), versus (2) some number of the students didn’t go, and all is that number (negation takes narrow scope over the quantifier,all>not; see Horn, 2001, Jackendoff, 1972; Syrett, Simon, & Nisula, 2014b). Twenty-five native English speakers produced scopally ambiguous sentences. Analyses of 317 co-occurring gestures revealed a preponderance of head gestures and use of semantically congruent head shakes, alignment of gestures with the negator, and lengthening of gesture strokes where interpretations involved narrow-scope negation. Results are discussed with reference to scope of negation and gesture (Harrison, 2010, 2013, 2014a, 2014b) particularly in comparison to variable patterns found for prosody (Syrett, Simon, & Nisula, 2014a).

Robot Learning of Assistive Manipulation Tasks by Demonstration via Head Gesture-based Interface.

Assistive robotic manipulators have the potential to support the lives of people suffering from severe motor impairments. They can support individuals with disabilities to independently perform daily living activities, such as drinking, eating, manipulation tasks, and opening doors. An attractive solution is to enable motor impaired users to teach a robot by providing demonstrations of daily living tasks. The user controls the robot 'manually' with an intuitive human-robot interface to provide demonstration, which is followed by the robot learning of the performed task. However, the control of robotic manipulators by motor impaired individuals is a challenging topic. In this paper, a novel head gesture-based interface for hands-free robot control and a framework for robot learning from demonstration are presented. The head gesture-based interface consists of a camera mounted on the user's hat, which records the changes in the viewed scene due to the head motion. The head gesture recognition is performed using the optical flow for feature extraction and support vector machine for gesture classification. The recognized head gestures are further mapped into robot control commands to perform object manipulation task. The robot learns the demonstrated task by generating the sequence of actions and Gaussian Mixture Model method is used to segment the demonstrated path of the robot's end-effector. During the robotic reproduction of the task, the modified Gaussian Mixture Model and Gaussian Mixture Regression are used to adapt to environmental changes. The proposed framework was evaluated in a real-world assistive robotic scenario in a small study involving 13 participants; 12 able-bodied and one tetraplegic. The presented results demonstrate a potential of the proposed framework to enable severe motor impaired individuals to demonstrate daily living tasks to robotic manipulators.

How to Build a Patient-Specific Hybrid Simulator for Orthopaedic Open Surgery: Benefits and Limits of Mixed-Reality Using the Microsoft HoloLens.

Orthopaedic simulators are popular in innovative surgical training programs, where trainees gain procedural experience in a safe and controlled environment. Recent studies suggest that an ideal simulator should combine haptic, visual, and audio technology to create an immersive training environment. This article explores the potentialities of mixed-reality using the HoloLens to develop a hybrid training system for orthopaedic open surgery. Hip arthroplasty, one of the most common orthopaedic procedures, was chosen as a benchmark to evaluate the proposed system. Patient-specific anatomical 3D models were extracted from a patient computed tomography to implement the virtual content and to fabricate the physical components of the simulator. Rapid prototyping was used to create synthetic bones. The Vuforia SDK was utilized to register virtual and physical contents. The Unity3D game engine was employed to develop the software allowing interactions with the virtual content using head movements, gestures, and voice commands. Quantitative tests were performed to estimate the accuracy of the system by evaluating the perceived position of augmented reality targets. Mean and maximum errors matched the requirements of the target application. Qualitative tests were carried out to evaluate workload and usability of the HoloLens for our orthopaedic simulator, considering visual and audio perception and interaction and ergonomics issues. The perceived overall workload was low, and the self-assessed performance was considered satisfactory. Visual and audio perception and gesture and voice interactions obtained a positive feedback. Postural discomfort and visual fatigue obtained a nonnegative evaluation for a simulation session of 40 minutes. These results encourage using mixed-reality to implement a hybrid simulator for orthopaedic open surgery. An optimal design of the simulation tasks and equipment setup is required to minimize the user discomfort. Future works will include Face Validity, Content Validity, and Construct Validity to complete the assessment of the hip arthroplasty simulator.

A Novel Eye-Tracking Device Designed with a Head Gesture Control Module

In this paper a novel eye-tracking device designed which uses tiny CCD cameras to capture the eye image from the screen with the help of a video capture card. In the head gesture control mode, a light source projector is turned on and the CCD camera detects the position of the light source. The

A study of the adaptive gesture interface for the severely physically handicapped

Technology has already been developed that does not require a person to physically touch a mouse or a screen. This technology, such as the Kinect, relies on motion sensors to pick up movements and gestures, using them as inputs. However, these gestures are pre-programmed and actually require the same amount of coordination as any other input through a physical device. This means that they are not immediately useful for those with motor dysfunction. Dr. Ikushi Yoda of the National Institute of Advanced Industrial Science and Technology (AIST), Japan, is working to resolve this problem and aims to provide motion sensors that can allow disabled people to smoothly access computers amongst other devices. Yoda and his interdisciplinary team are working to analyse and categorise a wide variety of gestures from disabled with motor dysfunction. Their aim is to programme these gestures to be recognised by motion-sensing devices as well as to introduce flexibility into the devices to allow for a wider range of detection. In order to devise new systems of input for users with various motor disabilities, Yoda and his team first had to gather data on the sorts of gestures they could create. To document these accurately, Yoda's team monitored a pool of disabled volunteers to collect movement data for classification and analysis. Yoda chose participants that represented a wide range of potential disabilities. This meant many gestures of different types and from different parts of the body could be acquired and analysed. These data were classified according to body part and used to develop a modulised gesture recognition engine. In addition to gathering important data, Yoda is also trying to optimise the research process using relatively cheap motion detection cameras. It is essential that any system eventually employed is at least accessible to the majority of the target disabled users. Yoda and his team have previously developed an interface based on head gestures for individuals with severe cerebral palsy to circumvent their inability to operate their motorised wheelchairs.

Gyro-Accelerometer based Control of an Intelligent Wheelchair

This paper presents a free-hand interface to control an electric wheelchair using thehead gesture for people with severe disabilities i.e. multiple sclerosis, quadriplegic patients and oldage people. The patient head acceleration and rotation rate are used to control the intelligentwheelchair. The patient head gesture is detected using accelerometer and gyroscope sensorsembedded on a single board MPU6050. The MEMS sensors outputs are combined using Kalmanfilter as sensor fusion to build a high accurate orientation sensor. The system uses an Arduinomega as microcontroller to perform data processing, sensor fusion and joystick emulation tocontrol the intelligent wheelchair and HC-SR04 ultrasonic sensors to provide safe navigation.Thewheelchair can be controlled using two modes. In the first mode, the wheelchair is controlled bythe usual joystick. In the second mode, the patient uses his head motion to control the wheelchair.The principal advantage of the proposed approach is that the switching between the two controlmodes is soft, straightforward and transparent to the user.

Behavioral Analysis of Kinetic Telepresence for Small Symmetric Group-to-Group Meetings

Nonverbal behavior analysis revealed the effect of MMSpace , a kinetic telepresence developed for social telepresence, on small symmetric group-to-group conversations. MMSpace consists of kinetic avatars, equipped with flat projection screen panels as faces, that can change their pose and position automatically to mirror the remote user's head motions. The advantage is the realistic kinetic expression of human head movements, which form gestures like nodding and indicate the focus of visual attention, through the use of four degree-of-freedom low-latency precision actuators. Another feature is the support of eye contact among remote participants, which is made possible by the avatar's kinetic pose changes and by adaptive camera selection for orienting the user's face toward the remote addressee. Its limitation is its room-scale infrastructure and restricted participant positions. Targeting a symmetric 2 $\times$ 2 setting, participants’ nonverbal behaviors, including gaze directions and head gestures, were compared among three conditions, MMSpace with/without physical motions and face-to-face settings. There was a significant difference between the conditions in terms of the duration of glance/mutual glances, total gaze transition time, amount of head gesturing, and co-occurrences of head gestures in the remote participants. The results indicate that the avatar's physical motion can elicit longer (mutual) glances with a shorter total transition time and more (co-)occurrences of head gestures, and it makes MMSpace -based conversations closer, in terms of these nonverbal statistics, to face-to-face ones compared with those of a static version of MMSpace without physical motion.

Robot Control by Human Head Gesture

Robot Control by Human Head Gesture

Head Motion and Head Gesture Based Control Robot

Head Motion and Head Gesture Based Control Robot

Automated Analysis and Prediction of Job Interview Performance

We present a computational framework for automatically quantifying verbal and nonverbal behaviors in the context of job interviews. The proposed framework is trained by analyzing the videos of 138 interview sessions with 69 internship-seeking undergraduates at the Massachusetts Institute of Technology (MIT). Our automated analysis includes facial expressions (e.g., smiles, head gestures, facial tracking points), language (e.g., word counts, topic modeling), and prosodic information (e.g., pitch, intonation, and pauses) of the interviewees. The ground truth labels are derived by taking a weighted average over the ratings of nine independent judges. Our framework can automatically predict the ratings for interview traits such as excitement, friendliness, and engagement with correlation coefficients of 0.70 or higher, and can quantify the relative importance of prosody, language, and facial expressions. By analyzing the relative feature weights learned by the regression models, our framework recommends to speak more fluently, use fewer filler words, speak as “we” (versus “I”), use more unique words, and smile more. We also find that the students who were rated highly while answering the first interview question were also rated highly overall (i.e., first impression matters). Finally, our MIT Interview dataset is available to other researchers to further validate and expand our findings.

SignFi

We propose SignFi to recognize sign language gestures using WiFi. SignFi uses Channel State Information (CSI) measured by WiFi packets as the input and a Convolutional Neural Network (CNN) as the classification algorithm. Existing WiFi-based sign gesture recognition technologies are tested on no more than 25 gestures that only involve hand and/or finger gestures. SignFi is able to recognize 276 sign gestures, which involve the head, arm, hand, and finger gestures, with high accuracy. SignFi collects CSI measurements to capture wireless signal characteristics of sign gestures. Raw CSI measurements are pre-processed to remove noises and recover CSI changes over sub-carriers and sampling time. Pre-processed CSI measurements are fed to a 9-layer CNN for sign gesture classification. We collect CSI traces and evaluate SignFi in the lab and home environments. There are 8,280 gesture instances, 5,520 from the lab and 2,760 from the home, for 276 sign gestures in total. For 5-fold cross validation using CSI traces of one user, the average recognition accuracy of SignFi is 98.01%, 98.91%, and 94.81% for the lab, home, and lab+home environment, respectively. We also run tests using CSI traces from 5 different users in the lab environment. The average recognition accuracy of SignFi is 86.66% for 7,500 instances of 150 sign gestures performed by 5 different users.

Prosody in the Auditory and Visual Domains: A Developmental Perspective.

The development of body movements such as hand or head gestures, or facial expressions, seems to go hand-in-hand with the development of speech abilities. We know that very young infants rely on the movements of their caregivers’ mouth to segment the speech stream, that infants’ canonical babbling is temporally related to rhythmic hand movements, that narrative abilities emerge at a similar time in speech and gestures, and that children make use of both modalities to access complex pragmatic intentions. Prosody has emerged as a key linguistic component in this speech-gesture relationship, yet its exact role in the development of multimodal communication is still not well understood. For example, it is not clear what the relative weights of speech prosody and body gestures are in language acquisition, or whether both modalities develop at the same time or whether one modality needs to be in place for the other to emerge. The present paper reviews existing literature on the interactions between speech prosody and body movements from a developmental perspective in order to shed some light on these issues.

Truth is in the head. A nod and shake compatibility effect

Studies from the embodiment perspective on language processing have shown facilitation or interference effects depending on the compatibility between verbal contents, concrete or abstract, and the motion of various parts of the body. The aim of the present study was to test whether such compatibility effects can be found when a higher cognitive process like truth evaluation is accomplished with head movements. Since nodding is a vertical head gesture typically performed with positive and affirmative responses, and shaking is a horizontal head gesture associated with negative and dissenting contents, faster response times can be expected when true information is evaluated by making a vertical head movement and false information by making a horizontal head movement.Three experiments were designed in order to test this motor compatibility effect. In the first experiment a series of very simple sentences were asked to be evaluated as true or false by dragging them vertically and horizontally with the head. It resulted that truth-value was assessed faster when it was compatible with the direction of the head movement, compared to when it was incompatible. In the second experiment participants were asked to evaluate the same sentences as the first experiment but by moving them with the mouse. In the third experiment, a non-evaluative classification task was given, where sentences concerning animals or objects were to be dragged by vertical and horizontal head movements. In the second and third experiment no compatibility effect was observed. Overall results support the hypothesis of an embodiment effect between the abstract processing of truth evaluation and the direction of the two head movements of nodding and shaking. Cultural aspects, cognitive implications, and the limits of these findings are discussed.

Head Gesture Controlled Wheelchair for Quadriplegic Patients

The health service sector has been continuously trying to improve the service given to the people in need of mobility assistance. As a result, more developers have been directed towards robotic wheelchairs. A robotic wheelchair is an intelligent wheelchair that has capabilities of navigating, detecting obstacles and moving automatically by utilizing sensors and artificial intelligence. We have developed a Robotic WheelChair for the quadriplegic patients for mobility assistance operated using the head gesture. The Robotic WheelChair includes accelerometer sensor, gyroscope sensor, ultrasonic sensor, relay, battery, DC stepper motor and raspberry pi. The MPU 6050 sensor detects the movement of the head and the signal is transmitted to the Pi. The controller processes the signal and enables the motion of wheelchair for its navigation. The ultrasonic sensors help to avoid obstacles, using the environment information gathered during navigation. The wheelchair is designed in a cost-effective way but ensures safety, flexibility, and mobility for the users.

Head Motion and Head Gesture-Based Robot Control: A Usability Study.

The assistive robot system adaptive head motion control for user-friendly support (AMiCUS) has been developed to increase the autonomy of motion impaired people. The six degrees of freedom robot arm with gripper is controlled with head motion and head gestures only, so especially tetraplegics benefit from collaboration with AMiCUS. In this paper, a usability study with a total number of 30 subjects was conducted to validate the AMiCUS interaction technology and design. 24 able-bodied subjects of demographically diverse groups and 6 tetraplegics participated in this paper. All subjects performed different pick and place tasks by controlling AMiCUS. The evaluation of the interaction design was carried out subjectively with a questionnaire as well as objectively by measurement of time, completion rate, and number of trials for correct head gesture performance. The influence of several factors like age, sex, motion impairment, and previous experience on head motion-based human-robot interaction was analyzed. The interaction design has been proven successful in laboratory environment and assessed overall positive by the subjects. The results of the presented paper confirm the usability of the assistive robot AMiCUS. AMiCUS has the potential to benefit tetraplegics by improving their independence in activities of daily living and adapted workplaces.

입력방식이 제한된 모바일 VR상에서 Inertial Sensor를 활용한 Head Gesture 기반 가상 객체 조작에 관한 연구

입력방식이 제한된 모바일 VR상에서 Inertial Sensor를 활용한 Head Gesture 기반 가상 객체 조작에 관한 연구