Robust Gaze Research Articles

A Robust Gaze Estimation Approach via Exploring Relevant Electrooculogram Features and Optimal Electrodes Placements

A Robust Gaze Estimation Approach via Exploring Relevant Electrooculogram Features and Optimal Electrodes Placements

Local eye-net: An attention based deep learning architecture for localization of eyes

Development of human machine interface has become a necessity for modern day machines to catalyze more autonomy and more efficiency. Gaze driven human intervention is an effective and convenient option for creating an interface to alleviate human errors. Facial landmark detection is very crucial for designing a robust gaze detection system. Regression based methods capacitate good spatial localization of the landmarks corresponding to different parts of the faces. But there are still scope of improvements which have been addressed by incorporating attention. In this paper, we have proposed a deep coarse-to-fine architecture called LocalEyenet for localization of only the eye regions that can be trained end-to-end. The model architecture, build on stacked hourglass backbone, learns the self-attention in feature maps which aids in preserving global as well as local spatial dependencies in face image. We have incorporated deep layer aggregation in each hourglass to minimize the loss of attention over the depth of architecture. Our model shows good generalization ability in cross-dataset evaluation and in real-time localization of eyes.

DynamicRead: Exploring Robust Gaze Interaction Methods for Reading on Handheld Mobile Devices under Dynamic Conditions

Enabling gaze interaction in real-time on handheld mobile devices has attracted significant attention in recent years. An increasing number of research projects have focused on sophisticated appearance-based deep learning models to enhance the precision of gaze estimation on smartphones. This inspires important research questions, including how the gaze can be used in a real-time application, and what type of gaze interaction methods are preferable under dynamic conditions in terms of both user acceptance and delivering reliable performance. To address these questions, we design four types of gaze scrolling techniques: three explicit technique based on Gaze Gesture, Dwell time, and Pursuit; and one implicit technique based on reading speed to support touch-free, page-scrolling on a reading application. We conduct a 20-participant user study under both sitting and walking settings and our results reveal that Gaze Gesture and Dwell time-based interfaces are more robust while walking and Gaze Gesture has achieved consistently good scores on usability while not causing high cognitive workload.

Static Laser Feedback Interferometry-Based Gaze Estimation for Wearable Glasses

Fast and robust gaze estimation is a key technology for wearable glasses, as it enables novel methods of user interaction as well as display enhancement applications, such as foveated rendering. State-of-the-art video-based systems lack a high update rate, integrateability, slippage robustness, and low power consumption. To overcome these limitations, we propose a model-based fusion algorithm to estimate gaze from multiple static laser feedback interferometry (LFI) sensors, which are capable of measuring distance toward the eye and the eye’s rotational velocity. The proposed system is ambient light robust and robust to glasses slippage. During evaluation, a gaze accuracy of 1.79° at an outstanding update rate of 1 kHz is achieved, while the sensors consume only a fraction of the power compared with the state-of-the-art video-based system.

Multi-View Multi-Modal Head-Gaze Estimation for Advanced Indoor User Interaction

Gaze estimation is one of the most promising technologies for supporting indoor monitoring and interaction systems. However, previous gaze estimation techniques generally work only in a controlled laboratory environment because they require a number of high-resolution eye images. This makes them unsuitable for welfare and healthcare facilities with the following challenging characteristics: 1) users’ continuous movements, 2) various lighting conditions, and 3) a limited amount of available data. To address these issues, we introduce a multi-view multi-modal head-gaze estimation system that translates the user’s head orientation into the gaze direction. The proposed system captures the user using multiple cameras with depth and infrared modalities to train more robust gaze estimators under the aforementioned conditions. To this end, we implemented a deep learning pipeline that can handle different types and combinations of data. The proposed system was evaluated using the data collected from 10 volunteer participants to analyze how the use of single/multiple cameras and modalities affect the performance of head-gaze estimators. Through various experiments, we found that 1) an infrared-modality provides more useful features than a depth-modality, 2) multi-view multi-modal approaches provide better accuracy than single-view single-modal approaches, and 3) the proposed estimators achieve a high inference efficiency that can be used in real-time applications.

An eye tracking based virtual reality system for use inside magnetic resonance imaging systems

Patients undergoing Magnetic Resonance Imaging (MRI) often experience anxiety and sometimes distress prior to and during scanning. Here a full MRI compatible virtual reality (VR) system is described and tested with the aim of creating a radically different experience. Potential benefits could accrue from the strong sense of immersion that can be created with VR, which could create sense experiences designed to avoid the perception of being enclosed and could also provide new modes of diversion and interaction that could make even lengthy MRI examinations much less challenging. Most current VR systems rely on head mounted displays combined with head motion tracking to achieve and maintain a visceral sense of a tangible virtual world, but this technology and approach encourages physical motion, which would be unacceptable and could be physically incompatible for MRI. The proposed VR system uses gaze tracking to control and interact with a virtual world. MRI compatible cameras are used to allow real time eye tracking and robust gaze tracking is achieved through an adaptive calibration strategy in which each successive VR interaction initiated by the subject updates the gaze estimation model. A dedicated VR framework has been developed including a rich virtual world and gaze-controlled game content. To aid in achieving immersive experiences physical sensations, including noise, vibration and proprioception associated with patient table movements, have been made congruent with the presented virtual scene. A live video link allows subject-carer interaction, projecting a supportive presence into the virtual world.

XSDL: stroboscopic differential lighting eye tracker with extended temporal support

Eye tracking (ET) for gaze interaction in wearable computing imposes harder constraints on computational efficiency and illumination conditions than remote ET. In this paper we present xSDL, an extended temporal support computer vision algorithm for accurate, robust, and efficient pupil detection and gaze estimation. The robustness and efficiency of xSDL partly come from the use of stroboscopic differential lighting (SDL), an extension of the differential lighting pupil detection technique developed in the 90’s. Due to the erratic behavior of eye movements, traditional computer vision tracking techniques (such as Kalman filters) do not perform well, so most ET techniques simply detect some eye feature (such as the pupil center) at every frame. Extended temporal support uses keyframes selected during eye fixations and a simple translation model of the pupil to further improve the computational performance of SDL. A prototype composed of two independent acquisition systems was developed to evaluate the performance of xSDL and other four state-of-the-art ET techniques under similar conditions. Our results show that xSDL outperforms those four algorithms, both in speed (close to 2000 Hz using 240 line frames) and accuracy.

Perceptual integration of head and eye cues to gaze direction in schizophrenia.

The perceptual mechanisms that underlie social experience in schizophrenia are increasingly becoming a target of empirical research. In the context of low-level vision, there is evidence for a reduction in the integration of sensory features in schizophrenia (e.g. increased thresholds for contour detection and motion coherence). In the context of higher-level vision, comparable differences in the integration of sensory features of the face could in theory impair the recognition of important social cues. Here we examine how the sense of where other people are looking relies upon the integration of eye-region cues and head-region cues. Adults with schizophrenia viewed face images designed to elicit the ‘Wollaston illusion’, a perceptual phenomenon in which the perceived gaze direction associated with a given pair of eyes is modulated by the surrounding sensory context. We performed computational modelling of these psychophysical data to quantify individual differences in the use of facial cues to gaze direction. We find that adults with schizophrenia exhibit a robust perceptual effect whereby their sense of other people's direction of gaze is strongly biased by sensory cues relating to head orientation in addition to eye region information. These results indicate that the visual integration of facial cues to gaze direction in schizophrenia is intact, helping to constrain theories of reduced integrative processing in higher-level and lower-level vision. In addition, robust gaze processing was evident in the tested participants despite reduced performance on a theory of mind task designed to assess higher-level social cognition.

Unconstrained and Calibration-Free Gaze Estimation in a Room-Scale Area Using a Monocular Camera

Gaze estimation using monocular cameras has significant commercial applicability, and many studies have been undertaken on head pose-invariant and calibration-free gaze estimation. The head positions in existing data sets used in these studies are, however, limited to the vicinity of the camera, and methods trained on such data sets are not applicable when subjects are at greater distances from the camera. In this paper, we create a room-scale gaze data set with large variations in head poses to achieve robust gaze estimation across a broader range of widths and depths. The head positions are much farther from the camera, and the resolution of the eye image is lower than in conventional data sets. To address this issue, we propose a likelihood evaluation method based on edge gradients with dense particles for iris tracking, which achieves robust tracking at low-resolution eye images. Cross-validation experiments show that our proposed method is more accurate than conventional methods on all the individuals in our data set.

Efficient eye typing with 9-direction gaze estimation

Vision based text entry systems aim to help disabled people achieve text communication using eye movement. Most previous methods have employed an existing eye tracker to predict gaze direction and design an input method based upon that. However, these methods can result in eye tracking quality becoming easily affected by various factors and lengthy amounts of time for calibration. Our paper presents a novel efficient gaze based text input method, which has the advantage of low cost and robustness. Users can type in words by looking at an on-screen keyboard and blinking. Rather than estimate gaze angles directly to track eyes, we introduce a method that divides the human gaze into nine directions. This method can effectively improve the accuracy of making a selection by gaze and blinks. We build a Convolutional Neural Network (CNN) model for 9-direction gaze estimation. On the basis of the 9-direction gaze, we use a nine-key T9 input method which is widely used in candy bar phones. Bar phones were very popular in the world decades ago and have cultivated strong user habits and language models. To train a robust gaze estimator, we created a large-scale dataset with images of eyes sourced from 25 people. According to the results from our experiments, our CNN model is able to accurately estimate different people's gaze under various lighting conditions by different devices. In considering disable people's needs, we removed the complex calibration process. The input methods can run in screen mode and portable off-screen mode. Moreover, The datasets used in our experiments are made available to the community to allow further experimentation.

Investigation of continual operation procedures via a user centered gaze control by means of flexible gaze gestures

This work presents the pursued research results of a study about the continual control of virtual control elements by gaze, which was released in the proceedings of the spring congress 2017 arranged by the Society of Ergonomics and Work Science (German: GfA). The motivation to provide a more intuitive and robust gaze interaction method leads to the usage of specific gaze gestures. These gestures are based on unique gaze patterns which are distinct from typical intended as well as unintended viewing. For enabling extensive interactions through those gestures a gesture set was developed and tested in a first attempt in terms of its self-descriptiveness and learnability. The second objective of this study arises from the necessity to provide the user with vital information contents about his operation possibilities during the interaction by gaze gestures. By means of the design parameterization two different interfaces were tested concerning its design parameter shape. The objective was to come to first conclusions about the guiding method of representing specific visual interface elements to establish a satisfying gaze control. The results show that the applied gesture set can be learned quite easily and helps the user to connect the symbolic of the gesture with the corresponding execution. Furthermore, the experimental results give evidence of the preference of reference points (grid points) as the guiding elements rather than a polyline (square) shape in the gaze-gesture-controlled interface Practical Relevance: The research of gaze control is on the cutting edge. The control by gestures, for example through touch operation, has asserted themselves in various fields of application. These methods were employed to promote trends like autonomous driving, service design and individualization. Powerful and at the same time cost efficient eye tracking systems favor the extensive application of this technology beyond the recent research fields of medical diagnostic and individual interaction solutions for handicapped people. Furthermore, gaze control has great potential to help the user in fields like teleoperation, complex interfaces and cockpits as well as in the public service sector. Consequently it constitutes an innovative, intuitive and inspiring way of human system interaction which could also be useful combined with other interaction modalities. Besides all these arising opportunities in the mentioned fields, there is a big necessity to conscientiously concern the advantages and disadvantages of this technology. According to the research which has been accomplished in other, more established fields of interaction a well-defined design guideline, both for the gaze-controlled interface and the applied control characteristic is inevitable. The presented work depicts another step towards this approach.

A Bayesian hierarchy for robust gaze estimation in human–robot interaction

In this text, we present a probabilistic solution for robust gaze estimation in the context of human–robot interaction. Gaze estimation, in the sense of continuously assessing gaze direction of an interlocutor so as to determine his/her focus of visual attention, is important in several important computer vision applications, such as the development of non-intrusive gaze-tracking equipment for psychophysical experiments in neuroscience, specialised telecommunication devices, video surveillance, human–computer interfaces (HCI) and artificial cognitive systems for human–robot interaction (HRI), our application of interest. We have developed a robust solution based on a probabilistic approach that inherently deals with the uncertainty of sensor models, but also and in particular with uncertainty arising from distance, incomplete data and scene dynamics. This solution comprises a hierarchical formulation in the form of a mixture model that loosely follows how geometrical cues provided by facial features are believed to be used by the human perceptual system for gaze estimation. A quantitative analysis of the proposed framework's performance was undertaken through a thorough set of experimental sessions. Results show that the framework performs according to the difficult requirements of HRI applications, namely by exhibiting correctness, robustness and adaptiveness.

Feature and motion-based gaze cuing is linked with reduced social competence

Gaze following is a fundamental ability that plays an important role in human social function. However, the link between these two processes remains elusive. On the one hand, typically developing persons show robust gaze following in laboratory cuing tasks. On the other hand, investigations with individuals with autism suggest that reduced social competence in this population may partly reflect an atypical access to social information through attending to perceptual changes that normally accompany gaze shifts, like luminance or motion transients. Here we investigated if gaze cuing in typically developing individuals was modulated by similar task-irrelevant perceptual changes. In Experiment 1, a social gaze cue was presented with or without a luminance change. In Experiment 2, a social gaze cue was presented together with a motion cue. Both experiments indicated reduced magnitudes of gaze cuing in persons with low social competence on trials containing an irrelevant perceptual change. This suggests that similarly to individuals with autism, typically developing persons with low social competence also utilize idiosyncratic perceptual changes in the environment to access social content, revealing strong links between basic gaze following abilities and a range of social competence within typical individuals.

Robust Head Mounted Wearable Eye Tracking System for Dynamical Calibration

In this work, a new head mounted eye tracking system is presented. Based on computer vision techniques, the system integrates eye images and head movement, in real time, performing a robust gaze point tracking. Nystagmus movements due to vestibulo-ocular reflex are monitored and integrated. The system proposed here is a strongly improved version of a previous platform called HATCAM, which was robust against changes of illumination conditions. The new version, called HAT-Move, is equipped with accurate inertial motion unit to detect the head movement enabling eye gaze even in dynamical conditions. HAT-Move performance is investigated in a group of healthy subjects in both static and dynamic conditions, i.e. when head is kept still or free to move. Evaluation was performed in terms of amplitude of the angular error between the real coordinates of the fixed points and those computed by the system in two experimental setups, specifically, in laboratory settings and in a 3D virtual reality (VR) scenario. The achieved results showed that HAT-Move is able to achieve eye gaze angular error of about 1 degree along both horizontal and vertical directions

Mental state attribution and the gaze cueing effect.

Theory of mind is said to be possessed by an individual if he or she is able to impute mental states to others. Recently, some authors have demonstrated that such mental state attributions can mediate the "gaze cueing" effect, in which observation of another individual shifts an observer's attention. One question that follows from this work is whether such mental state attributions produce mandatory modulations of gaze cueing. Employing the basic gaze cueing paradigm, together with a technique commonly used to assess mental-state attribution in nonhuman animals, we manipulated whether the gazing agent could see the same thing as the participant (i.e., the target) or had this view obstructed by a physical barrier. We found robust gaze cueing effects, even when the observed agent in the display could not see the same thing as the participant. These results suggest that the attribution of "seeing" does not necessarily modulate the gaze cueing effect.

Gaze behaviour during space perception and spatial decision making

A series of four experiments investigating gaze behavior and decision making in the context of wayfinding is reported. Participants were presented with screenshots of choice points taken in large virtual environments. Each screenshot depicted alternative path options. In Experiment 1, participants had to decide between them to find an object hidden in the environment. In Experiment 2, participants were first informed about which path option to take as if following a guided route. Subsequently, they were presented with the same images in random order and had to indicate which path option they chose during initial exposure. In Experiment 1, we demonstrate (1) that participants have a tendency to choose the path option that featured the longer line of sight, and (2) a robust gaze bias towards the eventually chosen path option. In Experiment 2, systematic differences in gaze behavior towards the alternative path options between encoding and decoding were observed. Based on data from Experiments 1 and 2 and two control experiments ensuring that fixation patterns were specific to the spatial tasks, we develop a tentative model of gaze behavior during wayfinding decision making suggesting that particular attention was paid to image areas depicting changes in the local geometry of the environments such as corners, openings, and occlusions. Together, the results suggest that gaze during a wayfinding tasks is directed toward, and can be predicted by, a subset of environmental features and that gaze bias effects are a general phenomenon of visual decision making.

Iris Center Corneal Reflection Method for Gaze Tracking Using Visible Light

Very few attempts, if any, have been made to use visible light in corneal reflection approaches to the problem of gaze tracking. The reasons usually given to justify the limited application of this type of illumination are that the required image features are less accurately depicted, and that visible light may disturb the user. The aim of this paper is to show that it is possible to overcome these difficulties and build an accurate and robust gaze tracker under these circumstances. For this purpose, visible light is used to obtain the corneal reflection or glint in a way analogous to the well-known pupil center corneal reflection technique. Due to the lack of contrast, the center of the iris is tracked instead of the center of the pupil. The experiments performed in our laboratory have shown very satisfactory results, allowing free-head movement and no need of recalibration.

Gabor Directional Binary Pattern: An Image Descriptor for Gaze Estimation

This paper proposes an image descriptor, Gabor Directional Binary Pattern (GDBP), for robust gaze estimation. In GDBP, Gabor magnitude information is extracted firstly from a cropped subimage. The local directional derivations are then utilized to encode the binary patterns in the given orientations. As an image descriptor, GDBP can suppress noises and robustness to illumination variations. Meanwhile, the encoding pattern can emphasize boundary. We use the GDBP features of eye regions and adopt the Support Vector Regression (SVR) to approximate the gaze mapping function, which is then used to predict the gaze direction with respect to the camera coordinate system. In the person-independent experiments, our dataset includes 4089 samples of 11 persons. Experimental results show that the gaze estimation can achieve an accuracy of less than by using the proposed GDBP and SVR.

A robust gaze detection method by compensating for facial movements based on corneal specularities

In this paper, we propose an analytical method for robust gaze detection by compensating for facial movements based on corneal specularities. Our method has following four novelties compared to previous works. (i) We used a USB (Universal Serial Bus) camera with four IR-LED (infra-red light emitting diodes) which were attached on each corner of a monitor, so we could simply calculate the user’s gaze position based on 2D relations between the center of pupil and four corneal reflections without considering the complex 3D relations among the camera, the monitor and the pupil coordinates, (ii) the detection accuracy of four corneal reflections was much enhanced by removing the imposter reflections based on the analysis of 3D structure of the human eye, (iii) by analyzing and compensating the shape change of quadrangle which was formed by four corneal reflections according to facial movement, we could reduce the gaze detection errors caused by facial movements, (iv) by compensating the error caused by angle Kappa based on user-dependent calibration, we could much enhance the accuracy of our gaze detection method. Experimental results showed that the gaze detection error was about 1.0° and we could know our method could track user’s gaze position at real-time speed.

VISION-BASED GAZE ESTIMATION METHOD FOR STEREOSCOPIC DISPLAY SYSTEMS

In this paper, we propose a new near infra-red (IR) vision-based gaze tracking method, which is not affected by facial movements in stereoscopic virtual reality (VR) systems. Our proposed method has following five novelties compared to previous works. (i) By using our gaze tracker instead of a conventional input device, users felt more interested and immersed in the stereoscopic VR system, (ii) by using an IR light passing filter and an IR illuminator, we were able to implement a robust gaze tracker irrespective of environmental lighting conditions, (iii) we used the simple 2D based eye and face gaze estimation method without considering the complicated 3D structure and motion of human eye and face, which could reduce the system complexities, (iv) to prevent the unintended movement of gaze position by natural eye blinking, we discriminated the eye blinking by measuring pupil size. And this information was used for button clicking and mode toggling based on the analysis of user's fatigue. That is, the measured fatigue information of user was used for the mode toggling between gaze tracking mode and the conventional mouse input mode, (v) the proposed method allowed the natural movement of user's head by tracking the IR rhombus-shaped pattern, which was not affected by the change of environmental lighting.Experimental results showed that the gaze estimation accuracy of the proposed method was 0.98 degrees. In addition, the subjective tests showed that the VR application using the proposed method was more interesting and immersive to users than those using conventional input devices.