Video-based Eye Tracker Research Articles

Performance of a simple remote video-based eye tracker with GPU acceleration

Eye tracking is a well-established tool that is often utilised in research. There are currently many different types of eye trackers available, but they are either expensive, or provide a relatively low sampling frequency. The eye tracker presented in this paper was developed in an effort to address the lack of low-cost high-speed eye trackers. It utilises the Graphical Processing Unit (GPU) in an attempt to parallelise aspects of the process to localize feature points in eye images to attain higher sampling frequencies. Moreover, the proposed implementation allows for the system to be used on a variety of different GPUs. The developed solution is capable of sampling at frequencies of 200 Hz and higher, while allowing for head movements within an area of 10×6×10 cm and an average accuracy of one degree of visual angle. The entire system can be built for less than 700 euros, and will run on a mid-range laptop.

The within-task criterion to determine successful and unsuccessful table tennis players

The aim of this study was to examine the differences in visual search behaviour between a group of expert-level and one of novice table tennis players, to determine the temporal and spatial aspects of gaze orientation associated with correct responses. Expert players were classified as successful or unsuccessful depending on their performance in a video-based test of anticipation skill involving two kinds of stroke techniques: forehand top spin and backhand drive. Eye movements were recorded binocularly with a video-based eye tracking system. Successful experts were more effective than novices and unsuccessful experts in accurately anticipating both type and direction of stroke, showing fewer fixations of longer duration. Participants fixated mainly on arm area during forehand top spin, and on hand–racket and trunk areas during backhand drive. This study can help to develop interventions that facilitate the acquisition of anticipatory skills by improving visual search strategies.

Pupil size influences the eye-tracker signal during saccades

While it is known that scleral search coils—measuring the rotation of the eye globe—and modern, video based eye trackers—tracking the center of the pupil and the corneal reflection (CR)—produce signals with different properties, the mechanisms behind the differences are less investigated. We measure how the size of the pupil affects the eye-tracker signal recorded during saccades with a common pupil-CR eye-tracker. Eye movements were collected from four healthy participants and one person with an aphakic eye while performing self-paced, horizontal saccades at different levels of screen luminance and hence pupil size. Results show that pupil-, and gaze-signals, but not the CR-signal, are affected by the size of the pupil; changes in saccade peak velocities in the gaze signal of more than 30% were found. It is important to be aware of this pupil size dependent change when comparing fine grained oculomotor behavior across participants and conditions.

Video-Based Eye Tracking in Sex Research: A Systematic Literature Review

Although eye tracking has been used for decades, it has gained popularity in the area of sex research only recently. The aim of this article is to examine the potential merits of eye tracking for this field. We present a systematic review of the current use of video-based eye-tracking technology in this area, evaluate the findings, and identify future research opportunities. A total of 34 relevant studies published between 2006 and 2014 were identified for inclusion by means of online databases and other methods. We grouped them into three main areas of research: body perception and attractiveness, forensic research, and sexual orientation. Despite the methodological and theoretical differences across the studies, eye tracking has been shown to be a promising tool for sex research. The article suggests there is much potential for further studies to employ this technique because it is noninvasive and yet still allows for the assessment of both conscious and unconscious perceptional processes. Furthermore, eye tracking can be implemented in investigations of various theoretical backgrounds, ranging from biology to the social sciences.

Effects of Prism Eyeglasses on Objective and Subjective Fixation Disparity.

In optometry of binocular vision, the question may arise whether prisms should be included in eyeglasses to compensate an oculomotor and/or sensory imbalance between the two eyes. The corresponding measures of objective and subjective fixation disparity may be reduced by the prisms, or the adaptability of the binocular vergence system may diminish effects of the prisms over time. This study investigates effects of wearing prisms constantly for about 5 weeks in daily life. Two groups of 12 participants received eyeglasses with prisms having either a base-in direction or a base-out direction with an amount up to 8 prism diopters. Prisms were prescribed based on clinical fixation disparity test plates at 6 m. Two dependent variables were used: (1) subjective fixation disparity was indicated by a perceived offset of dichoptic nonius lines that were superimposed on the fusion stimuli and (2) objective fixation disparity was measured with a video based eye tracker relative to monocular calibration. Stimuli were presented at 6 m and included either central or more peripheral fusion stimuli. Repeated measurements were made without the prisms and with the prisms after about 5 weeks of wearing these prisms. Objective and subjective fixation disparity were correlated, but the type of fusion stimulus and the direction of the required prism may play a role. The prisms did not reduce the fixation disparity to zero, but induced significant changes in fixation disparity with large effect sizes. Participants receiving base-out prisms showed hypothesized effects, which were concurrent in both types of fixation disparity. In participants receiving base-in prisms, the individual effects of subjective and objective effects were negatively correlated: the larger the subjective (sensory) effect, the smaller the objective (motor) effect. This response pattern was related to the vergence adaptability, i.e. the individual fusional vergence reserves.

The art of braking: Post saccadic oscillations in the eye tracker signal decrease with increasing saccade size

Recent research has shown that the pupil signal from video-based eye trackers contains post saccadic oscillations (PSOs). These reflect pupil motion relative to the limbus (Nyström, Hooge, & Holmqvist, 2013). More knowledge about video-based eye tracker signals is essential to allow comparison between the findings obtained from modern systems, and those of older eye tracking technologies (e.g. coils and measurement of the Dual Purkinje Image—DPI). We investigated PSOs in horizontal and vertical saccades of different sizes with two high quality video eye trackers. PSOs were very similar within observers, but not between observers. PSO amplitude decreased with increasing saccade size, and this effect was even stronger in vertical saccades; PSOs were almost absent in large vertical saccades. Based on this observation we conclude that the occurrence of PSOs is related to deceleration at the end of a saccade. That PSOs are saccade size dependent and idiosyncratic is a problem for algorithmic determination of saccade endings. Careful description of the eye tracker, its signal, and the procedure used to extract saccades is required to enable researchers to compare data from different eye trackers.

Dummy eye measurements of microsaccades: Testing the influence of system noise and head movements on microsaccade detection in a popular video-based eye tracker

Whereas early studies of microsaccades have predominantly relied on custom-built eye trackers and manual tagging of microsaccades, more recent work tends to use video-based eye tracking and automated algorithms for microsaccade detection. While data from these newer studies suggest that microsaccades can be reliably detected with video-based systems, this has not been systematically evaluated. I here present a method and data examining microsaccade detection in an often used video-based system (the Eyelink II system) and a commonly used detection algorithm (Engbert & Kliegl, 2003; Engbert & Mergenthaler, 2006). Recordings from human participants and those obtained using a pair of dummy eyes, mounted on a pair of glasses either worn by a human participant (i.e., with head motion) or a dummy head (no head motion) were compared. Three experiments were conducted. The first experiment suggests that when microsaccade measurements make use of the pupil detection mode, microsaccade detections in the absence of eye movements are sparse in the absence of head movements, but frequent with head movements (despite the use of a chin rest). A second experiment demonstrates that by using measurements that rely on a combination of corneal reflection and pupil detection, false microsaccade detections can be largely avoided as long as a binocular criterion is used. A third experiment examines whether past results may have been affected by possible incorrect detections due to small head movements. It shows that despite the many detections due to head movements, the typical modulation of microsaccade rate after stimulus onset is found only when recording from the participants’ eyes.

Abstract W P140: Post-stroke Oculomotor Abnormalities evident during Objective Eye Tracking but Not under Clinical Assessment

Introduction: Visual abnormalities and manual motor control have been studied extensively after stroke, but an understanding of oculomotor control post-stroke has not. Recent studies have revealed that in visually guided reaches arm movements are planned during eye movement execution, which may contribute to increased task complexity. In fact, in healthy controls during visually guided reaches, the onset of eye movement is delayed, its velocity reduced, and endpoint errors are larger relative to isolated eye movements. Our objective in this experiment was to examine the temporal properties of eye movement execution for stroke patients with no diagnosed visual impairment. The goal is to improve understanding of oculomotor control in stroke relative to normal function, and ultimately further understand its coordination with manual motor control during joint eye and hand movements. We hypothesized that stroke patients would show abnormal initiation or onset latency for saccades made in an eye movement task, as compared to healthy controls. Methods: We measured the kinematics of eye movements during point-to-point saccades; there was an initial static, fixation point and the stimulus was a flashed target on a computer monitor. We used a video-based eye tracker for objective recording of the eye at a sampling frequency of 2000 Hz (SR Research, Eyelink). 10 stroke subjects, over 4 months from injury and with no diagnosed visual impairment, and 10 healthy controls completed 432 saccades in a serial fashion. Results: Stroke patients had significantly faster onset latencies as compared to healthy controls during saccades (99.5ms vs. 245.2ms, p=0.00058). Conclusion: A better understanding of the variations in oculomotor control post-stroke, which may go unnoticed during clinical assessment, may improve understanding of how eye control synchronizes with arm or manual motor control. This knowledge could assist in tailoring rehabilitative strategies to amplify motor recovery. For next steps, we will perform objective eye and hand recordings during visually guided reaches post-stroke to better understand the harmonization or lack thereof after neurologic insult.

Pupil size dynamics during fixation impact the accuracy and precision of video-based gaze estimation

Video-based eye tracking relies on locating pupil center to measure gaze positions. Although widely used, the technique is known to generate spurious gaze position shifts up to several degrees in visual angle because pupil centration can change without eye movement during pupil constriction or dilation. Since pupil size can fluctuate markedly from moment to moment, reflecting arousal state and cognitive processing during human behavioral and neuroimaging experiments, the pupil size artifact is prevalent and thus weakens the quality of the video-based eye tracking measurements reliant on small fixational eye movements. Moreover, the artifact may lead to erroneous conclusions if the spurious signal is taken as an actual eye movement. Here, we measured pupil size and gaze position from 23 human observers performing a fixation task and examined the relationship between these two measures. Results disclosed that the pupils contracted as fixation was prolonged, at both small (<16s) and large (∼4min) time scales, and these pupil contractions were accompanied by systematic errors in gaze position estimation, in both the ellipse and the centroid methods of pupil tracking. When pupil size was regressed out, the accuracy and reliability of gaze position measurements were substantially improved, enabling differentiation of 0.1° difference in eye position. We confirmed the presence of systematic changes in pupil size, again at both small and large scales, and its tight relationship with gaze position estimates when observers were engaged in a demanding visual discrimination task.

Parametric and Nonparametric Analysis of Eye-Tracking Data by Anomaly Detection

An approach to smooth pursuit eye movement's analysis by means of stochastic anomaly detection is presented and applied to the problem of distinguishing between patients diagnosed with Parkinson's disease and normal controls. Both parametric Wiener model-based techniques and nonparametric modeling utilizing a description of the involved probability density functions in orthonormal bases are considered. The necessity of proper visual stimuli design for the accuracy of mathematical modeling is highlighted and a formal method for producing such stimuli is suggested. The efficacy of the approach is demonstrated on experimental data collected by means of a commercial video-based eye tracker.

The influence of crystalline lens accommodation on post-saccadic oscillations in pupil-based eye trackers

It is well known that the crystalline lens (henceforth lens) can oscillate (or ‘wobble’) relative to the eyeball at the end of saccades. Recent research has proposed that such wobbling of the lens is a source of post-saccadic oscillations (PSOs) seen in data recorded by eye trackers that estimate gaze direction from the location of the pupil. Since the size of the lens wobbles increases with accommodative effort, one would predict a similar increase of PSO-amplitude in data recorded with a pupil based eye tracker. In four experiments, we investigated the role of lens accommodation on PSOs in a video-based eye tracker. In Experiment 1, we replicated previous results showing that PSO-amplitudes increase at near viewing distances (large vergence angles), when the lens is highly accommodated. In Experiment 2a, we manipulated the accommodative state of the lens pharmacologically using eye drops at a fixed viewing distance and found, in contrast to Experiment 1, no significant difference in PSO-amplitude related to the accommodative state of the lens. Finally, in Experiment 2b, the effect of vergence angle was investigated by comparing PSO-amplitudes at near and far while maintaining a fixed lens accommodation. Despite the pharmacologically fixed degree of accommodation, PSO-amplitudes were systematically larger in the near condition. In summary, PSOs cannot exhaustively be explained by lens wobbles. Possible confounds related to pupil size and eye-camera angle are investigated in Experiments 3 and 4, and alternative mechanisms behind PSOs are probed in the discussion.

Video-Based Eye Tracking to Detect the Attention Shift

Eye and head movements evoked in response to obvious visual attention shifts. However, there has been little progress on the causes of absent-mindedness so far. The paper proposes an attention awareness system that captures the conditions regarding the interaction of eye gaze and head pose under various attentional switching in computer classroom. Via the algorithm of complexion area detection, eye location and eye tracking, the system detects the shifts of the subject's attention, records it and sends a notification to the class teacher. In five variant experiments of attentional shift, the authors find the cues of subjects when turning to other directions, moving forwards or backwards, and closing eyes. It applies even if the students wear glasses or have hair fringe. The findings of absent-mindedness are very important for psychologists or educators conducting post-hoc analysis. These experimental results show eye tracking are useful for detecting shift of attention.

Correcting video-based eye tracking signals for pupil size artifacts

Correcting video-based eye tracking signals for pupil size artifacts

A Low-Computational Approach on Gaze Estimation With Eye Touch System

Among various approaches to eye tracking systems, light-reflection based systems with non-imaging sensors, e.g., photodiodes or phototransistors, are known to have relatively low complexity; yet, they provide moderately accurate estimation of the point of gaze. In this paper, a low-computational approach on gaze estimation is proposed using the Eye Touch system, which is a light-reflection based eye tracking system, previously introduced by the authors. Based on the physical implementation of Eye Touch, the sensor measurements are now utilized in low-computational least-squares algorithms to estimate arbitrary gaze directions, unlike the existing light reflection-based systems, including the initial Eye Touch implementation, where only limited predefined regions were distinguished. The system also utilizes an effective pattern classification algorithm to be able to perform left, right, and double clicks based on respective eye winks with significantly high accuracy. In order to avoid accuracy problems for sensitive sensor biasing hardware, a robust custom microcontroller-based data acquisition system is developed. Consequently, the physical size and cost of the overall Eye Touch system are considerably reduced while the power efficiency is improved. The results of the experimental analysis over numerous subjects clearly indicate that the proposed eye tracking system can classify eye winks with 98% accuracy, and attain an accurate gaze direction with an average angular error of about 0.93 °. Due to its lightweight structure, competitive accuracy and low-computational requirements relative to video-based eye tracking systems, the proposed system is a promising human-computer interface for both stationary and mobile eye tracking applications.

Response time, visual search strategy, and anticipatory skills in volleyball players.

This paper aimed at comparing expert and novice volleyball players in a visuomotor task using realistic stimuli. Videos of a volleyball setter performing offensive action were presented to participants, while their eye movements were recorded by a head-mounted video based eye tracker. Participants were asked to foresee the direction (forward or backward) of the setter's toss by pressing one of two keys. Key-press response time, response accuracy, and gaze behaviour were measured from the first frame showing the setter's hand-ball contact to the button pressed by the participants. Experts were faster and more accurate in predicting the direction of the setting than novices, showing accurate predictions when they used a search strategy involving fewer fixations of longer duration, as well as spending less time in fixating all display areas from which they extract critical information for the judgment. These results are consistent with the view that superior performance in experts is due to their ability to efficiently encode domain-specific information that is relevant to the task.

Slowed saccades and increased square wave jerks in essential tremor.

Eye movements in essential tremor (ET) are poorly described and may present useful information on the underlying pathophysiology of the disorder. Sixty patients with ET, including 15 de novo untreated patients, and 60 age-matched controls constitute the study population. A video-based eye tracker was used to assess binocular eye position. Oculomotor function was assessed while subjects followed random horizontally and vertically step-displaced targets. For all reflexive saccades, latencies were increased in ET subjects by a mean of 16.3% (p<0.01). Saccades showed reduced peak velocities with a lengthy, wavering velocity plateau, followed by slowed decelerations. For larger 30°+ saccades, peak velocities were decreased by a mean of 25.2% (p<0.01) and durations increased by 31.8% (p<0.01). The frequency of square wave jerks (SWJs) in patients was more than triple that of controls (p<0.0001). Despite frequent interruptions by SWJs, fixations were otherwise stable and indistinguishable from controls (root mean square [RMS] velocity, p = 0.324). The abnormal eye movement parameters were independent of disease duration, tremor severity, and medication therapy. In contrast to normally swift onset and efficient acceleration/deceleration movements, saccades in ET are characterized by abnormally prolonged latencies and slowed velocity profiles. Although ET subjects maintain highly stable fixations, they are interrupted by increased numbers of SWJs. This study reveals novel oculomotor deficits in ET, which are distinct from the eye movement dysfunction of other movement disorders, supporting a role for eye tracking to assist in the differential diagnoses of not only atypical, but also more common movement disorders.

Direct measurement of the system latency of gaze-contingent displays

Gaze-contingent displays combine a display device with an eyetracking system to rapidly update an image on the basis of the measured eye position. All such systems have a delay, the system latency, between a change in gaze location and the related change in the display. The system latency is the result of the delays contributed by the eyetracker, the display computer, and the display, and it is affected by the properties of each component, which may include variability. We present a direct, simple, and low-cost method to measure the system latency. The technique uses a device to briefly blind the eyetracker system (e.g., for video-based eyetrackers, a device with infrared light-emitting diodes (LED)), creating an eyetracker event that triggers a change to the display monitor. The time between these two events, as captured by a relatively low-cost consumer camera with high-speed video capability (1,000Hz), is an accurate measurement of the system latency. With multiple measurements, the distribution of system latencies can be characterized. The same approach can be used to synchronize the eye position time series and a video recording of the visual stimuli that would be displayed in a particular gaze-contingent experiment. We present system latency assessments for several popular types of displays and discuss what values are acceptable for different applications, as well as how system latencies might be improved.

Mapping the Pupil-Glint Vector to Gaze Coordinates in a Simple Video-Based Eye Tracker

In a video-based eye tracker, the normalized pupil-glint vector changes as the eyes move. Using an appropriate model, the pupil-glint vector can be mapped to gaze coordinates. Using a simple hardware configuration with one camera and one infrared source, several mapping functions – some from literature and some derived here – were compared with one another with respect to the accuracy that could be achieved. The study served to confirm the results of a previous study with another data set and to expand on the possibilities that are considered from the previous study. The data of various participants was examined for trends which led to derivation of a mapping model that proved to be more accurate than all but one model from literature. It was also shown that the best calibration configuration for this hardware setup is one that contains fourteen targets while taking about 20 seconds for the procedure to be completed.

Parametric and non-parametric system identification of oculomotor system with application to the analysis of smooth pursuit eye movements in Parkinson’s disease

Parametric and non-parametric system identification of oculomotor system with application to the analysis of smooth pursuit eye movements in Parkinson’s disease

Capturing and evaluating blinks from video-based eyetrackers

Blinks are related to several emotional states, and the present report describes a simple, reliable way to measure blinks from the video stream of an eye obtained during eyetracking, where the source of the eye video is a video camera attached to a head-mounted eyetracker. Computer vision techniques are employed to determine the moments that a blink starts and ends, for the purpose of calculating blink frequency and duration. The video is first processed to show blocks of eyelid and pupil movements, and is then analyzed for blink starts and ends. The moment of a blink start is reported when the eyelid starts to move quickly, exceeding a predetermined threshold. The end of a blink arises when the pupil size increases by less than a separate threshold. We observed several different blink patterns from different subjects, and our algorithm was designed to work for all of these patterns. We evaluated our algorithm by manually measuring the true blinks of five different subjects while they were eyetracked. To test the sensitivity and specificity of the algorithm, we employed a series of threshold values to plot the receiver operating characteristic curves. Using the best thresholds, we achieved excellent sensitivity (>90%) and specificity (>99%) over the five subjects. Potential applications of this research include real-time, nonintrusive, continuous and automated measurements of mental workload and other emotional states related to blink rates and durations.