Binocular Eye Tracking Research Articles

Ultra-low-cost 3D gaze estimation: an intuitive high information throughput compliment to direct brain–machine interfaces

Eye movements are highly correlated with motor intentions and are often retained by patients with serious motor deficiencies. Despite this, eye tracking is not widely used as control interface for movement in impaired patients due to poor signal interpretation and lack of control flexibility. We propose that tracking the gaze position in 3D rather than 2D provides a considerably richer signal for human machine interfaces by allowing direct interaction with the environment rather than via computer displays. We demonstrate here that by using mass-produced video-game hardware, it is possible to produce an ultra-low-cost binocular eye-tracker with comparable performance to commercial systems, yet 800 times cheaper. Our head-mounted system has 30 USD material costs and operates at over 120 Hz sampling rate with a 0.5–1 degree of visual angle resolution. We perform 2D and 3D gaze estimation, controlling a real-time volumetric cursor essential for driving complex user interfaces. Our approach yields an information throughput of 43 bits s−1, more than ten times that of invasive and semi-invasive brain–machine interfaces (BMIs) that are vastly more expensive. Unlike many BMIs our system yields effective real-time closed loop control of devices (10 ms latency), after just ten minutes of training, which we demonstrate through a novel BMI benchmark-–the control of the video arcade game ‘Pong’.

An assessment of eye-gaze potential within immersive virtual environments

In collaborative situations, eye gaze is a critical element of behavior which supports and fulfills many activities and roles. In current computer-supported collaboration systems, eye gaze is poorly supported. Even in a state-of-the-art video conferencing system such as the access grid, although one can see the face of the user, much of the communicative power of eye gaze is lost. This article gives an overview of some preliminary work that looks towards integrating eye gaze into an immersive collaborative virtual environment and assessing the impact that this would have on interaction between the users of such a system. Three experiments were conducted to assess the efficacy of eye gaze within immersive virtual environments. In each experiment, subjects observed on a large screen the eye-gaze behavior of an avatar. The eye-gaze behavior of that avatar had previously been recorded from a user with the use of a head-mounted eye tracker. The first experiment was conducted to assess the difference between users' abilities to judge what objects an avatar is looking at with only head gaze being viewed and also with eye- and head-gaze data being displayed. The results from the experiment show that eye gaze is of vital importance to the subjects, correctly identifying what a person is looking at in an immersive virtual environment. The second experiment examined whether a monocular or binocular eye-tracker would be required. This was examined by testing subjects' ability to identify where an avatar was looking from their eye direction alone, or by eye direction combined with convergence. This experiment showed that convergence had a significant impact on the subjects' ability to identify where the avatar was looking. The final experiment looked at the effects of stereo and mono-viewing of the scene, with the subjects being asked to identify where the avatar was looking. This experiment showed that there was no difference in the subjects' ability to detect where the avatar was gazing. This is followed by a description of how the eye-tracking system has been integrated into an immersive collaborative virtual environment and some preliminary results from the use of such a system.

Design and Evaluation of Categorical Search Results Interfaces

This research investigates the design and evaluation of interfaces for the presentation of search results using categorization. Card sorting study was conducted to design categorized search results interfaces. Eye-tracking technology was utilized to evaluate the designed category interfaces, comparing them with a conventional list interface. Two factors of category interfaces, the approach used to order categories and the number of abstracts initially displayed within each category, are investigated. Forty participants performed two types of tasks, navigation and information search, using a PC integrated with a non-intrusive Tobii 1750 binocular eye tracking system. Performance measures such as time for search and percentage of tasks completed did not reveal significant differences among the category and list interfaces. Process measures such as overall number of fixations, mean fixation durations, and relative change from the baseline pupil diameter also did not show significant differences among the category and list interfaces. Although no significant differences are found, there is a consistent positive trend in the performance, process, and subjective measures that supports design of categorical search results interfaces.

Binocular Coordination of the Eyes during Reading

Saccadic eye movements and fixations are the behavioral means by which we visually sample text during reading. Human oculomotor control is governed by a complex neurophysiological system involving the brain stem, superior colliculus, and several cortical areas. A very widely held belief among researchers investigating primate vision is that the oculomotor system serves to orient the visual axes of both eyes to fixate the same target point in space. It is argued that such precise positioning of the eyes is necessary to place images on corresponding retinal locations, such that on each fixation a single, nondiplopic, visual representation is perceived. Vision works actively through a continual sampling process involving saccades and fixations. Here we report that during normal reading, the eyes do not always fixate the same letter within a word. We also demonstrate that saccadic targeting is yoked and based on a unified cyclopean percept of a whole word since it is unaffected if different word parts are delivered exclusively to each eye via a dichoptic presentation technique. These two findings together suggest that the visual signal from each eye is fused at a very early stage in the visual pathway, even when the fixation disparity is greater than one character (0.29 deg), and that saccade metrics for each eye are computed on the basis of that fused signal.

A neural network for 3D gaze recording with binocular eye trackers

Using eye tracking for the investigation of visual attention has become increasingly popular during the last few decades. Nevertheless, only a small number of eye tracking studies have employed 3D displays, although such displays would closely resemble our natural visual environment. Besides higher cost and effort for the experimental setup, the main reason for the avoidance of 3D displays is the problem of computing a subject's current 3D gaze position based on the measured binocular gaze angles. The geometrical approaches to this problem that have been studied so far involved substantial error in the measurement of 3D gaze trajectories. In order to tackle this problem, we developed an anaglyph-based 3D calibration procedure and used a well-suited type of artificial neural network—a parametrized self-organizing map (PSOM)—to estimate the 3D gaze point from a subject's binocular eye-position data. We report an experiment in which the accuracy of the PSOM gaze-point estimation is compared to a geometrical solution. The results show that the neural network approach produces more accurate results than the geometrical method, especially for the depth axis and for distant stimuli.

Gaze tracking accuracy in humans: Two eyes are better than one

Many people viewing a small distant object report the use of a dominant eye, which may change when viewing the same object in right or left viewing fields. This study was designed to determine if the accuracy in left and right gaze recordings from a video-based binocular eye tracking system with eye–head integration would change when observers looked at targets in different viewing fields. We also wanted to determine whether system accuracy was best using the dominant eye, the average of both eyes or an alignment switching strategy for various viewing fields. Bilateral eye and head movements were recorded to determine point of gaze (POG) for subjects viewing targets with the head neutral or rotated left or right with a stationary body (static) or while stepping toward a target and back (dynamic). Constant, absolute and variable POG and rotational errors were determined. Results showed that although errors were highest in the static condition for POG errors and in the dynamic condition for rotational errors, absolute rotational angle errors were similar for static and dynamic conditions, indicating that subject to object distance played an important role in determining POG errors. Furthermore, errors were not consistently lower for a given eye or for a given eye within a specific viewing field for the head rotations performed. Since low errors across the viewing fields were found by averaging left and right errors, it was concluded that this average and should be used in future studies involving relatively small eye-in-head rotations.

Gaze-contingent control for minimally invasive robotic surgery

Objective: Recovering tissue depth and deformation during robotically assisted minimally invasive procedures is an important step towards motion compensation, stabilization and co-registration with preoperative data. This work demonstrates that eye gaze derived from binocular eye tracking can be effectively used to recover 3D motion and deformation of the soft tissue.Methods: A binocular eye-tracking device was integrated into the stereoscopic surgical console. After calibration, the 3D fixation point of the participating subjects could be accurately resolved in real time. A CT-scanned phantom heart model was used to demonstrate the accuracy of gaze-contingent depth extraction and motion stabilization of the soft tissue. The dynamic response of the oculomotor system was assessed with the proposed framework by using autoregressive modeling techniques. In vivo data were also used to perform gaze-contingent decoupling of cardiac and respiratory motion.Results: Depth reconstruction, deformation tracking, and motion stabilization of the soft tissue were possible with binocular eye tracking. The dynamic response of the oculomotor system was able to cope with frequencies likely to occur under most routine minimally invasive surgical operations.Conclusion: The proposed framework presents a novel approach towards the tight integration of a human and a surgical robot where interaction in response to sensing is required to be under the control of the operating surgeon.

Eye-Movement Responses to Disparity Vergence Stimuli with Artificial Monocular Scotomas

The effects of artificial monocular scotomas on eye-movement responses to horizontal disparity vergence stimuli were studied in six subjects with normal binocular vision. Subjects viewed stereoscopic 1.5° horizontal step disparity vergence stimuli through liquid crystal shutter glasses. The central portion of the stimulus presented to the right eye was removed to simulate monocular artificial scotomas of variable diameters (2° to 10°). Eye movements were recorded with a binocular head-mounted eye tracker. Responses included pure vergence, vergence followed by saccades, and pure saccadic eye movements. The rate of responses with saccadic eye movements increased with the diameter of the artificial scotoma (p < 0.0001); there was an increase in the rate of responses starting with saccades (p < 0.0001), as well as an increase in the rate of saccades after initial vergence responses (p < 0.01). The probability of saccades after initial vergence responses was affected by the open-loop gain of the vergence response (p < 0.001). The open-loop gain decreased with increased diameters of the artificial scotomas (p < 0.0001). As the diameter of the artificial scotomas increased, the amplitude of the initial vergence eye-movement responses decreased, and the prevalence of saccadic eye movements and asymmetric vergence increased. The effects of the diameter of artificial monocular scotomas on eye-movement responses in subjects with normal binocular vision are consistent with the effects of diameter of suppression scotomas on eye-movement responses to disparity vergence stimuli in patients with infantile esotropia.

3-D eye movement analysis.

This paper presents a novel three-dimensional (3-D) eye movement analysis algorithm for binocular eye tracking within virtual reality (VR). The user's gaze direction, head position, and orientation are tracked in order to allow recording of the user's fixations within the environment. Although the linear signal analysis approach is itself not new, its application to eye movement analysis in three dimensions advances traditional two-dimensional approaches, since it takes into account the six degrees of freedom of head movements and is resolution independent. Results indicate that the 3-D eye movement analysis algorithm can successfully be used for analysis of visual process measures in VR. Process measures not only can corroborate performance measures, but also can lead to discoveries of the reasons for performance improvements. In particular, analysis of users' eye movements in VR can potentially lead to further insights into the underlying cognitive processes of VR subjects.

Cognitive Feedback Training using 3D Binocular Eye Tracker

Studies in visual search have shown that feedback training can improve visual inspection performance (speed and accuracy), provided it is given in a timely and appropriate manner. Traditionally, performance feedback, i.e. information about the outcome, serves as the basis of most feedback training schemes. Other forms of feedback, which provide search strategy information, may have a role to play in improving inspection performance. This form of feedback is referred to as ‘cognitive feedback’. This paper describes the setup for collecting and analyzing eye movements using a 3D binocular eye tracker, which serves as a tool for providing cognitive feedback training. It also emphasizes the various technical issues related with eye tracker integration and further discusses the use of a new 3D-fixation recognition algorithm.

Lens induced aniso-accommodation.

Despite the evidence for consensual accommodation in response to consensual accommodative stimuli, only a few studies have investigated the binocular accommodative response to unequal (aniso) accommodative stimuli. Past studies investigating an unequal binocular accommodative response (aniso-accommodation) to aniso-accommodative stimuli have been limited by viewing conditions and measurement technique making the results, which were equivocal, difficult to interpret. This investigation addressed these limitations by the following design parameters: (1) monocular dichoptic blur cuese were provided in the binocular stimulus target to provide subjects feedback on their aniso-accommodative response and to alert the investigator of a monocular blur suppression response; (2) a training period was provided; (3) in the subjective method, each eye's stigma was positioned near the dichoptic letter viewed by the other eye. By this method, a true aniso-accommodative response could be differentiated from successive consensual responses; (4) a large range of aniso-accommodative stimuli was used, 0.50-3.0 D, presented in incremental steps of 0.5 D, allowing measurement of an average 0.75 D aniso-accommodative response for the highest (3.0 D) aniso-accommodative stimulus; (5) aniso-accommodation was measured as a function of viewing distance. For four of seven subjects, the gain of the aniso-accommodative response was significantly greater at near than at far viewing distances; (6) aniso-accommodation was confirmed objectively with measures of the response to steady state and step aniso-accommodative stimuli, using a binocular SRI Dual Purkinje Eye Tracker Optometer System. The aniso-accommodative response to step stimuli showed a very long latency period (about 11 s) and a response time of 4.5 s. A potential benefit of aniso-accommodation would be to overcome small amounts of uncorrected anisometropic refractive error. This would preserve fine stereo acuity which is impaired by unequal intraocular image contrast. Aniso-accommodation also may provide an appropriate efferent feedback signal for each eye's unique refractive error which could be used to guide developmental isometropization (attainment of equal refractive error in the two eyes.).

Evaluation of a video tracking device for measurement of horizontal and vertical eye rotations during locomotion

We have evaluated a video-based method for measuring binocular horizontal and vertical eye movements of human subjects by comparing it with the magnetic search coil technique. This video tracking system (VTS) uses multiple infrared light sources and small video cameras to simultaneously measure the positions of reflected corneal images and the center of the pupil. The system has a linear range of approximately ±40° horizontally and ±30° vertically, a sampling rate of 120 Hz (180 Hz with the head fixed), and system noise with standard deviation of < 0.04°. The binocular eye-tracking system is light-weight (190 g), being mounted on goggles that, with the eyes in primary position, permit a field of view of 60° horizontally and vertically. The VTS is insensitive to translations of the tracker relative to the eyes. By placing the video preprocessing unit on a cart, eye movements may be recorded while subjects walk through distances up to 100 feet. In comparison with the magnetic search coil technique, the VTS generally provides reliable measurements of horizontal and vertical eye position; eye velocity is noisier than corresponding coil signals, but superior to electro-oculography.

Real time binocular detection of horizontal vertical and torsional eye movements by an infra red video-eye tracker.

So far it has only been possible to detect horizontal and vertical eye movements during real time; torsional eye movements have to be calculated from video-tape that have been processed several times, and each investigation takes considerable time. We developed a real time binocular infra red video eye tracker for routine clinical examination which allows real time graphical visualization of horizontal, vertical and torsional eye movements with a bandwidth of 12 Hz (25 Hz refresh frequency). Eye movements in three dimensions can now be obtained as easy as has been done before with electronystagmography for the horizontal component. In contrast to previous systems described, we analyze only those parts of the video-frame that are essential for detection of all three eye movements. This results in a huge data-reduction accompanied by a speed-up of analysis. The black pupil centre is detected by an iterative process of analyzing several horizontal and vertical lines of the even video field. The algorithm searches for regions of minimal intensity. Horizontal and vertical eye rotation is then calculated and transformed into Fick's coordinates. Based upon the centre of the pupil four concentric segments are defined in the iris to detect torsion and to enable correction for false rotation. Torsion is detected by cross-correlation of each segment and estimated from the outcome of the correlation coefficients. All transformations are established by predefined stored transformation matrices thus making real time calculations for transformation unnecessary. After corrections for eccentricity and assuming that fixation of the subjects tested was perfect, accuracy appears to be within 0.3 degrees-1.0 degrees for all three dimensions. Linearity after all corrections is within 4-9% within a visual field of 60 x 40 degrees (horizontal x vertical).