Human Eye Movements Research Articles

Gazing at Games: An Introduction to Eye Tracking Control

Eye tracking is a process that identifies a specific point in both space and time that is being looked at by the observer. This information can also be used in real-time to control applications using the eyes. Recent innovations in the video game industry include alternative input modalities to provide an enhanced, more immersive user experience. In particular, eye gaze control has recently been explored as an input modality in video games. This book is an introduction for those interested in using eye tracking to control or analyze video games and virtual environments. Key concepts are illustrated through three case studies in which gaze control and voice recognition have been used in combination to control virtual characters and applications. The lessons learned in the case studies are presented and issues relating to incorporating eye tracking in interactive applications are discussed. The reader will be given an introduction to human visual attention, eye movements and eye tracking technologies. Previous work in the field of studying fixation behavior in games and using eye tracking for video game interaction will also be presented. The final chapter discusses ideas for how this field can be developed further to create richer interaction for characters and crowds in virtual environments. Alternative means of interaction in video games are especially important for disabled users for whom traditional techniques, such as mouse and keyboard, may be far from ideal. This book is also relevant for those wishing to use gaze control in applications other than games.

Active Vision: Fixational Eye Movements Help Seeing Space in Time

The significance of the miniature eye movements that we make during visual fixation has been intensely debated for the last 80 years. Recent studies have revealed that these motions of the eyes fulfill an important functional role: helping to extract useful information from natural scenes.

Biometric identification based on the eye movements and graph matching techniques

The last few years a growing research interest has aroused in the field of biometrics, concerning the use of brain dependent characteristics generally known as behavioral features. Human eyes, often referred as the gates to the soul, can possibly comprise a rich source of idiosyncratic information which may be used for the recognition of an individual’s identity. In this paper an innovative experiment and a novel processing approach for the human eye movements is implemented, ultimately aiming at the biometric segregation of individual persons. In our experiment, the subjects observe face images while their eye movements are being monitored, providing information about each participant’s attention spots. The implemented method treats eye trajectories as 2-D distributions of points on the image plane. The efficiency of graph objects in the representation of structural information motivated us on the utilization of a non-parametric multivariate graph-based measure for the comparison of eye movement signals, yielding promising results at the task of identification according to behavioral characteristics of an individual.

Human saccadic eye movements

Human saccadic eye movements

Large-scale functional brain networks in human non-rapid eye movement sleep: insights from combined electroencephalographic/functional magnetic resonance imaging studies

This paper reviews the existing body of knowledge on the neural correlates of spontaneous oscillations, functional connectivity and brain plasticity in human non-rapid eye movement (NREM) sleep. The first section reviews the evidence that specific sleep events as slow waves and spindles are associated with transient increases in regional brain activity. The second section describes the changes in functional connectivity during NREM sleep, with a particular focus on changes within a low-frequency, large-scale functional brain network. The third section will discuss the possibility that spontaneous oscillations and differential functional connectivity are related to brain plasticity and systems consolidation, with a particular focus on motor skill acquisition. Implications for the mode of information processing per sleep stage and future experimental studies are discussed.

A Control System of Binocular Eyes Based on DSP TMS320F2812

This paper mainly introduces a binocular eyes system which is similar to the human eyes. The whole control system is based on neural network control model of human eyes, while the structure adopts spherical parallel mechanism to mimic the human eye movement. The whole Binocular control system consists of DSP embedded control system and PC platforms, which achieves the movement of the vestibular ocular reflex, convergence and conjugate movement, thus realizing good tracking of moving targets. This control system has compacted configuration and high reliability. A large number of experiments were done to validate the whole excellent system performance.

The odd human eye movements during oddity search are not suboptimal

Studies have shown that when humans search for a known target, eye movements are often guided towards target features (e.g., Findlay, 1997) with occasional fixational compensations for inhomogeneous processing across the visual field (Najemnik & Giesler, 2005). In a different type of search (oddity search), targets and distractors are not known beforehand and observers have to determine whether an element in the display differs from others (distractors) along some feature. Previous research has shown that oddity search has different behavioral properties from target known search, such as different effects of set-size (Bravo & Nakayama, 1992; Schoonveld et al., 2007). Here, we measure human saccadic eye movements during an oddity search task and develop an optimal foveated oddity searcher, which fixates locations that maximize search accuracy, to evaluate human eye movement strategies. Observers performed a yes/no search task in which they were to determine whether one of the five potential target locations contained a singleton target. Stimuli consisted of Gabor patches at three orientations (5° left, vertical, 5° right) embedded in white noise at five locations around an imaginary circle (6° radius). For target present trials, the target was randomly selected from the three possible elements, while one of the remaining elements occupied the other four locations (distractors). For target absent trials, all five locations contained the same randomly selected element. Participants viewed each stimulus for 700 ms (allowing approximately three eye movements) before making a decision. Unlike typical strategies for target known search, observers adopted an eye movement strategy for the oddity search in which they systematically fixated three locations regardless of the stimulus configuration. Such strategy shows little guidance towards the target location. We find that the optimal foveated oddity searcher, like humans, seeks to survey as many of the potential target locations as possible and shows little target guidance.

Three‐dimensional kinematics of saccadic eye movements in humans: is the “half‐angle rule” obeyed?

The "half-angle rule" must be satisfied for Listing's law to be obeyed during saccades: the eye velocity axis must tilt away from Listing's plane by half the angle of eye position eccentricity from primary position. We aimed to determine if this rule is satisfied during saccades. Horizontal and vertical saccades were recorded using the search coil technique, at different elevations and azimuths, in five normal humans. In all, eye positions were located on a twisted rather than planar surface. The mean tilt-angle coefficients (TACs; ratio of angle of eye velocity axis tilt to angle of eye position eccentricity) were 0.57 and 0.34 for horizontal and vertical saccades, respectively. TACs were significantly lower for vertical saccades. Thus, Listing's law is not perfectly obeyed during saccades. We suggest that the discrepancy in horizontal and vertical TACs causes eye positions to lie on a twisted rather than planar surface in three-dimensional space.

Interplay between spontaneous and induced brain activity during human non-rapid eye movement sleep.

Humans are less responsive to the surrounding environment during sleep. However, the extent to which the human brain responds to external stimuli during sleep is uncertain. We used simultaneous EEG and functional MRI to characterize brain responses to tones during wakefulness and non-rapid eye movement (NREM) sleep. Sounds during wakefulness elicited responses in the thalamus and primary auditory cortex. These responses persisted in NREM sleep, except throughout spindles, during which they became less consistent. When sounds induced a K complex, activity in the auditory cortex was enhanced and responses in distant frontal areas were elicited, similar to the stereotypical pattern associated with slow oscillations. These data show that sound processing during NREM sleep is constrained by fundamental brain oscillatory modes (slow oscillations and spindles), which result in a complex interplay between spontaneous and induced brain activity. The distortion of sensory information at the thalamic level, especially during spindles, functionally isolates the cortex from the environment and might provide unique conditions favorable for off-line memory processing.

A computational model for human eye-movements in military simulations

Models of eye movements of an observer searching for human targets are helpful in developing accurate models of target acquisition times and false positive detections. We develop a new model describing the distribution of gaze positions for an observer which includes both bottom-up (salience) and top-down (task dependent) factors. We validate the combined model against a bottom-up model from the literature and against the bottom up and top down parts alone using human performance data on stationary targets. The new model is shown to be significantly better. The new model requires a large amount of data about the terrain and the target that is obtained directly from the 3D simulation through an automated process.

Bone conducted vibration activates the vestibulo-ocular reflex in the guinea pig

The aim of the study was: (a) to test whether short duration (6ms) 500Hz bone-conducted vibration (BCV) of the skull in alert head free guinea pigs would elicit eye movements; (b) to test whether these eye movements were vestibular in origin; and (c) to determine whether they corresponded to human eye movements to such stimuli. In this way we sought to establish the guinea pig as an acceptable model for testing the mechanism of the effect BCV on the vestibulo-ocular reflex. Consistent short-latency stimulus-locked responses to BCV were observed. The magnitude of eye displacement was directly related to stimulus intensity as recorded by accelerometers cemented onto the animal's skull. The strongest and most consistent response component was intorsion of both eyes. In lateral-eyed animals intorsion is produced by the combined contraction of the inferior rectus and superior oblique muscles. In humans the same pair of muscles acts to cause depression of the eye. To test whether the movements were vestibular we selectively ablated the vestibular endorgans: 3 of the 8 animals underwent a bilateral intratympanic injection of gentamicin, an ototoxic aminoglycoside antibiotic, to ablate their vestibular receptors. After ablation there was an overall reduction in the magnitude of eye displacement, as well as a reduction in the effectiveness of the BCV stimulus to elicit eye movements. The animals’ hearing, as measured by the threshold for auditory brainstem responses, remained unchanged after gentamicin, confirming that the cochlea was not affected. The reduced magnitude of responses after vestibular receptor ablation demonstrates that the eye-movement responses to BCV are probably caused by the stimulation of vestibular receptors, which in turn activate the vestibulo-ocular reflex.

An interdisciplinary overview and intelligent control of human prosthetic eye movements system for the emotional support by a huggable pet-type robot from a biomechatronical viewpoint

Despite active research and significant progress in the last three decades on control of human eye movements, it remains challenging issue due to its applications in prosthetic eyes and robotics. Till now, no considerable investigation of this subject is presented in the interdisciplinary sciences. The goal of this paper is to present a distinguished survey of existing literature on the intelligent control of the human eye movements system applied in a huggable pet-type robot as a biomechatronic system.In this study, the basic knowledge of human eye movements control is explained to show how the neural networks in the brainstem control the human eye movements. The geometry and model of human eye movements system are investigated and this system is considered as a nonlinear control system. The specified model may only be an academic exercise. It can have scientific importance in understanding of the human movement system in general. Also, it can be useful for robotics.Intelligent methods such as artificial neural networks and fuzzy neural networks are proposed to control the human eye movements and numerical simulations are presented. It is discussed that the intelligent controls applied to control of human eye movements system are emulated from the neural controls in biological system.

Theta-Burst Stimulation of the Cerebellum Interferes with Internal Representations of Sensory-Motor Information Related to Eye Movements in Humans

Continuous theta-burst stimulation (cTBS) applied over the cerebellum exerts long-lasting effects by modulating long-term synaptic plasticity, which is thought to be the basis of learning and behavioral adaptation. To investigate the impact of cTBS over the cerebellum on short-term sensory-motor memory, we recorded in two groups of eight healthy subject each the visually guided saccades (VGSs), the memory-guided saccades (MGSs), and the multiple memory-guided saccades (MMGSs), before and after cTBS (cTBS group) or simulated cTBS (control group). In the cTBS group, cTBS determined hypometria of contralateral centrifugal VGSs and worsened the accuracy of MMGS bilaterally. In the control group, no significant differences were found between the two recording sessions. These results indicate that cTBS over the cerebellum causes eye movement effects that last longer than the stimulus duration. The VGS contralateral hypometria suggested that we eventually inhibited the fastigial nucleus on the stimulated side. MMGSs in normal subjects have a better final accuracy with respect to MGSs. Such improvement is due to the availability in MMGSs of the efference copy of the initial reflexive saccade directed toward the same peripheral target, which provides a sensory-motor information that is memorized and then used to improve the accuracy of the subsequent volitional memory-guided saccade. Thus, we hypothesize that cTBS disrupted the capability of the cerebellum to make an internal representation of the memorized sensory-motor information to be used after a short interval for forward control of saccades.

The Influence of the Saccade Direction on the Direction of the Consecutive Saccade during Free Viewing

Motter and Belky (1997) analyzed monkey eye movements during search tasks. They took the relative directional headings for consecutive saccades and found a slight directional bias against saccades to areas between the previously fixated stimulus and the current fixation location. In the current research, an analysis of human eye movements during free viewing was made. Eight images of natural scene were tested with 118 subjects. The subject viewed every image freely for 10 sec. The relative directional headings for consecutive saccade were broken out of the data set and analyzed for directional biases. Saccade direction polar histograms average across subjects showed directional biases: a consecutive saccade took a straight line slightly more than a left or right turn, and it went backward definitely more than the other directions.

A Time Delay Neural Network model for simulating eye gaze data

Human eye movement modelling is a new, challenging and promising research topic in computer vision. Human eye movement modelling aims at simulating the scan path in which a human being views an image, a scene or a video. The successful modelling of human eye movements potentially benefits a wide range of applications such as image retrieval, image annotation, medical image diagnosis and human visual perception. This article presents a model based on a Time Delay Neural Network (TDNN) to simulate eye gaze data. First, 120 Hz eye gaze data are acquired by a non-intrusive table-mounted eye tracker. Our proposed model is to simulate the image reading process of a single subject. Seven features are then extracted based on the knowledge of the human oculomotor system and the image contents to train a TDNN. Finally, the trained TDNN combined with a saccade control mechanism is used to simulate the scan path of a human being viewing an image. The proposed model can generate 600 points of raw eye gaze data in a 5-second eye viewing window. Both subjective and objective methods are used to evaluate the model by comparing its behaviour and characteristics with the real eye gaze data collected from an eye tracker. Qualitative assessment shows that the subject can hardly tell the differences between the scan path from the model and that from a human being. By evaluating the coincident probability Cp and coincident significance Cs , quantitative assessment shows that the results from the TDNN model are reasonable and similar to human scan paths.

Wake deterioration and sleep restoration of human learning

Summary While the benefit of sleep after learning in offline consolidation is established [1], a role for sleep before learning in promoting initial memory formation remains largely uncharacterized. Existing theoretical frameworks speculate that accrued time awake, associated with ongoing experience, decreases learning capacity, while specific non-rapid-eye-movement (NREM) oscillations support restoration of learning ability [1,2]. Despite these model predictions, it remains untested whether episodic learning capacity remains stable across the day, or is progressively compromised by continued time awake. Furthermore, it is similarly unclear whether the presence, rather than the detrimental absence, of sleep restores efficient learning ability, and if so, what aspect(s) of sleep physiology support such reinstatement [3,4]. We have tested these related hypotheses, and report here a learning interaction, such that episodic encoding capacity deteriorates across a daytime waking interval, but sleep and associated NREM spindle oscillations restore efficient learning ability.

Operant control of human eye movements

Saccade and smooth pursuit are the eye movements used by primates to shift gaze. In this article we review evidence for the effects of reinforcement on several dimensions of these responses such as their latencies, velocities or amplitudes. We propose that these responses are operant behaviours controlled by their consequences on performance of visually guided tasks. Studying the conditions under which particular eye movement patterns might emerge from the cumulative effects of reinforcement provides critical insights about how motor responses are attuned to environmental exigencies.

An integrative, experience-based theory of attentional control

Although diverse, theories of visual attention generally share the notion that attention is controlled by some combination of three distinct strategies: (1) exogenous cuing from locally contrasting primitive visual features, such as abrupt onsets or color singletons (e.g., L. Itti, C. Koch, & E. Neiber, 1998), (2) endogenous gain modulation of exogenous activations, used to guide attention to task-relevant features (e.g., V. Navalpakkam & L. Itti, 2007; J. Wolfe, 1994, 2007), and (3) endogenous prediction of likely locations of interest, based on task and scene gist (e.g., A. Torralba, A. Oliva, M. Castelhano, & J. Henderson, 2006). However, little work has been done to synthesize these disparate theories. In this work, we propose a unifying conceptualization in which attention is controlled along two dimensions: the degree of task focus and the contextual scale of operation. Previously proposed strategies-and their combinations-can be viewed as instances of this one mechanism. Thus, this theory serves not as a replacement for existing models but as a means of bringing them into a coherent framework. We present an implementation of this theory and demonstrate its applicability to a wide range of attentional phenomena. The model accounts for key results in visual search with synthetic images and makes reasonable predictions for human eye movements in search tasks involving real-world images. In addition, the theory offers an unusual perspective on attention that places a fundamental emphasis on the role of experience and task-related knowledge.

Application of zebrafish oculomotor behavior to model human disorders

To ensure high acuity vision, eye movements have to be controlled with astonishing precision by the oculomotor system. Many human diseases can lead to abnormal eye movements, typically of the involuntary oscillatory eye movements type called nystagmus. Such nystagmus can be congenital (infantile) or acquired later in life. Although the resulting eye movements are well characterized, there is only little information about the underlying etiology. This is in part owing to the lack of appropriate animal models. In this review article, we describe how the zebrafish with its quick maturing visual system can be used to model oculomotor pathologies. We compare the characteristics and assessment of human and zebrafish eye movements. We describe the oculomotor properties of the zebrafish mutant belladonna, which has non-crossing optical fibers, and is a particularly informative model for human oculomotor deficits. This mutant displays a reverse optokinetic response, spontaneous oscillations that closely mimic human congenital nystagmus and abnormal motor behavior linked to circular vection.

Physically-based modeling and simulation of extraocular muscles

Dynamic simulation of human eye movements, with realistic physical models of extraocular muscles (EOMs), may greatly advance our understanding of the complexities of the oculomotor system and aid in treatment of visuomotor disorders. In this paper we describe the first three dimensional (3D) biomechanical model which can simulate the dynamics of ocular motility at interactive rates. We represent EOMs using “strands”, which are physical primitives that can model an EOM’s complex nonlinear anatomical and physiological properties. Contact between the EOMs, the globe, and orbital structures can be explicitly modeled. Several studies were performed to assess the validity and utility of the model. EOM deformation during smooth pursuit was simulated and compared with published experimental data; the model reproduces qualitative features of the observed nonuniformity. The model is able to reproduce realistic saccadic trajectories when the lateral rectus muscle was driven by published measurements of abducens neuron discharge. Finally, acute superior oblique palsy, a pathological condition, was simulated to further evaluate the system behavior; the predicted deviation patterns agree qualitatively with experimental observations. This example also demonstrates potential clinical applications of such a model.