Guided Eye Movements Research Articles

What are you looking at? The neural basis of eye movement guidance to objects

What are you looking at? The neural basis of eye movement guidance to objects

Knowing where to go: Spatial memory guides eye and body movements in a naturalistic visual search task.

Most research on visual search has used simple tasks presented on a computer screen. However, in natural situations visual search almost always involves eye, head, and body movements in a three-dimensional (3D) environment. The different constraints imposed by these two types of search tasks might explain some of the discrepancies in our understanding concerning the use of memory resources and the role of contextual objects during search. To explore this issue, we analyzed a visual search task performed in an immersive virtual reality apartment. Participants searched for a series of geometric 3D objects while eye movements and head coordinates were recorded. Participants explored the apartment to locate target objects whose location and visibility were manipulated. For objects with reliable locations, we found that repeated searches led to a decrease in search time and number of fixations and to a reduction of errors. Searching for those objects that had been visible in previous trials but were only tested at the end of the experiment was also easier than finding objects for the first time, indicating incidental learning of context. More importantly, we found that body movements showed changes that reflected memory for target location: trajectories were shorter and movement velocities were higher, but only for those objects that had been searched for multiple times. We conclude that memory of 3D space and target location is a critical component of visual search and also modifies movement kinematics. In natural search, memory is used to optimize movement control and reduce energetic costs.

Quantifying task-related gaze.

Competing theories attempt to explain what guides eye movements when exploring natural scenes: bottom-up image salience and top-down semantic salience. In one study, we apply language-based analyses to quantify the well-known observation that task influences gaze in natural scenes. Subjects viewed ten scenes as if they were performing one of two tasks. We found that the semantic similarity between the task and the labels of objects in the scenes captured the task-dependence of gaze (t(39) = 13.083; p < 0.001). In another study, we examined whether image salience or semantic salience better predicts gaze during a search task, and if viewing strategies are affected by searching for targets of high or low semantic relevance to the scene. Subjects searched 100 scenes for a high- or low-relevance object. We found that image salience becomes a worse predictor of gaze across successive fixations, while semantic salience remains a consistent predictor (X2(1, N=40) = 75.148, p < .001). Furthermore, we found that semantic salience decreased as object relevance decreased (t(39) = 2.304; p = .027). These results suggest that semantic salience is a useful predictor of gaze during task-related scene viewing, and that even in target-absent trials, gaze is modulated by the relevance of a search target to the scene in which it might be located.

Conscious and unconscious memory and eye movements in context-guided visual search: A computational and experimental reassessment of Ramey, Yonelinas, and Henderson (2019)

Are eye movements unconsciously guided towards target locations in familiar scenes? In a recent eyetracking study, Ramey, Yonelinas, and Henderson (2019) measured eye-movement efficiency (scanpath ratio) and memory judgments when participants searched for targets in repeated and novel scenes. When trials judged new with high confidence were selected, scanpath ratio was lower for old scenes (misses) than for new scenes (correct rejections). In addition, familiarity as measured by recognition confidence did not significantly predict scanpath ratio. Ramey et al. attributed these results to unconscious learning guiding eye movements. In a re-assessment of Ramey et al.’s data, we show that their findings can be accounted for by a single-system computational model in which eye movements and memory judgments are driven by a common latent memory representation. In particular, (a) the scanpath ratio difference between high-confidence misses and correct rejections is a consequence of regression to the mean, while (b) the null correlation between familiarity and scanpath ratio, partly a natural consequence of the low reliability of the scanpath ratio measure, is also reproduced by the model. Two pre-registered experiments confirm a novel prediction of the alternative single-system model. This work offers a parsimonious account of Ramey et al.’s findings without recourse to unconscious guidance of eye movements.

A 2D Synthesized Image Improves the 3D Search for Foveated Visual Systems.

Current medical imaging increasingly relies on 3D volumetric data making it difficult for radiologists to thoroughly search all regions of the volume. In some applications (e.g., Digital Breast Tomosynthesis), the volumetric data is typically paired with a synthesized 2D image (2D-S) generated from the corresponding 3D volume. We investigate how this image pairing affects the search for spatially large and small signals. Observers searched for these signals in 3D volumes, 2D-S images, and while viewing both. We hypothesize that lower spatial acuity in the observers' visual periphery hinders the search for the small signals in the 3D images. However, the inclusion of the 2D-S guides eye movements to suspicious locations, improving the observer's ability to find the signals in 3D. Behavioral results show that the 2D-S, used as an adjunct to the volumetric data, improves the localization and detection of the small (but not large) signal compared to 3D alone. There is a concomitant reduction in search errors as well. To understand this process at a computational level, we implement a Foveated Search Model (FSM) that executes human eye movements and then processes points in the image with varying spatial detail based on their eccentricity from fixations. The FSM predicts human performance for both signals and captures the reduction in search errors when the 2D-S supplements the 3D search. Our experimental and modeling results delineate the utility of 2D-S in 3D search-reduce the detrimental impact of low-resolution peripheral processing by guiding attention to regions of interest, effectively reducing errors.

Saccadic trajectories deviate toward or away from optimally informative visual features

The saccades' path is influenced by visual distractors, making their trajectory curve away or toward them. Previous research suggested that the more salient the distractor, the more pronounced is the curvature. We investigate the saliency of spatial visual features, predicted by a constrained maximum entropy model to be optimal or non-optimal information carriers in fast vision, by using them as distractors in a saccadic task. Their effect was compared to that of luminance-based control distractors. Optimal features evoke a larger saccadic curvature compared to non-optimal features, and the magnitude and direction of deviation change as a function of the delay between distractor and saccade onset. Effects were similar to those found with high-luminance versus low-luminance distractors. Therefore, model-predicted information optimal features interfere with target-oriented saccades, following a dynamic attraction-repulsion pattern. This suggests that the visuo-oculomotor system rapidly and automatically processes optimally informative features while programming visually guided eye movements.

Head and body cues guide eye movements and facilitate target search in real-world videos.

Static gaze cues presented in central vision result in observer shifts of covert attention and eye movements, and benefits in perceptual performance in the detection of simple targets. Less is known about how dynamic gazer behaviors with head and body motion influence search eye movements and performance in perceptual tasks in real-world scenes. Participants searched for a target person (yes/no task, 50% presence), whereas watching videos of one to three gazers looking at a designated person (50% valid gaze cue, looking at the target). To assess the contributions of different body parts, we digitally erase parts of the gazers in the videos to create three different body parts/whole conditions for gazers: floating heads (only head movements), headless bodies (only lower body movements), and the baseline condition with intact head and body. We show that valid dynamic gaze cues guided participants' eye movements (up to 3 fixations) closer to the target, speeded the time to foveate the target, reduced fixations to the gazers, and improved target detection. The effect of gaze cues in guiding eye movements to the search target was the smallest when the gazer's head was removed from the videos. To assess the inherent information about gaze goal location for each body parts/whole condition, we collected perceptual judgments estimating gaze goals by a separate group of observers with unlimited time. Observers' perceptual judgments showed larger estimate errors when the gazer's head was removed. This suggests that the reduced eye movement guidance from lower body cueing is related to observers' difficulty extracting gaze information without the presence of the head. Together, the study extends previous work by evaluating the impact of dynamic gazer behaviors on search with videos of real-world cluttered scenes.

Rapid Input-Output Transformation between Local Field Potential and Spiking Activity during Sensation but not Action in the Superior Colliculus.

Sensorimotor transformation is the sequential process of registering a sensory signal in the environment and then responding with the relevant movement at an appropriate time. For visually guided eye movements, neural signatures in the form of spiking activity of neurons have been extensively studied along the dorsoventral axis of the superior colliculus (SC). In contrast, the local field potential (LFP), which represents the putative input to a region, remains largely unexplored in the SC. We therefore compared amplitude levels and onset times of both spike bursts and LFP modulations recorded simultaneously with a laminar probe along the dorsoventral axis of SC in 3 male monkeys performing the visually guided delayed saccade task. Both signals displayed a gradual transition from sensory activity in the superficial layers to a predominantly motor response in the deeper layers, although the transition from principally sensory to mostly motor response occurred ∼500 μm deeper for the LFP. For the sensory response, LFP modulation preceded spike burst onset by <5 ms in the superficial and intermediate layers and only when data were analyzed on a trial-by-trial basis. The motor burst in the spiking activity led LFP modulation by >25 ms in the deeper layers. The results reveal a fast and efficient input-output transformation between LFP modulation and spike burst in the visually responsive layers activity during sensation but not during action. The spiking pattern observed during the movement phase is likely dominated by intracollicular processing that is not captured in the LFP.SIGNIFICANCE STATEMENT What is the sequence of events between local field potential (LFP) modulation and spiking activity during sensorimotor transformation? A trial-by-trial analysis reveals that the LFP activity leads the spike burst in the superficial and intermediate layers of the superior colliculus during visual processing, while both trial-by-trial and the average analyses show that the spike burst leads the LFP modulation during movement generation. These results suggest an almost instantaneous LFP input, spike burst output transformation in the visually responsive layers of the superior colliculus when registering the stimulus. In contrast, substantial intracollicular processing likely results in a saccade-related spike burst that leads LFP modulation.

Sensory input-dependent gain modulation of the optokinetic nystagmus by mid-infrared stimulation in pigeons.

Neuromodulation serves as a cornerstone for brain sciences and clinical applications. Recent reports suggest that mid-infrared stimulation (MIRS) causes non-thermal modulation of brain functions. Current understanding of its mechanism hampers the routine application of MIRS. Here, we examine how MIRS influences the sensorimotor transformation in awaking-behaving pigeons, from neuronal signals to behavior. We applied MIRS and electrical stimulation (ES) to the pretectal nucleus lentiformis mesencephali (nLM), an essential retinorecipient structure in the pretectum, and examined their influences on the optokinetic nystagmus, a visually guided eye movement. We found MIRS altered eye movements by modulating a specific gain depending on the strength of visual inputs, in a manner different than the effect of ES. Simultaneous extracellular recordings and stimulation showed that MIRS could either excite and inhibit the neuronal activity in the same pretectal neuron depending on its ongoing sensory responsiveness levels in awake-behaving animals. Computational simulations suggest that MIRS modulates the resonance of a carbonyl group of the potassium channel, critical to the action potential generation, altering neuronal responses to sensory inputs and as a consequence, guiding behavior. Our findings suggest that MIRS could be a promising approach toward modulating neuronal functions for brain research and treating neurological diseases.

The importance of the positional probability of word final (but not word initial) characters for word segmentation and identification in children and adults' natural Chinese reading.

Word spacing is important in guiding eye movements during spaced alphabetic reading. Chinese is unspaced and it remains unclear as to how Chinese readers segment and identify words in reading. We conducted two parallel experiments to investigate whether the positional probabilities of the initial and the final characters of a multicharacter word affected word segmentation and identification in Chinese reading. Two-character words were selected as targets. In Experiment 1, the initial character's positional probability was manipulated as being either high or low, and the final character was kept identical across the two conditions. In Experiment 2, an analogous manipulation was made for the final character of the target word. We recorded adults' and children's eye movements when they read sentences containing these words. In Experiment 1, reading times on targets did not differ in the two conditions for both children and adults, providing no evidence that a word initial character's positional probability contributes to word segmentation. In Experiment 2, adults had shorter reading times and made fewer refixations on targets that comprised final characters with high relative to low positional probabilities; a similar effect was observed in children, but this effect had a slower time course. The results demonstrate that the positional probability of the final (but not the initial) character of a word influences segmentation commitments in reading. It suggests that Chinese readers identify where a currently fixated word ends, and via this commitment, by default, they identify where the subsequent word begins. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Decoding the Time Course of Spatial Information from Spiking and Local Field Potential Activities in the Superior Colliculus.

Place code representation is ubiquitous in circuits that encode spatial parameters. For visually guided eye movements, neurons in many brain regions emit spikes when a stimulus is presented in their receptive fields and/or when a movement is directed into their movement fields. Crucially, individual neurons respond for a broad range of directions or eccentricities away from the optimal vector, making it difficult to decode the stimulus location or the saccade vector from each cell's activity. We investigated whether it is possible to decode the spatial parameter with a population-level analysis, even when the optimal vectors are similar across neurons. Spiking activity and local field potentials (LFPs) in the superior colliculus (SC) were recorded with a laminar probe as monkeys performed a delayed saccade task to one of eight targets radially equidistant in direction. A classifier was applied offline to decode the spatial configuration as the trial progresses from sensation to action. For spiking activity, decoding performance across all eight directions was highest during the visual and motor epochs and lower but well above chance during the delay period. Classification performance followed a similar pattern for LFP activity too, except the performance during the delay period was limited mostly to the preferred direction. Increasing the number of neurons in the population consistently increased classifier performance for both modalities. Overall, this study demonstrates the power of population activity for decoding spatial information not possible from individual neurons.

Assessment of dynamic corneal nerve changes using static landmarks by in vivo large-area confocal microscopy-a longitudinal proof-of-concept study.

BackgroundThe purpose of the present proof-of-concept study was to use large-area in vivo confocal laser scanning microscopy (CLSM) mosaics to determine the migration rates of nerve branching points in the human corneal subbasal nerve plexus (SNP).MethodsThree healthy individuals were examined roughly weekly over a total period of six weeks by large-area in vivo confocal microscopy of the central cornea. An in-house developed prototype system for guided eye movement with an acquisition time of 40 s was used to image and generate large-area mosaics of the SNP. Kobayashi-structures and nerve entry points (EPs) were used as fixed structures to enable precise mosaic registration over time. The migration rate of 10 prominent nerve fiber branching points per participant was tracked and quantified over the longitudinal period.ResultsTotal investigation times of 10 minutes maximum per participant were used to generate mosaic images with an average size of 3.61 mm2 (range: 3.18–4.42 mm2). Overall mean branching point migration rates of (46.4±14.3), (48.8±15.5), and (50.9±13.9) µm/week were found for the three participants with no statistically significant difference. Longitudinal analyses of nerve branching point migration over time revealed significant time-dependent changes in migration rate only in participant 3 between the last two measurements [(63.7±12.3) and (43.0±12.5) µm/week, P<0.01]. Considering individual branching point dynamics, significant differences in nerve migration rate from the mean were only found in a few exceptions.ConclusionsThe results of this proof-of-concept study have demonstrated the feasibility of using in vivo confocal microscopy to study the migration rates of corneal subbasal nerves within large areas of the central human cornea (>1 mm2). The ability to monitor dynamic changes in the SNP opens a window to future studies of corneal nerve health and regenerative capacity in a number of systemic and ocular diseases. Since corneal nerves are considered part of the peripheral nervous system, this technique could also offer an objective diagnostic tool and biomarker for disease- or treatment-induced neuropathic changes.

A conflict between spatial selection and evidence accumulation in area LIP

The lateral intraparietal area (LIP) contains spatially selective neurons that help guide eye movements and, according to numerous studies, do so by accumulating sensory evidence in favor of one choice (e.g., look left) or another (look right). To examine this functional link, we trained two monkeys on an urgent motion discrimination task, a task with which the evolution of both the recorded neuronal activity and the subject’s choice can be tracked millisecond by millisecond. We found that while choice accuracy increased steeply with increasing sensory evidence, at the same time, the LIP selection signal became progressively weaker, as if it hindered performance. This effect was consistent with the transient deployment of spatial attention to disparate locations away from the relevant sensory cue. The results demonstrate that spatial selection in LIP is dissociable from, and may even conflict with, evidence accumulation during informed saccadic choices.

When the eyes have it and when not: How multiple sources of activation combine to guide eye movements during multiattribute decision making.

Memory plays a major but underexplored role in judgment and decision making (JDM). Studying eye movements-especially how people look at empty spatial locations when retrieving from memory information previously associated with those locations-provides useful information about how memory influences JDM. This so-called looking-at-nothing behavior is thought to reflect memory-driven allocation of attention. However, eye movements are also guided toward salient visual stimuli, such as test items presented on a screen. It is unclear how these multiple sources of activation combine to guide looking-at-nothing in JDM. We investigated this question in two experiments in which participants solved multiattribute categorization tasks using an exemplar-based decision strategy. In the first experiment, we tested how the occurrence and the strength of looking-at-nothing vary with the presentation format and the amount of training participants received. Looking-at-nothing occurred during categorizations when test-item information was presented auditorily and visually, but for the latter only after visual information was removed from the screen. It occurred both when training items were learned by heart and when they were presented 10 times on the screen. A second experiment revealed that an explicit instruction to imagine retrieval-relevant information during categorizations increased looking-at-nothing but did not change the decision-making process. The results shed light on the interaction between eye movements and attention to information in memory during JDM that can be explained in light of a shared priority map in memory. A detailed understanding of this interaction forms the basis for using eye movements to study memory processes in JDM. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Narrative Comprehension Guides Eye Movements in the Absence of Motion.

Viewers' attentional selection while looking at scenes is affected by both top-down and bottom-up factors. However, when watching film, viewers typically attend to the movie similarly irrespective of top-down factors-a phenomenon we call the tyranny of film. A key difference between still pictures and film is that film contains motion, which is a strong attractor of attention and highly predictive of gaze during film viewing. The goal of the present study was to test if the tyranny of film is driven by motion. To do this, we created a slideshow presentation of the opening scene of Touch of Evil. Context condition participants watched the full slideshow. No-context condition participants did not see the opening portion of the scene, which showed someone placing a time bomb into the trunk of a car. In prior research, we showed that despite producing very different understandings of the clip, this manipulation did not affect viewers' attention (i.e., the tyranny of film), as both context and no-context participants were equally likely to fixate on the car with the bomb when the scene was presented as a film. The current study found that when the scene was shown as a slideshow, the context manipulation produced differences in attentional selection (i.e., it attenuated attentional synchrony). We discuss these results in the context of the Scene Perception and Event Comprehension Theory, which specifies the relationship between event comprehension and attentional selection in the context of visual narratives.

The Beijing Sentence Corpus: A Chinese sentence corpus with eye movement data and predictability norms.

This report introduces the Beijing Sentence Corpus (BSC). This is a Chinese sentence corpus of eye-tracking data with relatively clear word boundaries. In addition, we report predictability norms for each word in the corpus. Eye movement corpora are available in alphabetic scripts such as English, German, and French. However, there is no publicly available corpus for Chinese. Thus, to study predictive processes during reading in Chinese, it is necessary to establish such a corpus. Also, given the clear word boundaries in the sentences, BSC is especially useful to provide evidence relevant to the theoretical debate of saccade target selection in Chinese. With the large-scale predictability norms, we conducted new analyses based on 60 BSC readers, testing the influences of launch word and target word properties while controlling for visual and oculomotor constraints, as well as sentence and subject-level individual differences. We discuss implications for guidance of eye movements in Chinese reading.

Trunk and head displacements stabilized to perform both horizontal and vertical saccadic eye movements.

Vision is crucial for humans to interact with their surrounding environment, and postural sway is reduced to allow short eye movements. However, the extent of subtle changes in postural control for horizontal and vertical eye movements remains unclear. The goal of this study was to investigate the effects of vertical and horizontal eye movements on head and trunk control in young adults. Fifteen healthy adults (23.4 ± 4.7years) stood upright in three conditions for 60s: fixation, horizontal, and vertical guided eye movements. In fixation, participants had to fixate on a stationary target. In both the horizontal and vertical eye movements, the target was presented with a frequency of 0.5Hz and a visual angle of 11°. Eye displacement was monitored using a SMI eye tracker (ETG2.0) and trunk and head sway were monitored using infrared markers (Optotrak 3020, NDI). The mean sway amplitude was lower in both directions for eye movements and lowest in the vertical direction compared to the fixation condition. The sway area was also lower in vertical eye movement than in the fixation condition. We also found that the sway reduction was greater at head than at trunk level. The median frequency sway in the anterior-posterior direction was higher in both eye movements than in fixation. Based upon these results, we suggest that to perform short eye movements, postural sway is more strongly controlled at the head level than at the trunk and in vertical eye movements than in horizontal movements.

Supplemental Material for When the Eyes Have It and When Not: How Multiple Sources of Activation Combine to Guide Eye Movements During Multiattribute Decision Making

Supplemental Material for When the Eyes Have It and When Not: How Multiple Sources of Activation Combine to Guide Eye Movements During Multiattribute Decision Making

Cortical Thickness Related to Compensatory Viewing Strategies in Patients With Macular Degeneration.

Retinal diseases like age-related macular degeneration (AMD) or hereditary juvenile macular dystrophies (JMD) lead to a loss of central vision. Many patients compensate for this loss with a pseudo fovea in the intact peripheral retina, the so-called “preferred retinal locus” (PRL). How extensive eccentric viewing associated with central vision loss (CVL) affects brain structures responsible for visual perception and visually guided eye movements remains unknown. CVL results in a reduction of cortical gray matter in the “lesion projection zone” (LPZ) in early visual cortex, but the thickness of primary visual cortex appears to be largely preserved for eccentric-field representations. Here we explore how eccentric viewing strategies are related to cortical thickness (CT) measures in early visual cortex and in brain areas involved in the control of eye movements (frontal eye fields, FEF, supplementary eye fields, SEF, and premotor eye fields, PEF). We determined the projection zones (regions of interest, ROIs) of the PRL and of an equally peripheral area in the opposite hemifield (OppPRL) in early visual cortex (V1 and V2) in 32 patients with MD and 32 age-matched controls (19–84 years) by functional magnetic resonance imaging. Subsequently, we calculated the CT in these ROIs and compared it between PRL and OppPRL as well as between groups. Additionally, we examined the CT of FEF, SEF, and PEF and correlated it with behavioral measures like reading speed and eccentric fixation stability at the PRL. We found a significant difference between PRL and OppPRL projection zones in V1 with increased CT at the PRL, that was more pronounced in the patients, but also visible in the controls. Although the mean CT of the eye fields did not differ significantly between patients and controls, we found a trend to a positive correlation between CT in the right FEF and SEF and fixation stability in the whole patient group and between CT in the right PEF and reading speed in the JMD subgroup. The results indicate a possible association between the compensatory strategies used by patients with CVL and structural brain properties in early visual cortex and cortical eye fields.

Scene meaningfulness guides eye movements even during mind-wandering.

During scene viewing, semantic information in the scene has been shown to play a dominant role in guiding fixations compared to visual salience (e.g., Henderson & Hayes, 2017). However, scene viewing is sometimes disrupted by cognitive processes unrelated to the scene. For example, viewers sometimes engage in mind-wandering, or having thoughts unrelated to the current task. How do meaning and visual salience account for fixation allocation when the viewer is mind-wandering, and does it differ from when the viewer is on-task? We asked participants to study a series of real-world scenes in preparation for a later memory test. Thought probes occasionally occurred after a subset of scenes to assess whether participants were on-task or mind-wandering. We used salience maps (Graph-Based Visual Saliency; Harel, Koch, & Perona, 2007) and meaning maps (Henderson & Hayes, 2017) to represent the distribution of visual salience and semantic richness in the scene, respectively. Because visual salience and meaning were represented similarly, we could directly compare how well they predicted fixation allocation. Our results indicate that fixations prioritized meaningful over visually salient regions in the scene during mind-wandering just as during attentive viewing. These results held across the entire viewing time. A re-analysis of an independent study (Krasich, Huffman, Faber, & Brockmole Journal of Vision, 20(9), 10, 2020) showed similar results. Therefore, viewers appear to prioritize meaningful regions over visually salient regions in real-world scenes even during mind-wandering.