Sequences Of Saccades Research Articles

Dynamics of Eye-Position Signals in the Dorsal Visual System

Many visual areas of the primate brain contain signals related to the current position of the eyes in the orbit. These cortical eye-position signals are thought to underlie the transformation of retinal input-which changes with every eye movement-into a stable representation of visual space. For this coding scheme to work, such signals would need to be updated fast enough to keep up with the eye during normal exploratory behavior. We examined the dynamics of cortical eye-position signals in four dorsal visual areas of the macaque brain: the lateral and ventral intraparietal areas (LIP; VIP), the middle temporal area (MT), and the medial-superior temporal area (MST). We recorded extracellular activity of single neurons while the animal performed sequences of fixations and saccades in darkness. The data show that eye-position signals are updated predictively, such that the representation shifts in the direction of a saccade prior to (<100 ms) the actual eye movement. Despite this early start, eye-position signals remain inaccurate until shortly after (10-150 ms) the eye movement. By using simulated behavioral experiments, we show that this brief misrepresentation of eye position provides a neural explanation for the psychophysical phenomenon of perisaccadic mislocalization, in which observers misperceive the positions of visual targets flashed around the time of saccadic eye movements. Together, these results suggest that eye-position signals in the dorsal visual system are updated rapidly across eye movements and play a direct role in perceptual localization, even when they are erroneous.

The Neural Basis of Parallel Saccade Programming: An fMRI Study

The neural basis of parallel saccade programming was examined in an event-related fMRI study using a variation of the double-step saccade paradigm. Two double-step conditions were used: one enabled the second saccade to be partially programmed in parallel with the first saccade while in a second condition both saccades had to be prepared serially. The intersaccadic interval, observed in the parallel programming (PP) condition, was significantly reduced compared with latency in the serial programming (SP) condition and also to the latency of single saccades in control conditions. The fMRI analysis revealed greater activity (BOLD response) in the frontal and parietal eye fields for the PP condition compared with the SP double-step condition and when compared with the single-saccade control conditions. By contrast, activity in the supplementary eye fields was greater for the double-step condition than the single-step condition but did not distinguish between the PP and SP requirements. The role of the frontal eye fields in PP may be related to the advanced temporal preparation and increased salience of the second saccade goal that may mediate activity in other downstream structures, such as the superior colliculus. The parietal lobes may be involved in the preparation for spatial remapping, which is required in double-step conditions. The supplementary eye fields appear to have a more general role in planning saccade sequences that may be related to error monitoring and the control over the execution of the correct sequence of responses.

A neural model of sequential movement planning and control of eye movements: Item-Order-Rank working memory and saccade selection by the supplementary eye fields

How does working memory store multiple spatial positions to control sequences of eye movements, particularly when the same items repeat at multiple list positions, or ranks, during the sequence? An Item-Order-Rank model of working memory shows how rank-selective representations enable storage and recall of items that repeat at arbitrary list positions. Rank-related activity has been observed in many areas including the posterior parietal cortices (PPC), prefrontal cortices (PFC) and supplementary eye fields (SEF). The model shows how rank information, originating in PPC, may support rank-sensitive PFC working memory representations and how SEF may select saccades stored in working memory. It also proposes how SEF may interact with downstream regions such as the frontal eye fields (FEF) during memory-guided sequential saccade tasks, and how the basal ganglia (BG) may control the flow of information. Model simulations reproduce behavioral, anatomical and electrophysiological data under multiple experimental paradigms, including visually- and memory-guided single and sequential saccade tasks. Simulations reproduce behavioral data during two SEF microstimulation paradigms, showing that their seemingly inconsistent findings about saccade latency can be reconciled.

Trans-saccadic processing of visual and motor planning during sequential eye movements

How the brain maintains perceptual continuity across eye movements that yield discontinuous snapshots of the world is still poorly understood. In this study, we adapted a framework from the dual-task paradigm, well suited to reveal bottlenecks in mental processing, to study how information is processed across sequential saccades. The pattern of RTs allowed us to distinguish among three forms of trans-saccadic processing (no trans-saccadic processing, trans-saccadic visual processing and trans-saccadic visual processing and saccade planning models). Using a cued double-step saccade task, we show that even though saccade execution is a processing bottleneck, limiting access to incoming visual information, partial visual and motor processing that occur prior to saccade execution is used to guide the next eye movement. These results provide insights into how the oculomotor system is designed to process information across multiple fixations that occur during natural scanning.

Critical role of cerebellar fastigial nucleus in programming sequences of saccades

The cerebellum plays an important role in programming accurate saccades. Cerebellar lesions affecting the ocular motor region of the fastigial nucleus (FOR) cause saccadic hypermetria; however, if a second target is presented before a saccade can be initiated (double-step paradigm), saccade hypermetria may be decreased. We tested the hypothesis that the cerebellum, especially FOR, plays a pivotal role in programming sequences of saccades. We studied patients with saccadic hypermetria because of either genetic cerebellar ataxia or surgical lesions affecting FOR and confirmed that the gain of initial saccades made to double-step stimuli was reduced compared with the gain of saccades to single target jumps. Based on measurements of the intersaccadic interval, we found that the ability to perform parallel processing of saccades was reduced or absent in all of our patients with cerebellar disease. Our results support the crucial role of the cerebellum, especially FOR, in programming sequences of saccades.

LINEAR VISUOMOTOR TRANSFORMATIONS IN MIDBRAIN SUPERIOR COLLICULUS CONTROL SACCADIC EYE-MOVEMENTS

It is well established that a localized population of neurons in the motor map of the midbrain superior colliculus (SC) drives a saccadic eye-head gaze shift. However, there is controversy as to how the brainstem saccade burst generators decode the SC activity. We focus on eye-movement generation by comparing two competing schemes from the recent literature that are both supported by neurophysiological evidence: the vector-averaging scheme versus the vector summation model. Whereas the former contains at least four nonlinearities to explain visuomotor planning and saccade execution, the latter relies predominantly on linear operations. We have demonstrated that the summation model accounts for the nonlinear main sequence of saccade kinematics, and predicted that this results from a spatial gradient in temporal burst profiles of SC cells: rostral cells have higher peak-firing rates and shorter burst durations than caudal cells. Yet, the number of spikes in their saccade-related bursts is identical. In contrast, the averaging model does not predict such activity profiles. We now also show that by incorporating the concept of predictive remapping in the spatial updating of saccade sequences, the phenomenon of target averaging in double-stimulation experiments, and the occurrence of goal-directed, but highly curved saccades in the double-step paradigm, can all be explained by the same linear summation mechanism. We argue that the linear model is more in line with neurophysiological data, while relying on fewer ad-hoc assumptions than the nonlinear vector-averaging scheme.

Optimal and Suboptimal Use of Postsaccadic Vision in Sequences of Saccades

Saccades are imprecise, due to sensory and motor noise. To avoid an accumulation of errors during sequences of saccades, a prediction derived from the efference copy can be combined with the reafferent visual feedback to adjust the following eye movement. By varying the information quantity of the visual feedback, we investigated how the reliability of the visual information affects the postsaccadic update in humans. Two elements of the visual scene were manipulated, the saccade target or the background, presented either together or in isolation. We determined the weight of the postsaccadic visual information by measuring the effect of intrasaccadic visual shifts on the following saccade. We confirmed that the weight of visual information evolves with information quantity as predicted for a statistically optimal system. In particular, we found that the visual background alone can guide the postsaccadic update, and that information from target and background are optimally combined. Moreover, these visual weights are adjusted dynamically and on a trial-to-trial basis to the level of visual noise determined by target eccentricity and reaction time. In contrast, we uncovered a dissociation between the visual signals used to update the next planned saccade (main saccade) and those used to generate an involuntary corrective saccade. The latter was exclusively based on visual information about the target, and discarded all information about the background: a suboptimal use of visual evidence.

Cognitive Impact of Eye Movements in Picture Viewing

We compared and analyzed eye tracking experiments to test the hypothesis that the patterns generated during picture viewing were mainly determined by eye movement generated links. Eye movement data were collected from participants who viewed photographs of artistic portraits during an active viewing. The links as determined by a popular evolutionary computational model, based on Genetic Algorithm, did not anticipate saccade sequences of eye movements. The optimized Genetic Algorithm (GA)-generated graphs differed considerably from the graphs generated by eye tracking. The existing evidence is consistent with the hypothesis that cognitive and, in turn, metacognitive factors take part in active gazing of eyes. The pattern generated by eye movements can only be substantiated with cognitive processes.

Guiding Binocular Saccades during Reading: A TMS Study of the PPC

Reading is an activity based on complex sequences of binocular saccades and fixations. During saccades, the eyes do not move together perfectly: saccades could end with a misalignment, compromising fused vision. During fixations, small disconjugate drift can partly reduce this misalignment. We hypothesized that maintaining eye alignment during reading involves active monitoring from posterior parietal cortex (PPC); this goes against traditional views considering only downstream binocular control. Nine young adults read a text; transcranial magnetic stimulation (TMS) was applied over the PPC every 5 ± 0.2 s. Eye movements were recorded binocularly with Eyelink II. Stimulation had three major effects: (1) disturbance of eye alignment during fixation; (2) increase of saccade disconjugacy leading to eye misalignment; (3) decrease of eye alignment reduction during fixation drift. The effects depend on the side; the right PPC was more involved in maintaining alignment over the motor sequence. Thus, the PPC is actively involved in the control of binocular eye alignment during reading, allowing clear vision. Cortical activation during reading is related to linguistic processes and motor control per se. The study might be of interest for the understanding of deficits of binocular coordination, encountered in several populations, e.g., in children with dyslexia.

Eye movement sequence generation in humans: Motor or goal updating?

Saccadic eye movements are often grouped in pre-programmed sequences. The mechanism underlying the generation of each saccade in a sequence is currently poorly understood. Broadly speaking, two alternative schemes are possible: first, after each saccade the retinotopic location of the next target could be estimated, and an appropriate saccade could be generated. We call this the goal updating hypothesis. Alternatively, multiple motor plans could be pre-computed, and they could then be updated after each movement. We call this the motor updating hypothesis. We used McLaughlin's intra-saccadic step paradigm to artificially create a condition under which these two hypotheses make discriminable predictions. We found that in human subjects, when sequences of two saccades are planned, the motor updating hypothesis predicts the landing position of the second saccade in two-saccade sequences much better than the goal updating hypothesis. This finding suggests that the human saccadic system is capable of executing sequences of saccades to multiple targets by planning multiple motor commands, which are then updated by serial subtraction of ongoing motor output.

Optimal habits can develop spontaneously through sensitivity to local cost

Habits and rituals are expressed universally across animal species. These behaviors are advantageous in allowing sequential behaviors to be performed without cognitive overload, and appear to rely on neural circuits that are relatively benign but vulnerable to takeover by extreme contexts, neuropsychiatric sequelae, and processes leading to addiction. Reinforcement learning (RL) is thought to underlie the formation of optimal habits. However, this theoretic formulation has principally been tested experimentally in simple stimulus-response tasks with relatively few available responses. We asked whether RL could also account for the emergence of habitual action sequences in realistically complex situations in which no repetitive stimulus-response links were present and in which many response options were present. We exposed naïve macaque monkeys to such experimental conditions by introducing a unique free saccade scan task. Despite the highly uncertain conditions and no instruction, the monkeys developed a succession of stereotypical, self-chosen saccade sequence patterns. Remarkably, these continued to morph for months, long after session-averaged reward and cost (eye movement distance) reached asymptote. Prima facie, these continued behavioral changes appeared to challenge RL. However, trial-by-trial analysis showed that pattern changes on adjacent trials were predicted by lowered cost, and RL simulations that reduced the cost reproduced the monkeys' behavior. Ultimately, the patterns settled into stereotypical saccade sequences that minimized the cost of obtaining the reward on average. These findings suggest that brain mechanisms underlying the emergence of habits, and perhaps unwanted repetitive behaviors in clinical disorders, could follow RL algorithms capturing extremely local explore/exploit tradeoffs.

Fitts’s Law and speed/accuracy trade-offs during sequences of saccades: Implications for strategies of saccadic planning

Strategies of saccadic planning must take into account both the required level of accuracy of the saccades, and the time and resources needed to plan and execute the movements. To determine relationships between accuracy and time, we studied sequences of saccades made to scan a set of stationary targets located at the corners of an imaginary square. Target separation and size varied. The time taken to complete saccadic sequences increased with the required level of precision, in agreement with the classical Fitts’s Law (1954) relationship. This was mainly due to the use of error-correcting secondary saccades, whose frequency increased with target separation and decreased with target size. Increases in the time spent fixating near each target did not increase the accuracy of the next primary saccade in the sequence. Instead, secondary saccades were the principal means of correcting landing errors of primary saccades. The results are consistent with a scanning strategy that discourages careful planning of individual saccades in favor of increasing the rate of saccadic production (i.e., exploration), using secondary saccades as needed to correct saccadic landing errors.

ScanMatch: A novel method for comparing fixation sequences

We present a novel approach to comparing saccadic eye movement sequences based on the Needleman-Wunsch algorithm used in bioinformatics to compare DNA sequences. In the proposed method, the saccade sequence is spatially and temporally binned and then recoded to create a sequence of letters that retains fixation location, time, and order information. The comparison of two letter sequences is made by maximizing the similarity score computed from a substitution matrix that provides the score for all letter pair substitutions and a penalty gap. The substitution matrix provides a meaningful link between each location coded by the individual letters. This link could be distance but could also encode any useful dimension, including perceptual or semantic space. We show, by using synthetic and behavioral data, the benefits of this method over existing methods. The ScanMatch toolbox for MATLAB is freely available online (www.scanmatch.co.uk).

Fitts's Law and the optimal planning of sequences of saccades

Fitts's Law and the optimal planning of sequences of saccades

Attentional enhancement along the path of a sequence of saccades

One of the central questions concerning the role of attention in saccadic control is the relationship between the selective filter that determines the effective target of a saccade and the attentional filter that serves perception. Results from several studies employing dual-task methods have show superior perceptual performance at the location that contains the target of a saccade, implying that a single attentional filter is used both by saccadic and perceptual systems (Godijn & Theeuwes, 2003; Kowler, Anderson, Dosher, & Blaser, 1995). Recently, Gersch, Kowler, and Dosher (2004) examined the links between attention and saccades executed as part of repetitive sequences of several eye movements, rather than as single eye movements programmed in isolation. Perceptual performance, which was tested during the pauses between saccades, was better at the target of the next saccade in the sequence than at other extrafoveal locations. Locations that were targets of any subsequent saccades in the sequence showed no perceptual enhancement, and thus were not treated differently by the perceptual system than locations that never were to be fixated at all. Linking extrafoveal visual attention exclusively to the goal of the next saccade would seem to place a severe limit on the ability to sample information across the visual field. Such a state of affairs would imply

Latency/accuracy trade-offs during sequences of saccades

Latency/accuracy trade-offs during sequences of saccades

Error in localising a target that is flashed near the time of an isolated saccade is not identical to the error found near the time of the last of a sequence of saccades

Error in localising a target that is flashed near the time of an isolated saccade is not identical to the error found near the time of the last of a sequence of saccades

Dynamic allocation of visual attention during the execution of sequences of saccades

Laboratory tasks used to study vision and attention usually require steady fixation, while natural visual processing occurs during the brief pauses between successive saccades. We studied vision and attentional allocation during intersaccadic pauses as subjects made repetitive sequences of saccades. Displays contained six outline squares located along the perimeter of an imaginary circle (diam 4� ). Saccades were made in sequence to every other square. The visual task was to identify the orientation (2AFC) of a Gabor test stimulus that appeared briefly (91 ms) along with superimposed noise in one of the squares during a randomly selected intersaccadic pause. Gabor location was cued in advance and noise frames were presented in all squares. Contrast thresholds during intersaccadic pauses were as much as 2–3 times higher than during steady fixation with comparable cueing. Thresholds improved over time during the intersaccadic pause, and the lowest extrafoveal thresholds (statistically indistinguishable from those at the same locations during steady fixation) were found for the location that was to be the target of the next saccade in the sequence. These results show that vision during intersaccadic pauses varies over space and time due to changes in the distribution of attention, as well as to visual suppression that may be related to the execution of the saccades themselves. Generation of sequences of accurate saccades encouraged a strategy of attentional allocation in which resources were dedicated primarily to the goal of the next saccade, leaving little attention for processing objects at other locations. � 2004 Elsevier Ltd. All rights reserved.

A theory of gaze behavior during real-world visual search

At the recent Psychonomics meeting, I introduced a computational model capable of describing the spatial coordinates of eye movements made during visual search. This model uses filter-based image processing techniques to represent real-world targets and search displays, then compares these representations to derive a search saliency map. The target of a simulated saccade is determined by the weighted centroid of activity on this map, with this centroid changing over time as a moving threshold removes those saliency map points offering the least evidence for the target. As a result of this moving threshold pruning points from the saliency map, a sequence of eye movements are produced that bring simulated gaze to the map's “hotspot”. Saccade programming is further constrained by a simulated fovea that retinally transforms the search display as the model's “eye” converges on the target. The model terminates with a target present response if any point on the saliency map exceeds a target present threshold. The model ends with a target-absent response if all saliency map activity is removed by the moving threshold without a target being detected. Current work extends this model to include an account of individual fixation durations. Temporal dynamics are produced by preventing the model from making an eye movement until a criterion distance is reached between the current fixation point and the centroid. Fixation duration is defined by the number of threshold movements needed to achieve this criterion distance. I test these model assumptions by monitoring the eye movements of observers viewing the same search displays input to the model, then comparing the simulated sequences of saccades and fixations to the behavioral data. Preliminary findings reveal considerable spatio-temporal agreement between these gaze patterns, both at an aggregate level (e.g., general tradeoffs between saccade latency and accuracy) as well as in the behavior of individual observers.

Voluntary Control of Multisaccade Gaze Shifts During Movement Preparation and Execution

Although the nature of gaze control regulating single saccades is relatively well documented, how such control is implemented to regulate multisaccade gaze shifts is not known. We used highly eccentric targets to elicit multisaccade gaze shifts and tested the ability of subjects to control the saccade sequence by presenting a second target on random trials. Their response allowed us to test the nature of control at many levels: before, during, and between saccades. Although the saccade sequence could be inhibited before it began, we observed clear signs of truncation of the first saccade, which confirmed that it could be inhibited in midflight as well. Using a race model that explains the control of single saccades, we estimated that it took about 100 ms to inhibit a planned saccade but took about 150 ms to inhibit a saccade during its execution. Although the time taken to inhibit was different, the high subject-wise correlation suggests a unitary inhibitory control acting at different levels in the oculomotor system. We also frequently observed responses that consisted of hypometric initial saccades, followed by secondary saccades to the initial target. Given the estimates of the inhibitory process provided by the model that also took into account the variances of the processes as well, the secondary saccades (average latency approximately 215 ms) should have been inhibited. Failure to inhibit the secondary saccade suggests that the intersaccadic interval in a multisaccade response is a ballistic stage. Collectively, these data indicate that the oculomotor system can control a response until a very late stage in its execution. However, if the response consists of multiple movements then the preparation of the second movement becomes refractory to new visual input, either because it is part of a preprogrammed sequence or as a consequence of being a corrective response to a motor error.