End Of Saccade Research Articles

Automated Analysis Pipeline for Extracting Saccade, Pupil, and Blink Parameters Using Video-Based Eye Tracking.

The tremendous increase in the use of video-based eye tracking has made it possible to collect eye tracking data from thousands of participants. The traditional procedures for the manual detection and classification of saccades and for trial categorization (e.g., correct vs. incorrect) are not viable for the large datasets being collected. Additionally, video-based eye trackers allow for the analysis of pupil responses and blink behaviors. Here, we present a detailed description of our pipeline for collecting, storing, and cleaning data, as well as for organizing participant codes, which are fairly lab-specific but nonetheless, are important precursory steps in establishing standardized pipelines. More importantly, we also include descriptions of the automated detection and classification of saccades, blinks, "blincades" (blinks occurring during saccades), and boomerang saccades (two nearly simultaneous saccades in opposite directions where speed-based algorithms fail to split them), This is almost entirely task-agnostic and can be used on a wide variety of data. We additionally describe novel findings regarding post-saccadic oscillations and provide a method to achieve more accurate estimates for saccade end points. Lastly, we describe the automated behavior classification for the interleaved pro/anti-saccade task (IPAST), a task that probes voluntary and inhibitory control. This pipeline was evaluated using data collected from 592 human participants between 5 and 93 years of age, making it robust enough to handle large clinical patient datasets. In summary, this pipeline has been optimized to consistently handle large datasets obtained from diverse study cohorts (i.e., developmental, aging, clinical) and collected across multiple laboratory sites.

Ocular following responses of the marmoset monkey are dependent on postsaccadic delay, spatiotemporal frequency, and saccade direction.

Ocular following is a short-latency, reflexive eye movement that tracks wide-field visual motion. It has been studied extensively in humans and macaques and is an appealing behavior for studying sensory-motor transformations in the brain because of its rapidity and rigidity. We explored ocular following in the marmoset, an emerging model in neuroscience because their lissencephalic brain allows direct access to most cortical areas for imaging and electrophysiological recordings. In three experiments, we tested ocular following responses in three adult marmosets. First, we varied the delay between saccade end and stimulus motion onset, from 10 to 300 ms. As in other species, tracking had shorter onset latencies and higher eye speeds with shorter postsaccadic delays. Second, using sine-wave grating stimuli, we explored the dependence of eye speed on spatiotemporal frequency. The highest eye speed was evoked at ∼16 Hz and ∼0.16 cycles per degree (cpd); however, the highest gain was elicited at ∼1.6 Hz and ∼1.2 cpd. The highest eye speed for each spatial frequency was observed at a different temporal frequency, but this interdependence was not consistent with complete speed tuning of the ocular following response. Finally, we found the highest eye speeds when saccade and stimulus motion directions were identical, although latencies were unaffected by direction difference. Our results showed qualitatively similar ocular following in marmosets, humans, and macaques, despite over an order of magnitude variation in body and eye size across species. This characterization will help future studies examining the neural basis of sensory-motor transformations.NEW & NOTEWORTHY Previous ocular following studies focused on humans and macaques. We examined the properties of ocular following responses in marmosets in three experiments, in which postsaccadic delay, spatial-temporal frequency of stimuli, and congruence of saccade and motion directions were manipulated. We have demonstrated short-latency ocular following in marmosets and discuss the similarities across three species that vary markedly in eye and head size. Our findings will help future studies examining the neural mechanism of sensory-motor transformations.

Saccadic modulation of neural excitability in auditory areas of the neocortex

In natural "active" vision, humans and other primates use eye movements (saccades) to sample bits of information from visual scenes. In the visual cortex, non-retinal signals linked to saccades shift visual cortical neurons into a high excitability state as each saccade ends. The extent of this saccadic modulation outside of the visual system is unknown. Here, we show that during natural viewing, saccades modulate excitability in numerous auditory cortical areas with a temporal pattern complementary to that seen in visual areas. Control somatosensory cortical recordings indicate that the temporal pattern is unique to auditory areas. Bidirectional functional connectivity patterns suggest that these effects may arise from regions involved in saccade generation. We propose that by using saccadic signals to yoke excitability states in auditory areas to those in visual areas, the brain can improve information processing in complex natural settings.

Contextual saccade adaptation induced by sequential saccades.

Saccade adaptation is the gradual adjustment of saccade end point to maintain spatial accuracy. Contextual adaptation refers to a situation in which the adaptation-related change in saccade end point is contingent on the behavioral context in which the saccade is made. For example, in some situations, the same saccade to the same retinotopic location can be simultaneously adapted in opposite directions depending on the context in which it is made. Saccade adaptation has traditionally been studied in isolated movements, but in everyday life, saccades are often planned and executed in sequences. The oculomotor system may therefore have adaptive mechanisms specific to sequential saccades. Here, in five experiments, we investigated contextual saccade adaptation in sequences of saccades. In the first experiment, we demonstrate that saccades to a given retinotopic location can be simultaneously adapted in opposite directions depending on whether they occur in isolation or in a sequence. In the other experiments, we measured the extent to which properties of the previous and following saccades in a sequence can induce contextual saccade adaptation. Overall, we find that the existence, direction, and amplitude of previous and subsequent saccades, as well as the order of the current saccade within a movement sequence, can all induce contextual adaptation. These novel findings demonstrate the surprising flexibility of the system in maintaining end point accuracy, and support the idea that saccades made in a movement sequence are planned concurrently rather than independently.NEW & NOTEWORTHY This study reveals a new type of contextual saccade adaptation: sequential saccades are able to induce contextual saccade adaptation when direction, amplitude, or the existence of preceding and following saccades are used as contexts. These novel findings are also consistent with the idea that saccades made in a sequence are planned concurrently rather than independently.

Regression-based analysis of combined EEG and eye-tracking data: Theory and applications.

Fixation-related potentials (FRPs), neural responses aligned to the end of saccades, are a promising tool for studying the dynamics of attention and cognition under natural viewing conditions. In the past, four methodological problems have complicated the analysis of such combined eye-tracking/electroencephalogram experiments: (1) the synchronization of data streams, (2) the removal of ocular artifacts, (3) the condition-specific temporal overlap between the brain responses evoked by consecutive fixations, and (4) the fact that numerous low-level stimulus and saccade properties also influence the postsaccadic neural responses. Although effective solutions exist for the first two problems, the latter two are only beginning to be addressed. In the current paper, we present and review a unified regression-based framework for FRP analysis that allows us to deconvolve overlapping potentials while also controlling for both linear and nonlinear confounds on the FRP waveform. An open software implementation is provided for all procedures. We then demonstrate the advantages of this proposed (non)linear deconvolution modeling approach for data from three commonly studied paradigms: face perception, scene viewing, and reading. First, for a traditional event-related potential (ERP) face recognition experiment, we show how this technique can separate stimulus ERPs from overlapping muscle and brain potentials produced by small (micro)saccades on the face. Second, in natural scene viewing, we model and isolate multiple nonlinear effects of saccade parameters on the FRP. Finally, for a natural sentence reading experiment using the boundary paradigm, we show how it is possible to study the neural correlates of parafoveal preview after removing spurious overlap effects caused by the associated difference in average fixation time. Our results suggest a principal way of measuring reliable eye movement-related brain activity during natural vision.

Visual information is required to reduce the global effect

When a distractor appears in close proximity to a saccade target, the saccadic end point is biased towards the distractor. This so-called global effect reduces with the latency of the saccade if the saccade is visually guided. We recently reported that the global effect does not reduce with the latency of a double-step memory-guided saccade. The aim of this study was to investigate why the global effect in memory-guided saccades does not show the typically observed reduction with saccadic latency. One possibility is that reduction of the global effect requires continuous access to visual information about target and distractor locations, which is lacking in the case of a memory-guided saccade. Alternatively, participants may be inclined to routinely preprogram a memory-guided saccade at the moment the visual information disappears, with the result that a memory-guided saccade is typically programmed on the basis of an earlier representation than necessary. To distinguish between these alternatives, two potential targets were presented, and participants were asked to make a saccade to one of them after a delay. In one condition, the target identity was precued, allowing preprogramming of the saccade, while in another condition, it was revealed by a retro cue after the delay. The global effect remained present in both conditions. Increasing visual exposure of target and distractor led to a reduction of the global effect, irrespective of whether participants could preprogram a saccade or not. The results suggest that continuous access to visual information is required in order to eliminate the global effect.

Dynamic combination of position and motion information when tracking moving targets.

To accurately foveate a moving target, the oculomotor system needs to estimate the position of the target at the saccade end, based on information about its position and ongoing movement, while accounting for neuronal delays and execution time of the saccade. We investigated human interceptive saccades and pursuit responses to moving targets defined by high and low luminance contrast or by chromatic contrast only (isoluminance). We used step-ramps with perpendicular directions between vertical target steps of 10 deg/s and horizontal ramps of 2.5 to 20 deg/s to separate errors with respect to the position step of the target in the vertical dimension, and errors related to target motion in the horizontal dimension. Interceptive saccades to targets of high and low luminance contrast landed close to the actual target positions, suggesting relatively accurate estimates of the amount of target displacement. Interceptive saccades to isoluminant targets were less accurate. They landed at positions the target had on average 100 ms before saccade onset. One account of this finding is that the integration of target motion is compromised for isoluminant targets moving in the periphery. In this case, the oculomotor system can use an accurate, but delayed position component, but cannot account for target movement. This deficit was also present for the postsaccadic pursuit speed. For the two luminance conditions, pursuit direction and speed were adjusted depending on the saccadic landing position. The rapid postsaccadic pursuit adjustments suggest shared position- and motion-related signals of target and eye for saccade and pursuit control.

Binocular Eye Movements Are Adapted to the Natural Environment.

Humans and many animals make frequent saccades requiring coordinated movements of the eyes. When landing on the new fixation point, the eyes must converge accurately or double images will be perceived. We asked whether the visual system uses statistical regularities in the natural environment to aid eye alignment at the end of saccades. We measured the distribution of naturally occurring disparities in different parts of the visual field. The central tendency of the distributions was crossed (nearer than fixation) in the lower field and uncrossed (farther) in the upper field in male and female participants. It was uncrossed in the left and right fields. We also measured horizontal vergence after completion of vertical, horizontal, and oblique saccades. When the eyes first landed near the eccentric target, vergence was quite consistent with the natural-disparity distribution. For example, when making an upward saccade, the eyes diverged to be aligned with the most probable uncrossed disparity in that part of the visual field. Likewise, when making a downward saccade, the eyes converged to enable alignment with crossed disparity in that part of the field. Our results show that rapid binocular eye movements are adapted to the statistics of the 3D environment, minimizing the need for large corrective vergence movements at the end of saccades. The results are relevant to the debate about whether eye movements are derived from separate saccadic and vergence neural commands that control both eyes or from separate monocular commands that control the eyes independently.SIGNIFICANCE STATEMENT We show that the human visual system incorporates statistical regularities in the visual environment to enable efficient binocular eye movements. We define the oculomotor horopter: the surface of 3D positions to which the eyes initially move when stimulated by eccentric targets. The observed movements maximize the probability of accurate fixation as the eyes move from one position to another. This is the first study to show quantitatively that binocular eye movements conform to 3D scene statistics, thereby enabling efficient processing. The results provide greater insight into the neural mechanisms underlying the planning and execution of saccadic eye movements.

Saccade latencies during a preferential looking task and objective scoring of grating acuity in children with and without visual impairments.

PurposeWe determined the latencies of orienting responses during a preferential looking task in children with normal vision and in children with visual impairments between 6 and 12 years old, and assessed the feasibility of scoring grating detection in these populations with video‐based eye tracking.MethodsChildren performed a computerized preferential looking test, while a remote eye tracker measured the children's eye movements. The stimuli consisted of a 2 × 2 grid, with three uniform grey fields and one target field consisting of a black‐and‐white square wave grating. The grating was presented randomly at one of the four locations. The spatial frequencies (1.05, 2.11 and 7.02 cyc/deg) were randomly interleaved, with 10 trials per spatial frequency. Three different methods were used to score the accuracy of the responses: (1) primary saccade ends on target, (1) gaze 50% of the presentation time on target, and (3) a combination of method 1 and 2 (i.e. primary saccade ends on target, and/or gaze 50% of the presentation time on target).ResultsThe combined scoring method was most reliable to determine whether children could resolve the gratings. Children with visual impairments had significantly lower accuracies than children with normal vision with all three scoring methods. In addition, saccade latencies decreased with age and were significantly longer (62 ± 15 ms) in children with visual impairments.ConclusionThe use of eye tracking to assess grating detection with a preferential looking task in clinical populations provides valuable additional information, including objective detection measures and developmental delays in saccade latencies.

P1-2-09. Saccadic eye movements in spinocerebellar degeneration – correspondence to clinical manifestation

Objectives The cerebellum is implicated in the control of saccade but saccade abnormality in cerebellar disorders is not fully delineated, especially in the vertical directions. To clarify the role of the cerebellum, we studied saccades in hereditary spinocerebellar degeneration (SCD) with pure cerebellar presentation. Methods Subjects were 20 SCD patients (SCA6, 31) and 19 normal age-matched controls. We used the visual guided (VGS) and memory guided saccade tasks (MGS) in which targets were presented in 8 possible directions with 45 degree separation, and recorded saccades with a video-based eye tracker. Results In SCA patients, latency was significantly delayed mainly in vertical and oblique directions, which correlated with SARA scores. The number of both overshoot and undershoot increased correlatively with SARA scores in all directions. Peak velocity showed no change, but the time from the beginning to the peak velocity of the saccade was shorter, whereas the time from the peak velocity to the end of saccade was longer. Conclusion The change in amplitude and velocity profile may reflect abnormal output from the cerebellar nuclei, whereas the pathophysiology of prolonged latency remains elusive. We hope the saccade analysis paves the way toward objective assessment of SCD and other neurological disorders.

Neuronal Representation of the Saccadic Timing Signals in Macaque Lateral Intraparietal Area.

Primates frequently make saccades direct fovea on interesting objects to receive acute visual information. However, saccade displaces the images on retina and disrupts the visual constancy. One possible mechanism to retain visual constancy is by integrating the presaccadic and postsaccadic visual information right at the time of saccade, which makes the timing of saccade crucial. So far, the saccadic timing signals have been found only in the subcortical regions, for example, the cerebellum and superior colliculus, but not in the neocortex. Here we report 2 types of saccadic timing signals in macaque lateral intraparietal area (LIP). First, many presaccadic response neurons started to decline activity either right around the start (saccade-on-decay) or the end (saccade-off-decay) of saccades. Notably, the time difference between saccade-off-decay and saccade-on-decay was highly correlated with the mean duration of saccades but not with the individual ones, and both saccade-off-decay and saccade-on-decay were better aligned with saccade end than saccade start-reflecting prediction. Second, the peak activity plateau of a group of postsaccadic response neurons was highly correlated with the actual duration of saccade-reflecting reality. While the predicted timing signals might facilitate the integration of visual information across saccades in LIP, the actual duration signals might calibrate the prediction errors.

Effector-based attention systems.

Visual processing is known to be enhanced at the end point of eye movements. Feedback within the oculomotor system has been shown to drive these alterations in visual processing. However, we do not simply view the world; we also reach out and interact using our hands. Consequently, it is not surprising that visual processing has also been shown to be altered in near-hand space. A growing body of work documents a myriad of alterations in near-hand visual processing, with little consensus on the neural underpinnings of the effect of the hand. Since movement of the eyes and hands is governed by parallel frontoparietal networks and since within the oculomotor system feedback from these motor control regions has been shown to drive enhanced visual processing at saccade end points, it is plausible that a similar feedback mechanism is at play in near-hand improvements in visual processing. Here, we compare and contrast oculomotor-driven and hand-driven changes in visual processing and provide support for the hypothesis that feedback within the reaching and grasping systems enhances visual processing near the hand in a novel way.

Short-latency allocentric control of saccadic eye movements.

We found that the saccade end point was shifted from the actual target position toward the direction expected from allocentric coding when a large frame in the background was transiently shifted during the period of target presentation. The effect occurred within 150 ms. The present study provides direct evidence that the brain rapidly uses allocentric coding of a target to control immediate saccades.

Masking the saccadic smear.

Static visual stimuli are smeared across the retina during saccades, but in normal conditions this smear is not perceived. Instead, we perceive the visual scene as static and sharp. However, retinal smear is perceived if stimuli are shown only intrasaccadically, but not if the stimulus is additionally shown before a saccade begins, or after the saccade ends (Campbell & Wurtz, 1978). This inhibition has been compared to forward and backward metacontrast masking, but with spatial relations between stimulus and mask that are different from ordinary metacontrast during fixation. Previous studies of smear masking have used subjective measures of smear perception. Here we develop a new, objective technique for measuring smear masking, based on the spatial localization of a gap in the smear created by very quickly blanking the stimulus at various points during the saccade. We apply this technique to show that smear masking survives dichoptic presentation (suggesting that it is therefore cortical in origin), as well as separations of as much as 6° between smear and mask.

Adaptation of Saccadic Sequences with and without Remapping.

It is relatively easy to adapt visually-guided saccades because the visual vector and the saccade vector match. The retinal error at the saccade landing position is compared to the prediction error, based on target location and efference copy. If these errors do not match, planning processes at the level(s) of the visual and/or motor vector processing are assumed to be inaccurate and the saccadic response is adjusted. In the case of a sequence of two saccades, the final error can be attributed to the last saccade vector or to the entire saccadic displacement. Here, we asked whether and how adaptation can occur in the case of remapped saccades, such as during the classic double-step saccade paradigm, where the visual and motor vectors of the second saccade do not coincide and so the attribution of error is ambiguous. Participants performed saccades sequences to two targets briefly presented prior to first saccade onset. The second saccade target was either briefly re-illuminated (sequential visually-guided task) or not (remapping task) upon first saccade offset. To drive adaptation, the second target was presented at a displaced location (backward or forward jump condition or control—no jump) at the end of the second saccade. Pre- and post-adaptation trials were identical, without the re-appearance of the target after the second saccade. For the 1st saccade endpoints, there was no change as a function of adaptation. For the 2nd saccade, there was a similar increase in gain in the forward jump condition (52% and 61% of target jump) in the two tasks, whereas the gain decrease in the backward condition was much smaller for the remapping task than for the sequential visually-guided task (41% vs. 94%). In other words, the absolute gain change was similar between backward and forward adaptation for remapped saccades. In conclusion, we show that remapped saccades can be adapted, suggesting that the error is attributed to the visuo-motor transformation of the remapped visual vector. The mechanisms by which adaptation takes place for remapped saccades may be similar to those of forward sequential visually-guided saccades, unlike those involved in adaptation for backward sequential visually-guided saccades.

Modeling Inter-trial Variability of Saccade Trajectories: Effects of Lesions of the Oculomotor Part of the Fastigial Nucleus

This study investigates the inter-trial variability of saccade trajectories observed in five rhesus macaques (Macaca mulatta). For each time point during a saccade, the inter-trial variance of eye position and its covariance with eye end position were evaluated. Data were modeled by a superposition of three noise components due to 1) planning noise, 2) signal-dependent motor noise, and 3) signal-dependent premotor noise entering within an internal feedback loop. Both planning noise and signal-dependent motor noise (together called accumulating noise) predict a simple S-shaped variance increase during saccades, which was not sufficient to explain the data. Adding noise within an internal feedback loop enabled the model to mimic variance/covariance structure in each monkey, and to estimate the noise amplitudes and the feedback gain. Feedback noise had little effect on end point noise, which was dominated by accumulating noise. This analysis was further extended to saccades executed during inactivation of the caudal fastigial nucleus (cFN) on one side of the cerebellum. Saccades ipsiversive to an inactivated cFN showed more end point variance than did normal saccades. During cFN inactivation, eye position during saccades was statistically more strongly coupled to eye position at saccade end. The proposed model could fit the variance/covariance structure of ipsiversive and contraversive saccades. Inactivation effects on saccade noise are explained by a decrease of the feedback gain and an increase of planning and/or signal-dependent motor noise. The decrease of the fitted feedback gain is consistent with previous studies suggesting a role for the cerebellum in an internal feedback mechanism. Increased end point variance did not result from impaired feedback but from the increase of accumulating noise. The effects of cFN inactivation on saccade noise indicate that the effects of cFN inactivation cannot be explained entirely with the cFN’s direct connections to the saccade-related premotor centers in the brainstem.

Two distinct types of remapping in primate cortical area V4

Visual neurons typically receive information from a limited portion of the retina, and such receptive fields are a key organizing principle for much of visual cortex. At the same time, there is strong evidence that receptive fields transiently shift around the time of saccades. The nature of the shift is controversial: Previous studies have found shifts consistent with a role for perceptual constancy; other studies suggest a role in the allocation of spatial attention. Here we present evidence that both the previously documented functions exist in individual neurons in primate cortical area V4. Remapping associated with perceptual constancy occurs for saccades in all directions, while attentional shifts mainly occur for neurons with receptive fields in the same hemifield as the saccade end point. The latter are relatively sluggish and can be observed even during saccade planning. Overall these results suggest a complex interplay of visual and extraretinal influences during the execution of saccades.

Remapping, Spatial Stability, and Temporal Continuity: From the Pre-Saccadic to Postsaccadic Representation of Visual Space in LIP.

As our eyes move, we have a strong percept that the world is stable in space and time; however, the signals in cortex coming from the retina change with each eye movement. It is not known how this changing input produces the visual percept we experience, although the predictive remapping of receptive fields has been described as a likely candidate. To explain how remapping accounts for perceptual stability, we examined responses of neurons in the lateral intraparietal area while animals performed a visual foraging task. When a stimulus was brought into the response field of a neuron that exhibited remapping, the onset of the postsaccadic representation occurred shortly after the saccade ends. Whenever a stimulus was taken out of the response field, the presaccadic representation abruptly ended shortly after the eyes stopped moving. In the 38% (20/52) of neurons that exhibited remapping, there was no more than 30 ms between the end of the presaccadic representation and the start of the postsaccadic representation and, in some neurons, and the population as a whole, it was continuous. We conclude by describing how this seamless shift from a presaccadic to postsaccadic representation could contribute to spatial stability and temporal continuity.

The kinematics of far-near re-fixation saccades.

We have analyzed the three-dimensional spatiotemporal characteristics of saccadic refixations between far and near targets in three behaviorally trained rhesus monkeys. The kinematics underlying these rapid eye movements can be accurately described by rotations of the eyes in four different planes, namely, first disconjugate rotations in the horizontal plane of regard converging the eyes toward the near target, followed by rotations in each eye's vertical direction plane, and finally, disconjugate rotations in a common frontoparallel plane. This compounded rotation of the eye was underlying an initially fast-rising variable torsion that typically overshot the final torsion, which the eyes attained at the time of target acquisition. The torsion consisted of a coarse, widely varying component of opposite polarity in the two eyes, which contained a more robust, much smaller modulation that sharply increased toward the end of saccades. The reorientation of the eyes in torsion depended on each eye's azimuth, elevation, and target distance. We conclude that refixation saccades are generated by motor commands that control ocular torsion in concert with the saccade generator, which operates in Donders-Listing kinematics underlying Listing's law.

Contextual factors modulate concurrent planning of sequential saccades.

Natural vision typically involves making multiple eye movements to interpret complex visual scenes. Although previous work has shown that individual saccadic end points are modulated by cognitive context, whether and how contextual factors quantitatively influence the planning of sequential saccades is still unclear. We compared performance of subjects in a modified double-step task under different task instructions (FOLLOW vs. REDIRECT; Ray, Schall, & Murthy, 2004) as well as task structure (40% and 100% FOLLOW). The results support the idea of restricted concurrent preparation when the second saccade was part of the sequence as per task demands as opposed to being inadvertently made following an error. Also, increasing the probability of double-target trials in the task (100% vs. 40% FOLLOW) tended to enhance concurrent planning even when the serial order of saccades continued to remain important. Taken together, these data reveal how the concurrent planning of sequential saccades can be contextually regulated by means of task instruction and trial statistics.