Successive Fixations Research Articles

Quantifying task-related gaze.

Competing theories attempt to explain what guides eye movements when exploring natural scenes: bottom-up image salience and top-down semantic salience. In one study, we apply language-based analyses to quantify the well-known observation that task influences gaze in natural scenes. Subjects viewed ten scenes as if they were performing one of two tasks. We found that the semantic similarity between the task and the labels of objects in the scenes captured the task-dependence of gaze (t(39) = 13.083; p < 0.001). In another study, we examined whether image salience or semantic salience better predicts gaze during a search task, and if viewing strategies are affected by searching for targets of high or low semantic relevance to the scene. Subjects searched 100 scenes for a high- or low-relevance object. We found that image salience becomes a worse predictor of gaze across successive fixations, while semantic salience remains a consistent predictor (X2(1, N=40) = 75.148, p < .001). Furthermore, we found that semantic salience decreased as object relevance decreased (t(39) = 2.304; p = .027). These results suggest that semantic salience is a useful predictor of gaze during task-related scene viewing, and that even in target-absent trials, gaze is modulated by the relevance of a search target to the scene in which it might be located.

Mammals Achieve Common Neural Coverage of Visual Scenes Using Distinct Sampling Behaviors.

Most vertebrates use head and eye movements to quickly change gaze orientation and sample different portions of the environment with periods of stable fixation. Visual information must be integrated across fixations to construct a complete perspective of the visual environment. In concert with this sampling strategy, neurons adapt to unchanging input to conserve energy and ensure that only novel information from each fixation is processed. We demonstrate how adaptation recovery times and saccade properties interact and thus shape spatiotemporal tradeoffs observed in the motor and visual systems of mice, cats, marmosets, macaques, and humans. These tradeoffs predict that in order to achieve similar visual coverage over time, animals with smaller receptive field sizes require faster saccade rates. Indeed, we find comparable sampling of the visual environment by neuronal populations across mammals when integrating measurements of saccadic behavior with receptive field sizes and V1 neuronal density. We propose that these mammals share a common statistically driven strategy of maintaining coverage of their visual environment over time calibrated to their respective visual system characteristics.

Generalized Hurst Hypothesis: Description of Time-Series in Communication Systems

In this paper, we focus on the generalization of the Hurst empirical law and suggest a set of reduced parameters for quantitative description of long-time series. These series are usually considered as a specific response of a complex system (economic, geophysical, electromagnetic and other systems), where successive fixations of external factors become impossible. We consider applying generalized Hurst laws to obtain a new set of reduced parameters in data associated with communication systems. We analyze three hypotheses. The first one contains one power-law exponent. The second one incorporates two power-law exponents, which are in many cases complex-conjugated. The third hypothesis has three power-law exponents, two of which are complex-conjugated as well. These hypotheses describe with acceptable accuracy (relative error does not exceed 2%) a wide set of trendless sequences (TLS) associated with radiometric measurements. Generalized Hurst laws operate with R/S curves not only in the asymptotic region, but in the entire domain. The fitting parameters can be used as the reduced parameters for the description of the given data. The paper demonstrates that this general approach can also be applied to other TLS.

Looking (for) patterns: Similarities and differences between infant and adult free scene-viewing patterns.

Systematic tendencies such as the center and horizontal bias are known to have a large influence on how and where we move our eyes during static onscreen free scene viewing. However, it is unknown whether these tendencies are learned viewing strategies or are more default tendencies in the way we move our eyes. To gain insight into the origin of these tendencies we explore the systematic tendencies of infants (3 - 20-month-olds, N = 157) and adults (N = 88) in three different scene viewing data sets. We replicated com-mon findings, such as longer fixation durations and shorter saccade amplitudes in infants compared to adults. The leftward bias was never studied in infants, and our results indi-cate that it is not present, while we did replicate the leftward bias in adults. The general pattern of the results highlights the similarity between infant and adult eye movements. Similar to adults, infants’ fixation durations increase with viewing time and the depend-encies between successive fixations and saccades show very similar patterns. A straight-forward conclusion to draw from this set of studies is that infant and adult eye movements are mainly driven by similar underlying basic processes.

Predicting Saccadic Eye Movements in Free Viewing of Webpages

Attention modeling for webpages has emerged as a new research direction in computer vision. Despite an amount of research effort, most studies have focused on estimating webpage saliency to reveal the static location of human fixations. Without temporal information, existing models cannot interpret the dynamic properties of the actual attention process in free-viewing webpages. To solve this problem, we propose a webpage-based saccadic model in this study to model dynamic visual search behaviors of humans when they view webpages. In the first stage, we utilize the support vector machine to learn the mapping from multilevel saliency features to an initial probability of being fixated. In the second stage, we combine the mechanisms of spatial bias and inhibition of return with the estimation of the initial probability to iteratively predict a sequence of successive fixations for each webpage image. Experimental results on a benchmark eye-tracking data set for webpages have demonstrated that the proposed model outperforms the state-of-the-art saccadic methods.

Long-term evolutionary conflict, Sisyphean arms races, and power in Fisher's geometric model.

Evolutionary conflict and arms races are important drivers of evolution in nature. During arms races, new abilities in one party select for counterabilities in the second party. This process can repeat and lead to successive fixations of novel mutations, without a long‐term increase in fitness. Models of co‐evolution rarely address successive fixations, and one of the main models that use successive fixations—Fisher's geometric model—does not address co‐evolution. We address this gap by expanding Fisher's geometric model to the evolution of joint phenotypes that are affected by two parties, such as probability of infection of a host by a pathogen. The model confirms important intuitions and offers some new insights. Conflict can lead to long‐term Sisyphean arms races, where parties continue to climb toward their fitness peaks, but are dragged back down by their opponents. This results in far more adaptive evolution compared to the standard geometric model. It also results in fixation of mutations of larger effect, with the important implication that the common modeling assumption of small mutations will apply less often under conflict. Even in comparison with random abiotic change of the same magnitude, evolution under conflict results in greater distances from the optimum, lower fitness, and more fixations, but surprisingly, not larger fixed mutations. We also show how asymmetries in selection strength, mutation size, and mutation input allow one party to win over another. However, winning abilities come with diminishing returns, helping to keep weaker parties in the game.

Anticipatory eye fixations reveal tool knowledge for tool interaction.

Action-oriented eye-tracking studies have shown that eye fixations reveal much about current behavioral intentions. The eyes typically fixate those positions of a tool or an object where the fingers will be placed next, or those positions in a scene, where obstacles need to be avoided to successfully reach or transport a tool or object. Here, we asked to what extent eye fixations can also reveal active cognitive inference processes, which are expected to integrate bottom-up visual information with internal knowledge for planning suitable object interactions task-dependently. In accordance to the available literature, we expected that task-relevant knowledge will include sensorimotor, semantic, and mechanical aspects. To investigate if and in which way this internal knowledge influences eye fixation behavior while planning an object interaction, we presented pictures of familiar and unfamiliar tools and instructed participants to either pantomime 'lifting' or 'using' the respective tool. When confronted with unfamiliar tools, participants fixated the tool's effector part closer and longer in comparison with familiar tools. This difference was particularly prominent during 'using' trials when compared with 'lifting' trials. We suggest that this difference indicates that the brain actively extracts mechanical information about the unknown tool in order to infer its appropriate usage. Moreover, the successive fixations over a trial indicate that a dynamic, task-oriented, active cognitive process unfolds, which integrates available tool knowledge with visually gathered information to plan and determine the currently intended tool interaction.

An eye fixation-related potentials analysis of the P300 potential for fixations onto a target object when exploring natural scenes.

The P300 event-related potential has been extensively studied in electroencephalography with classical paradigms that force observers to not move their eyes. This potential is classically used to infer whether a target or a task-relevant stimulus was presented. Few researches have studied this potential through more ecological paradigms where observers were able to move their eyes. In this study, we examined with an ecological paradigm and an adapted methodology the P300 potential using a visual search task that involves eye movements to actively explore natural scenes and during which eye movements and electroencephalographic activity were coregistered. Averaging the electroencephalography signal time-locked to fixation onsets, a P300 potential was observed for fixations onto the target object but not for other fixations recorded for the same visual search or for fixations recorded during the free viewing without any task. Our approach consists of using control experimental conditions with similar eye movements to ensure that the P300 potential was attributable to the fact that the observer gazed at the target rather than to other factors such as eye movement pattern (the size of the previous saccade) or the "overlap issue" between the potentials elicited by two successive fixations. We also proposed to model the time overlap issue of the potentials elicited by consecutive fixations with various durations. Our results show that the P300 potential can be studied in ecological situations without any constraint on the type of visual exploration, with some precautions in the interpretation of results due to the overlap issue.

Time course of the P300 Eye-Fixation Related Potential during the visual search for a target embedded in natural scenes

The Event Related Potential P300 has been intensively studied for over forty years (Polish, 2012). It is elicited at each updating of the stimulus representation, reflecting a cascade of cognitive processes engaging attentional and memory mechanisms. This potential is a positive component, appearing 300ms after stimulus onset, with maximal amplitude in centro-parietal regions. This study is mainly concerned by the analysis of the appearance and evolution of the P300 elicited by the onset of successive fixations recorded during a visual search using an ecological experimental paradigm. This goal is achieved through the co-registration of electroencephalographic (EEG) and eye-tracking (ET) signals. EEG and ET signals were recorded for thirty-four observers when exploring natural scenes during 5 sec to answer if a target object was or not present. To solve the main difficulty due to overlaps between the observed Eye Fixation Related Potentials (EFRP) elicited by consecutive fixations, we have used a new methodology based on the xDawn algorithm (Rivet, et al.; 2009). From each trial, a temporal window of 2s centered on the first fixation inside the Region of Interest (target). This duration was chosen to include five consecutive fixations (two before the first entrance, and two after). The fixations were tagged depending to their rank and position (inside/outside) the ROI. Neural activities were modelled as a cumulative response of specific latent responses elicited by each fixation with the corresponding onset. After xDawn estimation, the topographic maps have shown a potential identified as the P300 component elicited by the visual input through the consecutive fixations landed on the ROI. This result confirms previous studies but has been obtained for the first time on ecological paradigm thanks to the co-registration of EEG and ET signals and a new methodology to denoise potentials from overlaps of potentials elicited by consecutive fixations. Meeting abstract presented at VSS 2015

Integrating retinotopic features in spatiotopic coordinates.

The receptive fields of early visual neurons are anchored in retinotopic coordinates (Hubel and Wiesel, 1962). Eye movements shift these receptive fields and therefore require that different populations of neurons encode an object's constituent features across saccades. Whether feature groupings are preserved across successive fixations or processing starts anew with each fixation has been hotly debated (Melcher and Morrone, 2003; Melcher, 2005, 2010; Knapen et al., 2009; Cavanagh et al., 2010a,b; Morris et al., 2010). Here we show that feature integration initially occurs within retinotopic coordinates, but is then conserved within a spatiotopic coordinate frame independent of where the features fall on the retinas. With human observers, we first found that the relative timing of visual features plays a critical role in determining the spatial area over which features are grouped. We exploited this temporal dependence of feature integration to show that features co-occurring within 45 ms remain grouped across eye movements. Our results thus challenge purely feedforward models of feature integration (Pelli, 2008; Freeman and Simoncelli, 2011) that begin de novo after every eye movement, and implicate the involvement of brain areas beyond early visual cortex. The strong temporal dependence we quantify and its link with trans-saccadic object perception instead suggest that feature integration depends, at least in part, on feedback from higher brain areas (Mumford, 1992; Rao and Ballard, 1999; Di Lollo et al., 2000; Moore and Armstrong, 2003; Stanford et al., 2010).

Dynamic programming for re-mapping noisy fixations in translation tasks

Eyetrackers which allow for free head movements are in many cases imprecise to the extent that reading patterns become heavily distorted. The poor usability and interpretability of these gaze patterns is corroborated by a "naïve" fixation-to-symbol mapping, which often wrongly maps the possibly drifted center of the observed fixation onto the symbol directly below it. In this paper I extend this naïve fixation-to-symbol mapping by introducing background knowledge about the translation task. In a first step, the sequence of fixation-to-symbol mappings is extended into a lattice of several possible fixated symbols, including those on the line above and below the naïve fixation mapping. In a second step a dynamic programming algorithm applies a number of heuristics to find the best path through the lattice, based on the probable distance in characters, in words and in pixels between successive fixations and the symbol locations, so as to smooth the gazing path according to the background gazing model. A qualitative and quantitative evaluation shows that the algorithm increases the accuracy of the re-mapped symbol sequence.

Eye movements blink the attentional blink.

When presented with a sequence of visual stimuli in rapid succession, participants often fail to detect a second salient target, a phenomenon referred as the attentional blink (AB; Raymond, Shapiro, & Arnell, 1992; Shapiro, Raymond, & Arnell, 1997). On the basis of a vast corpus of experiments, several cognitive theories suggest that the blink results from a discrete structuring of attention, sampling information from temporal episodes during which several items can access encoding process (Wyble, Bowman, & Nieuwenstein, 2009; Wyble, Potter, Bowman, & Nieuwenstein, 2011). The objective of this work is to explore the AB when multiple items are presented at the fovea during ocular movements. The authors reasoned that each fixation may cohesively form an episode and hence expected that the blink may vanish within a single fixation. In turn, they expected saccades to accentuate episodic borders and hence shorten the regime of interference when 2 targets are presented fovealy in successive fixations. Evidence is provided in favor of this hypothesis, showing that the blink vanishes when both targets are presented in the core of a single fixation (far from the saccadic boundaries) and that it recovers more rapidly in successive fixations. These studies support current views that episodes should have an effect on the AB and provide evidence that eye movements play an important role in the formation of episodes.

Visual Exploration and Object Recognition by Lattice Deformation

Mechanisms of explicit object recognition are often difficult to investigate and require stimuli with controlled features whose expression can be manipulated in a precise quantitative fashion. Here, we developed a novel method (called “Dots”), for generating visual stimuli, which is based on the progressive deformation of a regular lattice of dots, driven by local contour information from images of objects. By applying progressively larger deformation to the lattice, the latter conveys progressively more information about the target object. Stimuli generated with the presented method enable a precise control of object-related information content while preserving low-level image statistics, globally, and affecting them only little, locally. We show that such stimuli are useful for investigating object recognition under a naturalistic setting – free visual exploration – enabling a clear dissociation between object detection and explicit recognition. Using the introduced stimuli, we show that top-down modulation induced by previous exposure to target objects can greatly influence perceptual decisions, lowering perceptual thresholds not only for object recognition but also for object detection (visual hysteresis). Visual hysteresis is target-specific, its expression and magnitude depending on the identity of individual objects. Relying on the particular features of dot stimuli and on eye-tracking measurements, we further demonstrate that top-down processes guide visual exploration, controlling how visual information is integrated by successive fixations. Prior knowledge about objects can guide saccades/fixations to sample locations that are supposed to be highly informative, even when the actual information is missing from those locations in the stimulus. The duration of individual fixations is modulated by the novelty and difficulty of the stimulus, likely reflecting cognitive demand.

The contribution of low-level features at the centre of gaze to saccade target selection

Does it matter what observers are looking at right now to determine where they will look next? We recorded eye movements and computed colour, local orientation, motion, and geometrical invariants on dynamic natural scenes. The distributions of differences between features at successive fixations were compared with those from random scanpaths of varying similarity to natural scanpaths. Although distributions show significant differences, these feature correlations are mainly due to spatio-temporal correlations in natural scenes and a target selection bias, e.g. towards moving objects. Our results indicate that low-level features at fixation contribute little to the choice of the next saccade target.

An exact steady state solution of Fisher’s geometric model and other models

Because nearly neutral substitutions are thought to contribute substantially to molecular evolution, and much of our insight about the workings of nearly neutral evolution relies on theory, solvable models of this process are of particular interest. Here, I present an analytical method for solving models of nearly neutral evolution at steady state. The steady state solution applies to any constant fitness landscape under a dynamic of successive fixations, each of which occurs on the background of the population’s most recent common ancestor. Because this dynamic neglects the effects of polymorphism in the population beyond the mutant allele under consideration, the steady state solution provides a decent approximation of evolutionary dynamics when the population mutation rate is low ( N u ≪ 1 ) . To demonstrate the method, I apply it to two examples: Fisher’s geometric model (FGM), and a simple model of molecular evolution. Since recent papers have studied the steady state behavior of FGM under this dynamic, I analyze its behavior in detail and compare the results with previous work.

Eye Movements: Building a Stable World from Glance to Glance

Frequent exploratory eye-movements called saccades pose for the visual system the problem of combining information from successive fixations into an apparently seam less conscious experience. A new study shows that information from successive fixations is combined, not by fusing fixation 'snapshots', but by integrating more complex visual attributes at a mid-high level of analysis.

Subcortical Modulation of Attention Counters Change Blindness

Change blindness is the failure to see large changes in a visual scene that occur simultaneously with a global visual transient. Such visual transients might be brief blanks between visual scenes or the blurs caused by rapid or saccadic eye movements between successive fixations. Shifting attention to the site of the change counters this "blindness" by improving change detection and reaction time. We developed a change blindness paradigm for visual motion and then showed that presenting an attentional cue diminished the blindness in both humans and old world monkeys. We then replaced the visual cue with weak electrical stimulation of an area in the monkey's brainstem, the superior colliculus, to see if activation at such a late stage in the eye movement control system contributes to the attentional shift that counters change blindness. With this stimulation, monkeys more easily detected changes and had shorter reaction times, both characteristics of a shift of attention.

Saccades actively maintain perceptual continuity.

People make saccades--rapid eye movements to a new fixation--approximately three times per second. This would seemingly disrupt perceptual continuity, yet our brains construct a coherent, stable view of the world from these successive fixations. There is conflicting evidence regarding the effects of saccades on perceptual continuity: some studies report that they are disruptive, with little information carryover between saccades; others report that carryover is substantial. Here we show that saccades actively contribute to perceptual continuity in humans in two different ways. When bistable stimuli are presented intermittently, saccades executed during the blank interval shorten the duration of states of ambiguous figures, indicating that saccades can erase immediately past perceptual states. On the other hand, they prolong the McCollough effect, indicating that saccades strengthen learned contingencies. Our results indicate that saccades help, rather than hinder, perceptual continuity.

Spatiotopic temporal integration of visual motion across saccadic eye movements.

Saccadic eye movements pose many challenges for stable and continuous vision, such as how information from successive fixations is amalgamated into a single precept. Here we show in humans that motion signals are temporally integrated across separate fixations, but only when the motion stimulus falls either on the same retinal region (retinotopic integration) or on different retinal positions that correspond to the same external spatial coordinates (spatiotopic integration). We used individual motion signals that were below detection threshold, implicating spatiotopic trans-saccadic integration in relatively early stages of visual processing such as the middle temporal area (MT) or V5 of visual cortex. The trans-saccadic buildup of important congruent visual information while irrelevant non-congruent information fades could provide a simple and robust strategy to stabilize perception during eye movements.

Change detection is impaired in children with dyslexia.

The severe deficits in rapid automatized naming demonstrated by children with developmental dyslexia has usually been interpreted in terms of a deficit in speed of access to the lexicon rather than as a possible deficit in speed of visual object recognition. Yet fluent reading requires rapid visual recognition and semantic interpretation of new letters and words appearing in successive fixations of the eyes. Thus we wondered whether change detection performance was related to reading ability. We investigated whether children with developmental dyslexia (DD) were less able to detect change in a simple display--gap--display paradigm than normal reading (NR) children of the same age and children with impaired reading and mentation (LD). In a first experimental phase, the DDs required a longer initial exposure of four letter items in order to detect change of a single letter at a level of 71% correct, compared with NRs performing at the same level. Thus the deficit in reading in DD is associated with a deficit in early processes associated with visual recognition. In a second experimental phase (using the individual target display exposures measured in the first phase), cues appeared during the 250 ms gap for a period of either 0 (no cue), 50 or 200 ms immediately prior to the presentation of the second (comparison) display. Children of all groups showed dependence on the presence of the cue to help make a judgement of change (versus no change), with the NRs least affected. When change was detected in the presence of a cue, the NRs were better able to identify the new letter than either of the other groups. However, only about 50% of the correct detections were accompanied by a correct identification. Despite published reports of a mini-neglect for left visual field in dyslexic adults, none of our groups showed such an effect. However, a significant upper visual field (UpVF) advantage in change detection performance was found across groups, which we interpret in terms of the interactions of the ventral and dorsal streams.