Foveal Vision Research Articles

A self-learning cognitive architecture exploiting causality from rewards

Inspired by the human vision system and learning, we propose a novel cognitive architecture that understands the content of raw videos in terms of objects without using labels. The architecture achieves four objectives: (1) Decomposing raw frames in objects by exploiting foveal vision and memory. (2) Describing the world by projecting objects on an internal canvas. (3) Extracting relevant objects from the canvas by analyzing the causal relation between objects and rewards. (4) Exploiting the information of relevant objects to facilitate the reinforcement learning (RL) process. In order to speed up learning, and better identify objects that produce rewards, the architecture implements learning by causality from the perspective of Wiener and Granger using object trajectories stored in working memory and the time series of external rewards. A novel non-parametric estimator of directed information using Renyi’s entropy is designed and tested. Experiments on three environments show that our architecture extracts most of relevant objects. It can be thought of as ‘understanding’ the world in an object-oriented way. As a consequence, our architecture outperforms state-of-the-art deep reinforcement learning in terms of training speed and transfer learning.

Fluctuations in central foveal thickness and association with vision outcomes with anti-VEGF therapy for nAMD: HARBOR post hoc analysis

ObjectiveTo evaluate correlations between variability in central foveal thickness (CFT) and vision with ranibizumab in a HARBOR post hoc analysis.Methods and analysisPatients with neovascular age-related macular degeneration (nAMD; N=1097) received...

Interaction of Foveal and Peripheral Vision on Implementation of Systemic Visual Functions

Interaction of Foveal and Peripheral Vision on Implementation of Systemic Visual Functions

Boynton Award Lecture: Cone resilience, rod vulnerability - how precise retinal topography will help beat age-related macular degeneration

AMD is both the most prevalent disease with a CNS neurodegeneration and the most approachable due to the precision and in vivo visibility of photoreceptors and their support cells and very large biological effects. We hypothesize that a center of cone resilience due to xanthophyll carotenoids in foveal Müller glia overlies a larger surround of rod vulnerability due to age-related lipidization of Bruch's membrane and choriocapillary endothelium degeneration. A timeline of disease progression is in view for drusen-driven disease, across the lifespan and aligning spatially on the biology of foveal cone vision. Rods elucidate the next chapter of deposit-driven AMD end-stages as well as a new neuroscience for lysosome-related organelles in the retinal pigment epithelium.

Visual search in naturalistic scenes from foveal to peripheral vision: A comparison between dynamic and static displays.

How important foveal, parafoveal, and peripheral vision are depends on the task. For object search and letter search in static images of real-world scenes, peripheral vision is crucial for efficient search guidance, whereas foveal vision is relatively unimportant. Extending this research, we used gaze-contingent Blindspots and Spotlights to investigate visual search in complex dynamic and static naturalistic scenes. In Experiment 1, we used dynamic scenes only, whereas in Experiments 2 and 3, we directly compared dynamic and static scenes. Each scene contained a static, contextually irrelevant target (i.e., a gray annulus). Scene motion was not predictive of target location. For dynamic scenes, the search-time results from all three experiments converge on the novel finding that neither foveal nor central vision was necessary to attain normal search proficiency. Since motion is known to attract attention and gaze, we explored whether guidance to the target was equally efficient in dynamic as compared to static scenes. We found that the very first saccade was guided by motion in the scene. This was not the case for subsequent saccades made during the scanning epoch, representing the actual search process. Thus, effects of task-irrelevant motion were fast-acting and short-lived. Furthermore, when motion was potentially present (Spotlights) or absent (Blindspots) in foveal or central vision only, we observed differences in verification times for dynamic and static scenes (Experiment 2). When using scenes with greater visual complexity and more motion (Experiment 3), however, the differences between dynamic and static scenes were much reduced.

Eye movements elevate crowding in idiopathic infantile nystagmus syndrome.

Idiopathic infantile nystagmus syndrome is a disorder characterised by involuntary eye movements, which leads to decreased acuity and visual function. One such function is visual crowding – a process whereby objects that are easily recognised in isolation become impaired by nearby flankers. Crowding typically occurs in the peripheral visual field, although elevations in foveal vision have been reported in congenital nystagmus, similar to those found with amblyopia. Here, we examine whether elevated foveal crowding with nystagmus is driven by similar mechanisms to those of amblyopia – long-term neural changes associated with a sensory deficit – or by the momentary displacement of the stimulus through nystagmus eye movements. A Landolt-C orientation identification task was used to measure threshold gap sizes with and without either horizontally or vertically placed Landolt-C flankers. We assume that a sensory deficit should give equivalent crowding in these two dimensions, whereas an origin in eye movements should give stronger crowding with horizontal flankers given the predominantly horizontal eye movements of nystagmus. We observe elevations in nystagmic crowding that are above crowding in typical vision but below that of amblyopia. Consistent with an origin in eye movements, elevations were stronger with horizontal than vertical flankers in nystagmus, but not in typical or amblyopic vision. We further demonstrate the same horizontal elongation in typical vision with stimulus movement that simulates nystagmus. Consequently, we propose that the origin of nystagmic crowding lies in the eye movements, either through image smear of the target and flanker elements or through relocation of the stimulus into the peripheral retina.

A Mixed Statistical and Machine Learning Approach for the Analysis of Multimodal Trail Making Test Data

Eye-tracking can offer a novel clinical practice and a non-invasive tool to detect neuropathological syndromes. In this paper, we show some analysis on data obtained from the visual sequential search test. Indeed, such a test can be used to evaluate the capacity of looking at objects in a specific order, and its successful execution requires the optimization of the perceptual resources of foveal and extrafoveal vision. The main objective of this work is to detect if some patterns can be found within the data, to discern among people with chronic pain, extrapyramidal patients and healthy controls. We employed statistical tests to evaluate differences among groups, considering three novel indicators: blinking rate, average blinking duration and maximum pupil size variation. Additionally, to divide the three patient groups based on scan-path images—which appear very noisy and all similar to each other—we applied deep learning techniques to embed them into a larger transformed space. We then applied a clustering approach to correctly detect and classify the three cohorts. Preliminary experiments show promising results.

Binocular visual function and fixational control in patients with macular disease: A review.

For normally sighted observers, the centre of the macula—the fovea—provides the sharpest vision and serves as the reference point for the oculomotor system. Typically, healthy observers have precise oculomotor control and binocular visual performance that is superior to monocular performance. These functions are disturbed in patients with macular disease who lose foveal vision. An adaptation to central vision loss is the development of a preferred retinal locus (PRL) in the functional eccentric retina, which is determined with a fixation task during monocular viewing. Macular disease often affects the two eyes unequally, but its impact on binocular function and fixational control is poorly understood. Given that patients’ natural viewing condition is binocular, the aim of this article was to review current research on binocular visual function and fixational oculomotor control in macular disease. Our findings reveal that there is no overall binocular gain across a range of visual functions, although clear evidence exists for subgroups of patients who exhibit binocular summation or binocular inhibition, depending on the clinical characteristics of their two eyes. The monocular PRL of the better eye has different characteristics from that of the worse eye, but during binocular viewing the PRL of the better eye drives fixational control and may serve as the new reference position for the oculomotor system. We conclude that evaluating binocular function in patients with macular disease reveals important clinical aspects that otherwise cannot be determined solely from examining monocular functions, and can lead to better disease management and interventions.

Numerosity Perception in Peripheral Vision.

Peripheral vision has different functional priorities for mammals than foveal vision. One of its roles is to monitor the environment while central vision is focused on the current task. Becoming distracted too easily would be counterproductive in this perspective, so the brain should react to behaviourally relevant changes. Gist processing is good for this purpose, and it is therefore not surprising that evidence from both functional brain imaging and behavioural research suggests a tendency to generalize and blend information in the periphery. This may be caused by the balance of perceptual influence in the periphery between bottom-up (i.e., sensory information) and top-down (i.e., prior or contextual information) processing channels. Here, we investigated this interaction behaviourally using a peripheral numerosity discrimination task with top-down and bottom-up manipulations. Participants compared numerosity between the left and right peripheries of a screen. Each periphery was divided into a centre and a surrounding area, only one of which was a task relevant target region. Our top-down task modulation was the instruction which area to attend – centre or surround. We varied the signal strength by altering the stimuli durations i.e., the amount of information presented/processed (as a combined bottom-up and recurrent top-down feedback factor). We found that numerosity perceived in target regions was affected by contextual information in neighbouring (but irrelevant) areas. This effect appeared as soon as stimulus duration allowed the task to be reliably performed and persisted even at the longest duration (1 s). We compared the pattern of results with an ideal-observer model and found a qualitative difference in the way centre and surround areas interacted perceptually in the periphery. When participants reported on the central area, the irrelevant surround would affect the response as a weighted combination – consistent with the idea of a receptive field focused in the target area to which irrelevant surround stimulation leaks in. When participants report on surround, we can best describe the response with a model in which occasionally the attention switches from task relevant surround to task irrelevant centre – consistent with a selection model of two competing streams of information. Overall our results show that the influence of spatial context in the periphery is mandatory but task dependent.

Gaze-Contingent Retinal Speckle Suppression for Perceptually-Matched Foveated Holographic Displays.

Computer-generated holographic (CGH) displays show great potential and are emerging as the next-generation displays for augmented and virtual reality, and automotive heads-up displays. One of the critical problems harming the wide adoption of such displays is the presence of speckle noise inherent to holography, that compromises its quality by introducing perceptible artifacts. Although speckle noise suppression has been an active research area, the previous works have not considered the perceptual characteristics of the Human Visual System (HVS), which receives the final displayed imagery. However, it is well studied that the sensitivity of the HVS is not uniform across the visual field, which has led to gaze-contingent rendering schemes for maximizing the perceptual quality in various computer-generated imagery. Inspired by this, we present the first method that reduces the "perceived speckle noise" by integrating foveal and peripheral vision characteristics of the HVS, along with the retinal point spread function, into the phase hologram computation. Specifically, we introduce the anatomical and statistical retinal receptor distribution into our computational hologram optimization, which places a higher priority on reducing the perceived foveal speckle noise while being adaptable to any individual's optical aberration on the retina. Our method demonstrates superior perceptual quality on our emulated holographic display. Our evaluations with objective measurements and subjective studies demonstrate a significant reduction of the human perceived noise.

Sports athletes use predictive saccades! But why?

In sports, high-level athletes are able to predict the actions of an opposing player. Interestingly, such predictions are also reflected by the athlete’s gaze behaviour. In cricket, for example, a bowler bowls the ball in the direction of a batsman. The ball first bounces on the ground before the batsman tries to hit the ball with his bat. During the ball flight, players very often initiate two predictive saccades: one to the predicted ball-bounce point and a second to the predicted ball-bat-contact point. That means, they move their eyes ahead of the ball and “wait” for the ball at the new fixation location, potentially using their peripheral vision to update information about the ball’s trajectory. In this study, we investigated whether predictive saccades are linked to the processing of information in peripheral vision and if predictive saccades are superior to continuously following the ball with foveal vision using smooth-pursuit eye-movements (SPEMs). In four experiments (Experiments 1a, 1b, 2a, and 2b), we first compared speed-discrimination performance when using foveal vision (during SPEMs) or peripheral vision (during fixations). We found that speed discrimination performance was better during pursuit than during fixation. In the next two experiments (Experiment 3 and 4), we evoked the typical eye-movements observed in cricket. Results show that the information gathered during SPEMs is sufficient to estimate when the target will hit the fixated location, and that peripheral monitoring does not help or is not used to improve this estimation. Finally, in the last experiment (Experiment 6), we show that it may actually be beneficial to use SPEMs to predict the TTC of a moving target rather than predictive saccades. Thus, predictive saccades that move fixation ahead of a target are unlikely to be performed to enhance the ability to peripherally monitor a moving target.

The integrated Stiles-Crawford effect: understanding the role of pupil size and outer-segment length in foveal vision

The integrated Stiles-Crawford effect: understanding the role of pupil size and outer-segment length in foveal vision

Happy centre, happy whole: Foveal vision determines the perceived emotion of face ensembles

Happy centre, happy whole: Foveal vision determines the perceived emotion of face ensembles

Detection, Inspection and Re-Inspection: A functional approach to gaze behavior towards complex scenes

Traditional approaches examining fixations towards natural scenes either focus on their spatial distribution, or aim to explain their exact sequence. Whereas heat-maps potentially conflate fixations serving different perceptual functions, scan-paths may aim for an unnecessary level of detail. Here, we introduce a novel way of analysing fixations. We hypothesize that fixations mainly serve three goals: Detection, Inspection and Re-Inspection of visual objects. Detection (D) for foveal vision is achieved by the first fixation landing on a given object, Inspection (I) by successive scanning immediately after Detection, and Re-Inspection (R) following the intermediate exploration of other objects. N = 101 participants freely viewed 700 images of everyday scenes. To test consistency within and differences between D, I and R fixations, we generated separate fixation maps for each category and image, separately for two subsets of observers. Results indicate large consistencies for D, I and R fixations across observers, with maps sharing 95%, 93% and 88% of their variance on average. At the same time, the shared variance between D, I and R was only 65% on average. To explore the nature of these differences, we computed the proportions of D, I and R fixations landing on different types of objects and found they could vary substantially. For instance, 57% of Inspection, but only 34% of Detection fixations landed on Text. Finally, we probed individual differences in the proportions of dwell time allocated to D, I and R. These individual biases towards D, I or R were both, considerable and highly consistent across images (all r > .96). Taken together, the DIR model can capture distinct aspects of fixation behavior on an observer-general and individual level. We propose it as a useful level of description for analysing and understanding the diverse perceptual functions of gaze behavior.

The impact of spectacle lenses for myopia control on visual functions.

PurposeSpectacle lenses containing multiple small peripheral elements have been developed for myopia control in children. It is important that their effect on vision be quantified by (i) fixation through the peripheral portion, thereby using foveal vision and (ii) by fixation through the central portion and presentation of peripheral targets.MethodsThe above approaches were used in five studies to evaluate two novel spectacle lens designs: spectacle lenses with Highly Aspherical Lenslets (HAL) and Slightly Aspherical Lenslets (SAL). A single vision lens served as a control. Visually normal adults participated in each study. The first two studies had subjects fixate through the periphery of the lenses. High and low (10%) contrast visual acuity was measured with the Freiburg Vision Test and reading speed for high and low contrast words measured with a sentence generator. The other three studies assessed peripheral vision while subjects fixated through the central portion of the lens. Peripheral contrast sensitivity was measured using two cycles per degree drifting Gabor stimuli. Peripheral motion perception was further evaluated using random dot stimuli. Finally, attention was measured using an established test of useful field of view with three levels of complexity.ResultsThe periphery of the HAL lens significantly reduced low contrast visual acuity, but not high contrast visual acuity, while the effect of the SAL lens was not significant for either. Neither test lens affected reading speed for high contrast words, but the HAL lens significantly affected performance for low contrast words. Neither test lens affected peripheral motion perception or useful field of view.ConclusionsLow contrast visual acuity and reading was slightly reduced while high contrast visual acuity was unaffected when fixating through the periphery of the novel lens designs. None of the peripheral measures of vision was affected by the novel lens designs.

Fast and nonuniform dynamics of perisaccadic vision in the central fovea

Humans use rapid eye movements (saccades) to inspect stimuli with the foveola, the region of the retina where receptors are most densely packed. It is well established that visual sensitivity is generally attenuated during these movements, a phenomenon known as saccadic suppression. This effect is commonly studied with large, often peripheral, stimuli presented during instructed saccades. However, little is known about how saccades modulate the foveola and how the resulting dynamics unfold during natural visual exploration. Here we measured the foveal dynamics of saccadic suppression in a naturalistic high-acuity task, a task designed after primates' social grooming, which-like most explorations of fine patterns-primarily elicits minute saccades (microsaccades). Leveraging on recent advances in gaze-contingent display control, we were able to systematically map the perisaccadic time course of sensitivity across the foveola. We show that contrast sensitivity is not uniform across this region and that both the extent and dynamics of saccadic suppression vary within the foveola. Suppression is stronger and faster in the most central portion, where sensitivity is generally higher and selectively rebounds at the onset of a new fixation. These results shed light on the modulations experienced by foveal vision during the saccade-fixation cycle and explain some of the benefits of microsaccades.

Foveal vision

William Tuten and Wolf Harmening introduce the anatomical and functional signatures of foveated vision in humans.

Object-Gaze Distance: Quantifying Near- Peripheral Gaze Behavior in Real-World Applications.

Eye tracking (ET) has shown to reveal the wearer’s cognitive processes using the measurement of the central point of foveal vision. However, traditional ET evaluation methods have not been able to take into account the wearers’ use of the peripheral field of vision. We propose an algorithmic enhancement to a state-of-the-art ET analysis method, the Object- Gaze Distance (OGD), which additionally allows the quantification of near-peripheral gaze behavior in complex real-world environments. The algorithm uses machine learning for area of interest (AOI) detection and computes the minimal 2D Euclidean pixel distance to the gaze point, creating a continuous gaze-based time-series. Based on an evaluation of two AOIs in a real surgical procedure, the results show that a considerable increase of interpretable fixation data from 23.8 % to 78.3 % of AOI screw and from 4.5 % to 67.2 % of AOI screwdriver was achieved, when incorporating the near-peripheral field of vision. Additionally, the evaluation of a multi-OGD time series representation has shown the potential to reveal novel gaze patterns, which may provide a more accurate depiction of human gaze behavior in multi-object environments.

Modeling Mental Qualities

Conscious experiences are characterized by mental qualities, such as those involved in seeing red, feeling pain, or smelling cinnamon. The standard framework for modeling mental qualities represents them via points in multidimensional spaces, where distances between points inversely correspond to degrees of phenomenal similarity. This article argues that the standard framework is structurally inadequate and develops a new framework that is more powerful and flexible. The core problem for the standard framework is that it cannot capture precision structure: for example, consider the phenomenal contrast between seeing an object as crimson in foveal vision versus merely as red in peripheral vision. The solution the article proposes is to model mental qualities using regions, rather than points. The article explains how this seemingly simple formal innovation not only provides a natural way of modeling precision but also yields a variety of further theoretical fruits: it enables one to formulate novel hypotheses about the space and structures of mental qualities, formally differentiate two dimensions of phenomenal similarity, generate a probabilistic model of the phenomenal sorites, and deploy a new theoretical tool in the empirical investigation of consciousness. A noteworthy consequence of this new framework is that the structure of the mental qualities of conscious experiences is fundamentally different from the structure of the perceptible qualities of external objects.

Temporal dynamics of neural activity in macaque frontal cortex assessed with large-scale recordings

The cortical network controlling the arm and hand when grasping objects consists of several areas in parietal and frontal cortex. Recently, more anterior prefrontal areas have also been implicated in object grasping, but their exact role is currently unclear. To investigate the neuronal encoding of objects during grasping in these prefrontal regions and their relation with other cortical areas of the grasping network, we performed large-scale recordings (more than 2000 responsive sites) in frontal cortex of monkeys during a saccade-reach-grasp task. When an object appeared in peripheral vision, the first burst of activity emerged in prearcuate areas (the FEF and area 45B), followed by dorsal and ventral premotor cortex, and a buildup of activity in primary motor cortex. After the saccade, prearcuate activity remained elevated while primary motor and premotor activity rose in anticipation of the upcoming arm and hand movement. Remarkably, a large number of premotor and prearcuate sites responded when the object appeared in peripheral vision and remained active when the object came into foveal vision. Thus, prearcuate and premotor areas continuously encode object information when directing gaze and grasping objects.