Retinal Cues Research Articles

Modeling collective behaviors from optic flow and retinal cues

Modeling collective behaviors from optic flow and retinal cues

The contribution of semantic distance knowledge to size constancy in perception and grasping when visual cues are limited

To achieve a stable perception of object size in spite of variations in viewing distance, our visual system needs to combine retinal image information and distance cues. Previous research has shown that, not only retinal cues, but also extraretinal sensory signals can provide reliable information about depth and that different neural networks (perception versus action) can exhibit preferences in the use of these different sources of information during size-distance computations. Semantic knowledge of distance, a purely cognitive signal, can also provide distance information. Do the perception and action systems show differences in their ability to use this information in calculating object size and distance? To address this question, we presented ‘glow-in-the-dark’ objects of different physical sizes at different real distances in a completely dark room. Participants viewed the objects monocularly through a 1-mm pinhole. They either estimated the size and distance of the objects or attempted to grasp them. Semantic knowledge was manipulated by providing an auditory cue about the actual distance of the object: “20 cm”, “30 cm”, and “40 cm”. We found that semantic knowledge of distance contributed to some extent to size constancy operations during perceptual estimation and grasping, but size constancy was never fully restored. Importantly, the contribution of knowledge about distance to size constancy was equivalent between perception and action. Overall, our study reveals similarities and differences between the perception and action systems in the use of semantic distance knowledge and suggests that this cognitive signal is useful but not a reliable depth cue for size constancy under restricted viewing conditions.

Microfluidic and Microscale Assays to Examine Regenerative Strategies in the Neuro Retina.

Bioengineering systems have transformed scientific knowledge of cellular behaviors in the nervous system (NS) and pioneered innovative, regenerative therapies to treat adult neural disorders. Microscale systems with characteristic lengths of single to hundreds of microns have examined the development and specialized behaviors of numerous neuromuscular and neurosensory components of the NS. The visual system is comprised of the eye sensory organ and its connecting pathways to the visual cortex. Significant vision loss arises from dysfunction in the retina, the photosensitive tissue at the eye posterior that achieves phototransduction of light to form images in the brain. Retinal regenerative medicine has embraced microfluidic technologies to manipulate stem-like cells for transplantation therapies, where de/differentiated cells are introduced within adult tissue to replace dysfunctional or damaged neurons. Microfluidic systems coupled with stem cell biology and biomaterials have produced exciting advances to restore vision. The current article reviews contemporary microfluidic technologies and microfluidics-enhanced bioassays, developed to interrogate cellular responses to adult retinal cues. The focus is on applications of microfluidics and microscale assays within mammalian sensory retina, or neuro retina, comprised of five types of retinal neurons (photoreceptors, horizontal, bipolar, amacrine, retinal ganglion) and one neuroglia (Müller), but excludes the non-sensory, retinal pigmented epithelium.

Adaptive heading performance during self-motion perception.

Previous studies have documented that the perception of self-motion direction can be extracted from the patterns of image motion on the retina (also termed optic flow). Self-motion perception remains stable even when the optic-flow information is distorted by concurrent gaze shifts from body/eye rotations. This has been interpreted that extraretinal signals-efference copies of eye/body movements-are involved in compensating for retinal distortions. Here, we tested an alternative hypothesis to the extraretinal interpretation. We hypothesized that accurate self-motion perception can be achieved from a purely optic-flow-based visual strategy acquired through experience, independent of extraretinal mechanism. To test this, we asked human subjects to perform a self-motion direction discrimination task under normal optic flow (fixation condition) or distorted optic flow resulted from either realistic (pursuit condition) or simulated (simulated condition) eye movements. The task was performed either without (pre- and posttraining) or with (during training) the feedback about the correct answer. We first replicated the previous observation that before training, direction perception was greatly impaired in the simulated condition where the optic flow was distorted and extraretinal eye movement signals were absent. We further showed that after a few training sessions, the initial impairment in direction perception was gradually improved. These results reveal that behavioral training can enforce the exploitation of retinal cues to compensate for the distortion, without the contribution from the extraretinal signals. Our results suggest that self-motion perception is a flexible and adaptive process which might depend on neural plasticity in relevant cortical areas.

Misperception of motion in depth originates from an incomplete transformation of retinal signals.

Depth perception requires the use of an internal model of the eye-head geometry to infer distance from binocular retinal images and extraretinal 3D eye-head information, particularly ocular vergence. Similarly, for motion in depth perception, gaze angle is required to correctly interpret the spatial direction of motion from retinal images; however, it is unknown whether the brain can make adequate use of extraretinal version and vergence information to correctly transform binocular retinal motion into 3D spatial coordinates. Here we tested this hypothesis by asking participants to reconstruct the spatial trajectory of an isolated disparity stimulus moving in depth either peri-foveally or peripherally while participants' gaze was oriented at different vergence and version angles. We found large systematic errors in the perceived motion trajectory that reflected an intermediate reference frame between a purely retinal interpretation of binocular retinal motion (not accounting for veridical vergence and version) and the spatially correct motion. We quantify these errors with a 3D reference frame model accounting for target, eye, and head position upon motion percept encoding. This model could capture the behavior well, revealing that participants tended to underestimate their version by up to 17%, overestimate their vergence by up to 22%, and underestimate the overall change in retinal disparity by up to 64%, and that the use of extraretinal information depended on retinal eccentricity. Since such large perceptual errors are not observed in everyday viewing, we suggest that both monocular retinal cues and binocular extraretinal signals are required for accurate real-world motion in depth perception.

Retinal Stabilization Reveals Limited Influence of Extraretinal Signals on Heading Tuning in the Medial Superior Temporal Area.

Heading perception in primates depends heavily on visual optic-flow cues. Yet during self-motion, heading percepts remain stable, even though smooth-pursuit eye movements often distort optic flow. According to theoretical work, self-motion can be represented accurately by compensating for these distortions in two ways: via retinal mechanisms or via extraretinal efference-copy signals, which predict the sensory consequences of movement. Psychophysical evidence strongly supports the efference-copy hypothesis, but physiological evidence remains inconclusive. Neurons that signal the true heading direction during pursuit are found in visual areas of monkey cortex, including the dorsal medial superior temporal area (MSTd). Here we measured heading tuning in MSTd using a novel stimulus paradigm, in which we stabilize the optic-flow stimulus on the retina during pursuit. This approach isolates the effects on neuronal heading preferences of extraretinal signals, which remain active while the retinal stimulus is prevented from changing. Our results from 3 female monkeys demonstrate a significant but small influence of extraretinal signals on the preferred heading directions of MSTd neurons. Under our stimulus conditions, which are rich in retinal cues, we find that retinal mechanisms dominate physiological corrections for pursuit eye movements, suggesting that extraretinal cues, such as predictive efference-copy mechanisms, have a limited role under naturalistic conditions.SIGNIFICANCE STATEMENT Sensory systems discount stimulation caused by an animal's own behavior. For example, eye movements cause irrelevant retinal signals that could interfere with motion perception. The visual system compensates for such self-generated motion, but how this happens is unclear. Two theoretical possibilities are a purely visual calculation or one using an internal signal of eye movements to compensate for their effects. The latter can be isolated by experimentally stabilizing the image on a moving retina, but this approach has never been adopted to study motion physiology. Using this method, we find that extraretinal signals have little influence on activity in visual cortex, whereas visually based corrections for ongoing eye movements have stronger effects and are likely most important under real-world conditions.

Functional Neuroanatomy of Time-To-Passage Perception

The time until an approaching object passes the observer is referred to as time-to-passage (TTP). Accurate judgment of TTP is critical for visually guided navigation, such as when walking, riding a bicycle, or driving a car. Previous research has shown that observers are able to make TTP judgments in the absence of information about local retinal object expansion. In this paper we combine psychophysics and functional MRI (fMRI) to investigate the neural substrate of TTP processing. In a previous psychophysical study, we demonstrated that when local retinal expansion cues are not available, observers take advantage of multiple sources of information to judge TTP, such as optic flow and object retinal velocities, and integrate these cues through a flexible and economic strategy. To induce strategy changes, we introduced trials with motion but without coherent optic flow (0% coherence of the background), and trials with coherent, but noisy, optic flow (75% coherence of the background). In a functional magnetic resonance imaging (fMRI) study we found that coherent optic flow cues resulted in better behavioral performance as well as higher and broader cortical activations across the visual motion processing pathway. Blood oxygen-level-dependent (BOLD) signal changes showed significant involvement of optic flow processing in the precentral sulcus (PreCS), postcentral sulcus (PostCS) and middle temporal gyrus (MTG) across all conditions. Not only highly activated during motion processing, bilateral hMT areas also showed a complex pattern in TTP judgment processing, which reflected a flexible TTP response strategy.

Peripheral Visual Cues Contribute to the Perception of Object Movement During Self-Movement.

Safe movement through the environment requires us to monitor our surroundings for moving objects or people. However, identification of moving objects in the scene is complicated by self-movement, which adds motion across the retina. To identify world-relative object movement, the brain thus has to ‘compensate for’ or ‘parse out’ the components of retinal motion that are due to self-movement. We have previously demonstrated that retinal cues arising from central vision contribute to solving this problem. Here, we investigate the contribution of peripheral vision, commonly thought to provide strong cues to self-movement. Stationary participants viewed a large field of view display, with radial flow patterns presented in the periphery, and judged the trajectory of a centrally presented probe. Across two experiments, we demonstrate and quantify the contribution of peripheral optic flow to flow parsing during forward and backward movement.

Potential Systematic Interception Errors are Avoided When Tracking the Target with One\u2019s Eyes

Directing our gaze towards a moving target has two known advantages for judging its trajectory: the spatial resolution with which the target is seen is maximized, and signals related to the eyes’ movements are combined with retinal cues to better judge the target’s motion. We here explore whether tracking a target with one’s eyes also prevents factors that are known to give rise to systematic errors in judging retinal speeds from resulting in systematic errors in interception. Subjects intercepted white or patterned disks that moved from left to right across a large screen at various constant velocities while either visually tracking the target or fixating the position at which they were required to intercept the target. We biased retinal motion perception by moving the pattern within the patterned targets. This manipulation led to large systematic errors in interception when subjects were fixating, but not when they were tracking the target. The reduction in the errors did not depend on how smoothly the eyes were tracking the target shortly before intercepting it. We propose that tracking targets with one’s eyes when one wants to intercept them makes one less susceptible to biases in judging their motion.

Visually guided adjustments of body posture in the roll plane

Body position relative to gravity is continuously updated to prevent falls. Therefore, the brain integrates input from the otoliths, truncal graviceptors, proprioception and vision. Without visual cues estimated direction of gravity mainly depends on otolith input and becomes more variable with increasing roll-tilt. Contrary, the discrimination threshold for object orientation shows little modulation with varying roll orientation of the visual stimulus. Providing earth-stationary visual cues, this retinal input may be sufficient to perform self-adjustment tasks successfully, with resulting variability being independent of whole-body roll orientation. We compared conditions with informative (earth-fixed) and non-informative (body-fixed) visual cues. If the brain uses exclusively retinal input (if earth-stationary) to solve the task, trial-to-trial variability will be independent from the subject's roll orientation. Alternatively, central integration of both retinal (earth-fixed) and extra-retinal inputs will lead to increasing variability when roll-tilted. Subjects, seated on a motorized chair, were instructed to (1) align themselves parallel to an earth-fixed line oriented earth-vertical or roll-tilted 75° clockwise; (2) move a body-fixed line (aligned with the body-longitudinal axis or roll-tilted 75° counter-clockwise to it) by adjusting their body position until the line was perceived earth-vertical. At 75° right-ear-down position, variability increased significantly (p<0.05) compared to upright in both paradigms, suggesting that, despite the earth-stationary retinal cues, extra-retinal input is integrated. Self-adjustments in the roll-tilted position were significantly (p<0.01) more precise for earth-fixed cues than for body-fixed cues, underlining the importance of earth-stable visual cues when estimates of gravity become more variable with increasing whole-body roll.

A Cholinergic Mechanism for Reward Timing within Primary Visual Cortex

Neurons in rodent primary visual cortex (V1) relate operantly conditioned stimulus-reward intervals with modulated patterns of spiking output, but little is known about the locus or mechanism of this plasticity. Here we show that cholinergic basal forebrain projections to V1 are necessary for the neural acquisition, but not the expression, of reward timing in the visual cortex of awake, behaving animals. We then mimic reward timing invitro by pairing white matter stimulation with muscarinic receptor activation at a fixed interval and show that this protocol results in the prolongation of electrically evoked spike train durations out to the conditioned interval. Together, these data suggest that V1 possesses the circuitry and plasticity to support reward time prediction learning and the cholinergic system serves as an important reinforcement signal which, invivo, conveys to the cortex the outcome of behavior.

Misperception of aspect ratio in binocularly viewed surfaces

The horizontal–vertical illusion, in which the vertical dimension is overestimated relative to the horizontal direction, has been explained in terms of the statistical relationship between the lengths of lines in the world, and the lengths of their projections onto the retina (Howe & Purves, 2002). The current study shows that this illusion affects the apparent aspect ratio of shapes, and investigates how it interacts with binocular cues to surface slant. One way in which statistical information could give rise to the horizontal–vertical illusion would be through prior assumptions about the distribution of slant. This prior would then be expected to interact with retinal cues to slant. We determined the aspect ratio of stereoscopically viewed ellipses that appeared circular. We show that observers’ judgements of aspect ratio were affected by surface slant, but that the largest image vertical:horizontal aspect ratio that was considered to be a surface with a circular profile was always found for surfaces close to fronto-parallel. This is not consistent with a Bayesian model in which the horizontal–vertical illusion arises from a non-uniform prior probability distribution for slant. Rather, we suggest that assumptions about the slant of surfaces affect apparent aspect ratio in a manner that is more heuristic, and partially dissociated from apparent slant.

How we perceive the trajectory of an approaching object

Various equations that describe how observers could recover the trajectory of an approaching object have been put forward. Many are relatively complex formulations that recover the veridical trajectory by scaling retinal cues, such as looming and changing disparity. However, these equations do not seem to describe human perception as observers typically misjudge trajectory angles. Thus, we examine whether a simpler formulation--one that does not predict veridical judgments-may better explain performance. We test the hypothesis that perceived trajectory is based on a speed ratio: the ratio of lateral angular speed to the sum of looming and changing disparity signals. To discriminate between this and alternative proposals, we examined the effect of object size on trajectory perception: The speed ratio hypothesis predicts that perceived trajectory will become less eccentric with increasing object size, while the alternatives predict that perceived trajectory will be independent of object size. Observers performed a trajectory judgment task in which they compared the trajectory direction of two approaching objects, of the same or different size, seen in separate intervals. We estimated perceptually parallel trajectories from their responses. In Experiment 1, objects differed in horizontal and vertical size, and in Experiment 2, they differed only in vertical size. In both experiments, observers' data showed a clear effect of object size and were close to predictions of the speed ratio hypothesis. We conclude that the alternate proposals we tested were not supported and that the speed ratio account is a sufficient account of the data.

Acquisition of multiple prior distributions in tactile temporal order judgment

The Bayesian estimation theory proposes that the brain acquires the prior distribution of a task and integrates it with sensory signals to minimize the effect of sensory noise. Psychophysical studies have demonstrated that our brain actually implements Bayesian estimation in a variety of sensory-motor tasks. However, these studies only imposed one prior distribution on participants within a task period. In this study, we investigated the conditions that enable the acquisition of multiple prior distributions in temporal order judgment of two tactile stimuli across the hands. In Experiment 1, stimulation intervals were randomly selected from one of two prior distributions (biased to right hand earlier and biased to left hand earlier) in association with color cues (green and red, respectively). Although the acquisition of the two priors was not enabled by the color cues alone, it was significant when participants shifted their gaze (above or below) in response to the color cues. However, the acquisition of multiple priors was not significant when participants moved their mouths (opened or closed). In Experiment 2, the spatial cues (above and below) were used to identify which eye position or retinal cue position was crucial for the eye-movement-dependent acquisition of multiple priors in Experiment 1. The acquisition of the two priors was significant when participants moved their gaze to the cues (i.e., the cue positions on the retina were constant across the priors), as well as when participants did not shift their gazes (i.e., the cue positions on the retina changed according to the priors). Thus, both eye and retinal cue positions were effective in acquiring multiple priors. Based on previous neurophysiological reports, we discuss possible neural correlates that contribute to the acquisition of multiple priors.

Development of eye-specific projections to the lateral geniculate nucleus

How do precise circuits form in the mammalian CNS? Eye-specific retino-LGN projections are a prominent model pathway for exploring this question and in particular, for exploring the role of neuronal activity in synaptogenesis and axonal refinement. Previous work has shown that eliminating spontaneous retinal activity can alter both the normal segregation and the maintenance of eye-specific retino-LGN projections. Whether the pattern of retinal activity is important for segregation, however, remains the focus of intense investigation. We addressed the general requirement for retinal cues in eye-specific retino-LGN segregation and maintenance in mutant mice that exhibit delayed axonal degeneration (Wallerian degeneration slow; Wlds mice). The Wlds mouse allows for complete eye removal (enucleation) and thus all retinal cues, without the normal accompanying loss of ganglion cell axons. We find that eye-specific pathways are formed and maintained normally in non-enucleated Wlds mice. The results of combined tracing/enucleation experiments in Wlds mice support the hypothesis that the retina is a source of instructive signals for the initial eye-specific segregation process and for maintaining eye-specific circuits in the period immediately following their segregation. Over time, however, retinal cues become decreasingly important, and eventually retinal cues are not required at all for maintenance of eye-specific connections in the LGN.

Extra-retinal signals support the estimation of 3D motion

In natural settings, our eyes tend to track approaching objects. To estimate motion, the brain should thus take account of eye movements, perhaps using retinal cues (retinal slip of static objects) or extra-retinal signals (motor commands). Previous work suggests that extra-retinal ocular vergence signals do not support the perceptual judgments. Here, we re-evaluate this conclusion, studying motion judgments based on retinal slip and extra-retinal signals. We find that (1) each cue can be sufficient, and, (2) retinal and extra-retinal signals are combined, when estimating motion-in-depth. This challenges the accepted view that observers are essentially blind to eye vergence changes.

Translation Speed Compensation in the Dorsal Aspect of the Medial Superior Temporal Area

The dorsal aspect of the medial superior temporal area (MSTd) is involved in the computation of heading direction from the focus of expansion (FOE) of the visual image. Our laboratory previously found that MSTd neurons adjust their focus tuning curves to compensate for shifts in the FOE produced by eye rotation (Bradley et al., 1996) as well as for changes in pursuit speed (Shenoy et al., 2002). The translation speed of an observer also affects the shift of the FOE. To investigate whether MSTd neurons can adjust their focus tuning curves to compensate for varying translation speeds, we recorded extracellular responses from 93 focus-tuned MSTd neurons in two rhesus monkeys (Macaca mulatta) performing pursuit eye movements across displays of varying translation speeds. We found that MSTd neurons had larger shifts in their tuning curves for slow translation speeds and smaller shifts for fast translation speeds. These shifts aligned the focus tuning curves with the true heading direction and not with the retinal position of the FOE. Because the eye was pursuing at the same rate for varying translation speeds, these results indicate that retinal cues related both to translation speed and extraretinal signals from pursuit eye movements are used by MSTd neurons to compute heading direction.

Pursuit speed compensation in cortical area MSTd.

When we move forward the visual images on our retinas expand. Humans rely on the focus, or center, of this expansion to estimate their direction of self-motion or heading and, as long as the eyes are still, the retinal focus corresponds to the heading. However, smooth pursuit eye movements add visual motion to the expanding retinal image and displace the focus of expansion. In spite of this, humans accurately judge their heading during pursuit eye movements even though the retinal focus no longer corresponds to the heading. Recent studies in macaque suggest that correction for pursuit may occur in the dorsal aspect of the medial superior temporal area (MSTd); neurons in this area are tuned to the retinal position of the focus and they modify their tuning to partially compensate for the focus shift caused by pursuit. However, the question remains whether these neurons shift focus tuning more at faster pursuit speeds, to compensate for the larger focus shifts created by faster pursuit. To investigate this question, we recorded from 40 MSTd neurons while monkeys made pursuit eye movements at a range of speeds across simulated self- or object motion displays. We found that most MSTd neurons modify their focus tuning more at faster pursuit speeds, consistent with the idea that they encode heading and other motion parameters regardless of pursuit speed. Across the population, the median rate of compensation increase with pursuit speed was 51% as great as required for perfect compensation. We recorded from the same neurons in a simulated pursuit condition, in which gaze was fixed but the entire display counter-rotated to produce the same retinal image as during real pursuit. This condition yielded the result that retinal cues contribute to pursuit compensation; the rate of compensation increase was 30% of that required for accurate encoding of heading. The difference between these two conditions was significant (P < 0.05), indicating that extraretinal cues also contribute significantly. We found a systematic antialignment between preferred pursuit and preferred visual motion directions. Neurons may use this antialignment to combine retinal and extraretinal compensatory cues. These results indicate that many MSTd neurons compensate for pursuit velocity, pursuit direction as previously reported and pursuit speed, and further implicate MSTd as a critical stage in the computation of egomotion.

Ordinal depth information from accommodation?

The ability to judge egocentric distance was assessed in two groups of six observers using a manual pointing task. The purpose of the study was to determine the extent to which blur-driven accommodation can provide information on target distance in the absence of any retinal cues to distance. Observers were extremely accurate when carrying out the pointing task in a ‘fullcue’ condition. In contrast, observers were extremely poor at carrying out the task when accommodation was the only distance cue available. Responses on individual trials bore little relationship to the actual target distance in any of the observers. On the other hand, accommodation weakly biased the mean responses in some observers. This bias appears to be due to the observers' effective use of accommodation to determine whether the target presented in one trial was nearer or further away than the target presented in the previous trial. Accommodation therefore appears to provide ordinal information, although the distance signal may actually arise from accommodation-driven vergence. The poverty of accommodation as a source of metric information was highlighted in a second group of observers who all demonstrated a strong bias when perceiving distance in the presence of an initially ambiguous retinal cue. It is concluded that accommodation can act as a source of ordinal distance information in the absence of other cues to distance but the contribution of accommodation to normal distance perception in full-cue conditions is questioned.

The contribution of vision and proprioception to judgements of finger proximity.

We sought to determine whether an increase in judged egocentric distance created by increasing vergence-specified distance would be negated when participants pointed at their own finger. It was found that ocular position dominates limb proprioception in the judgement of finger distance in the sagittal plane when vision is available. In contrast, an increase in perceived egocentric distance was largely attenuated by the presence of limb proprioception in reduced visual cue conditions. We conclude that the relative contribution of vergence to perceived distance depends upon the strength of the vergence effort signal when there are other cues present. Furthermore, if the distance percept includes a major contribution from retinal cues, then the visual component will dominate the limb proprioception component. If the visual component is largely determined by vergence information, limb proprioception will make a significant contribution and actually dominate when the vergence effort signal is weak. The results extend previous studies that have found a similar relationship between ocular position and limb proprioception in the perception of a finger's location in the coronal plane.