Retinal Motion Signals Research Articles

Predictive posture stabilization before contact with moving objects: equivalence of smooth pursuit tracking and peripheral vision.

Postural stabilization is essential to effectively interact with our environment. Humans preemptively adjust their posture to counteract impending disturbances, such as those encountered during interactions with moving objects, a phenomenon known as anticipatory postural adjustments (APAs). APAs are thought to be influenced by predictive models that incorporate object motion via retinal motion and extraretinal signals. Building on our previous work that examined APAs in relation to the perceived momentum of moving objects, here we explored the impact of object motion within different visual field sectors on the human capacity to anticipate motion and prepare APAs for contact between virtual moving objects and the limb. Participants interacted with objects moving toward them under different gaze conditions. In one condition, participants fixated on either a central point (central fixation) or left-right of the moving object (peripheral fixation), whereas in another, they followed the moving object with smooth pursuit eye movements (SPEMs). We found that APAs had the smallest magnitude in the central fixation condition and that no notable differences in APAs were apparent between the SPEM and peripheral fixation conditions. This suggests that the visual system can accurately perceive motion of objects in peripheral vision for posture stabilization. Using Bayesian model averaging, we also evaluated the contribution of different gaze variables, such as eye velocity and gain (ratio of eye and object velocity) and showed that both eye velocity and gain signals were significant predictors of APAs. Taken together, our study underscores the roles of oculomotor signals in the modulation of APAs.NEW & NOTEWORTHY We show that the human visuomotor system can detect motion in peripheral vision and make anticipatory adjustments to posture before contact with moving objects, just as effectively as when the eye movement system tracks those objects with smooth pursuit eye movements. These findings pave the way for research into how age-induced changes in spatial vision, eye movements, and motion perception could affect the control of limb movements and postural stability during motion-mediated interactions with objects.

Retinal motion statistics during natural locomotion.

Walking through an environment generates retinal motion, which humans rely on to perform a variety of visual tasks. Retinal motion patterns are determined by an interconnected set of factors, including gaze location, gaze stabilization, the structure of the environment, and the walker's goals. The characteristics of these motion signals have important consequences for neural organization and behavior. However, to date, there are no empirical in situ measurements of how combined eye and body movements interact with real 3D environments to shape the statistics of retinal motion signals. Here, we collect measurements of the eyes, the body, and the 3D environment during locomotion. We describe properties of the resulting retinal motion patterns. We explain how these patterns are shaped by gaze location in the world, as well as by behavior, and how they may provide a template for the way motion sensitivity and receptive field properties vary across the visual field.

Identifying cortical areas that underlie the transformation from 2D retinal to 3D head-centric motion signals

Accurate motion perception requires that the visual system integrate the 2D retinal motion signals received by the two eyes into a single representation of 3D motion. However, most experimental paradigms present the same stimulus to the two eyes, signaling motion limited to a 2D fronto-parallel plane. Such paradigms are unable to dissociate the representation of 3D head-centric motion signals (i.e., 3D object motion relative to the observer) from the associated 2D retinal motion signals. Here, we used stereoscopic displays to present separate motion signals to the two eyes and examined their representation in visual cortex using fMRI. Specifically, we presented random-dot motion stimuli that specified various 3D head-centric motion directions. We also presented control stimuli, which matched the motion energy of the retinal signals, but were inconsistent with any 3D motion direction. We decoded motion direction from BOLD activity using a probabilistic decoding algorithm. We found that 3D motion direction signals can be reliably decoded in three major clusters in the human visual system. Critically, in early visual cortex (V1-V3), we found no significant difference in decoding performance between stimuli specifying 3D motion directions and the control stimuli, suggesting that these areas represent the 2D retinal motion signals, rather than 3D head-centric motion itself. In voxels in and surrounding hMT and IPS0 however, decoding performance was consistently superior for stimuli that specified 3D motion directions compared to control stimuli. Our results reveal the parts of the visual processing hierarchy that are critical for the transformation of retinal into 3D head-centric motion signals and suggest a role for IPS0 in their representation, in addition to its sensitivity to 3D object structure and static depth.

Neural bases of self- and object-motion in a naturalistic vision.

To plan movements toward objects our brain must recognize whether retinal displacement is due to self‐motion and/or to object‐motion. Here, we aimed to test whether motion areas are able to segregate these types of motion. We combined an event‐related functional magnetic resonance imaging experiment, brain mapping techniques, and wide‐field stimulation to study the responsivity of motion‐sensitive areas to pure and combined self‐ and object‐motion conditions during virtual movies of a train running within a realistic landscape. We observed a selective response in MT to the pure object‐motion condition, and in medial (PEc, pCi, CSv, and CMA) and lateral (PIC and LOR) areas to the pure self‐motion condition. Some other regions (like V6) responded more to complex visual stimulation where both object‐ and self‐motion were present. Among all, we found that some motion regions (V3A, LOR, MT, V6, and IPSmot) could extract object‐motion information from the overall motion, recognizing the real movement of the train even when the images remain still (on the screen), or moved, because of self‐movements. We propose that these motion areas might be good candidates for the “flow parsing mechanism,” that is the capability to extract object‐motion information from retinal motion signals by subtracting out the optic flow components.

Decoding of retinal motion signals by cells in macaque MT

Decoding of retinal motion signals by cells in macaque MT

Real-motion signals in human early visual cortex

Eye movements induce visual motion that can complicate the stable perception of the world. The visual system compensates for such self-induced visual motion by integrating visual input with efference copies of eye movement commands. This mechanism is central as it does not only support perceptual stability but also mediates reliable perception of world-centered objective motion. In humans, it remains elusive whether visual motion responses in early retinotopic cortex are driven by objective motion or by retinal motion associated with it. To address this question, we used fMRI to examine functional responses of sixteen visual areas to combinations of planar objective motion and pursuit eye movements. Observers were exposed to objective motion that was faster, matched or slower relative to pursuit, allowing us to compare conditions that differed in objective motion velocity while retinal motion and eye movement signals were matched. Our results show that not only higher level motion regions such as V3A and V6, but also early visual areas signaled the velocity of objective motion, hence the product of integrating retinal with non-retinal signals. These results shed new light on mechanisms that mediate perceptual stability and real-motion perception, and show that extra-retinal signals related to pursuit eye movements influence processing in human early visual cortex.

Systematic misperceptions of 3-D motion explained by Bayesian inference.

People make surprising but reliable perceptual errors. Here, we provide a unified explanation for systematic errors in the perception of three-dimensional (3-D) motion. To do so, we characterized the binocular retinal motion signals produced by objects moving through arbitrary locations in 3-D. Next, we developed a Bayesian model, treating 3-D motion perception as optimal inference given sensory noise in the measurement of retinal motion. The model predicts a set of systematic perceptual errors, which depend on stimulus distance, contrast, and eccentricity. We then used a virtual-reality headset as well as a standard 3-D desktop stereoscopic display to test these predictions in a series of perceptual experiments. As predicted, we found evidence that errors in 3-D motion perception depend on the contrast, viewing distance, and eccentricity of a stimulus. These errors include a lateral bias in perceived motion direction and a surprising tendency to misreport approaching motion as receding and vice versa. In sum, we present a Bayesian model that provides a parsimonious account for a range of systematic misperceptions of motion in naturalistic environments.

Distinct fMRI Responses to Self-Induced versus Stimulus Motion during Free Viewing in the Macaque

Visual motion responses in the brain are shaped by two distinct sources: the physical movement of objects in the environment and motion resulting from one's own actions. The latter source, termed visual reafference, stems from movements of the head and body, and in primates from the frequent saccadic eye movements that mark natural vision. To study the relative contribution of reafferent and stimulus motion during natural vision, we measured fMRI activity in the brains of two macaques as they freely viewed >50 hours of naturalistic video footage depicting dynamic social interactions. We used eye movements obtained during scanning to estimate the level of reafferent retinal motion at each moment in time. We also estimated the net stimulus motion by analyzing the video content during the same time periods. Mapping the responses to these distinct sources of retinal motion, we found a striking dissociation in the distribution of visual responses throughout the brain. Reafferent motion drove fMRI activity in the early retinotopic areas V1, V2, V3, and V4, particularly in their central visual field representations, as well as lateral aspects of the caudal inferotemporal cortex (area TEO). However, stimulus motion dominated fMRI responses in the superior temporal sulcus, including areas MT, MST, and FST as well as more rostral areas. We discuss this pronounced separation of motion processing in the context of natural vision, saccadic suppression, and the brain's utilization of corollary discharge signals. Visual motion arises not only from events in the external world, but also from the movements of the observer. For example, even if objects are stationary in the world, the act of walking through a room or shifting one's eyes causes motion on the retina. This "reafferent" motion propagates into the brain as signals that must be interpreted in the context of real object motion. The delineation of whole-brain responses to stimulus versus self-generated retinal motion signals is critical for understanding visual perception and is of pragmatic importance given the increasing use of naturalistic viewing paradigms. The present study uses fMRI to demonstrate that the brain exhibits a fundamentally different pattern of responses to these two sources of retinal motion.

Adaptation to human locomotion speed

Visual judgments of human movement play an important role in social interactions, but relatively little is known about how retinal motion signals are used to estimate human movement speed. We report a new effect which demonstrates that these judgments are subject to modification by exposure to dynamic images. Participants viewed videos of real scenes depicting either groups of figures walking along a High Street or contestants running in the London Marathon. When video playback was speeded up or slowed down slightly relative to natural speed, participants could readily report whether the figures in each video appeared to be moving unnaturally quickly or slowly. However after adapting to slowed-down walking, natural walking speed appeared too fast, and after adapting to speeded-up walking, natural walking speed appeared too slow. Corresponding effects were found for running videos. Adaptation to natural-speed playback had no effect on apparent locomotion speed. These effects are quite different in a number of respects from those previously reported in studies of retinal velocity adaptation using simple patterns such as gratings. Unlike the stimuli used in most previous studies our videos contained a range of speeds and directions due to the unpredictability of natural scenes. Walkers and runners moved in different directions at different speeds, and at varying distances from the camera. Participants also engaged in free viewing rather than fixation. Furthermore over the range of retinal velocities our stimuli contained, adaptation to simple moving patterns causes a significant reduction in apparent velocity at all test velocities, including at the adapting velocity. Our data are consistent with the operation of a qualitatively different process in judgements of locomotion speed in natural scenes.

Enhancement of motion perception in the direction opposite to smooth pursuit eye movement.

When eyes track a moving target, a stationary background environment moves in the direction opposite to the eye movement on the observer's retina. Here, we report a novel effect in which smooth pursuit can enhance the retinal motion in the direction opposite to eye movement, under certain conditions. While performing smooth pursuit, the observers were presented with a counterphase grating on the retina. The counterphase grating consisted of two drifting component gratings: one drifting in the direction opposite to the eye movement and the other drifting in the same direction as the pursuit. Although the overall perceived motion direction should be ambiguous if only retinal information is considered, our results indicated that the stimulus almost always appeared to be moving in the direction opposite to the pursuit direction. This effect was ascribable to the perceptual dominance of the environmentally stationary component over the other. The effect was robust at suprathreshold contrasts, but it disappeared at lower overall contrasts. The effect was not associated with motion capture by a reference frame served by peripheral moving images. Our findings also indicate that the brain exploits eye-movement information not only for eye-contingent image motion suppression but also to develop an ecologically plausible interpretation of ambiguous retinal motion signals. Based on this biological assumption, we argue that visual processing has the functional consequence of reducing the apparent motion blur of a stationary background pattern during eye movements and that it does so through integration of the trajectories of pattern and color signals.

Computations underlying the visuomotor transformation for smooth pursuit eye movements.

Smooth pursuit eye movements are driven by retinal motion and enable us to view moving targets with high acuity. Complicating the generation of these movements is the fact that different eye and head rotations can produce different retinal stimuli but giving rise to identical smooth pursuit trajectories. However, because our eyes accurately pursue targets regardless of eye and head orientation (Blohm G, Lefèvre P. J Neurophysiol 104: 2103-2115, 2010), the brain must somehow take these signals into account. To learn about the neural mechanisms potentially underlying this visual-to-motor transformation, we trained a physiologically inspired neural network model to combine two-dimensional (2D) retinal motion signals with three-dimensional (3D) eye and head orientation and velocity signals to generate a spatially correct 3D pursuit command. We then simulated conditions of 1) head roll-induced ocular counterroll, 2) oblique gaze-induced retinal rotations, 3) eccentric gazes (invoking the half-angle rule), and 4) optokinetic nystagmus to investigate how units in the intermediate layers of the network accounted for different 3D constraints. Simultaneously, we simulated electrophysiological recordings (visual and motor tunings) and microstimulation experiments to quantify the reference frames of signals at each processing stage. We found a gradual retinal-to-intermediate-to-spatial feedforward transformation through the hidden layers. Our model is the first to describe the general 3D transformation for smooth pursuit mediated by eye- and head-dependent gain modulation. Based on several testable experimental predictions, our model provides a mechanism by which the brain could perform the 3D visuomotor transformation for smooth pursuit.

The neuronal basis of on-line visual control in smooth pursuit eye movements

Smooth pursuit eye movements allow us to maintain the image of a moving target on the fovea. Smooth pursuit consists of separate phases such as initiation and steady-state. These two phases are supported by different visual-motor mechanisms in cortical areas including the middle temporal (MT), the medial superior temporal (MST) areas and the frontal eye field (FEF). Retinal motion signals are responsible for beginning the process of pursuit initiation, whereas extraretinal signals play a role in maintaining tracking speed. Smooth pursuit often requires on-line gain adjustments during tracking in response to a sudden change in target motion. For example, a brief sinusoidal perturbation of target motion induces a corresponding perturbation of eye motion. Interestingly, the perturbation ocular response is enhanced when baseline pursuit velocity is higher, even though the stimulus frequency and amplitude are constant. This on-line gain control mechanism is not simply due to visually driven activity of cortical neurons. Visual and pursuit signals are primarily processed in cortical MT/MST and the magnitude of perturbation responses could be regulated by the internal gain parameter in FEF. Furthermore, the magnitude and the gain slope of perturbation responses are altered by smooth pursuit adaptation using repeated trials of a step-ramp tracking with two different velocities (double-velocity paradigm). Therefore, smooth pursuit adaptation, which is attributed to the cerebellar plasticity mechanism, could affect the on-line gain control mechanism.

Evidence for a retinal velocity memory underlying the direction of anticipatory smooth pursuit eye movements

To compute spatially correct smooth pursuit eye movements, the brain uses both retinal motion and extraretinal signals about the eyes and head in space (Blohm and Lefèvre 2010). However, when smooth eye movements rely solely on memorized target velocity, such as during anticipatory pursuit, it is unknown if this velocity memory also accounts for extraretinal information, such as head roll and ocular torsion. To answer this question, we used a novel behavioral updating paradigm in which participants pursued a repetitive, spatially constant fixation-gap-ramp stimulus in series of five trials. During the first four trials, participants' heads were rolled toward one shoulder, inducing ocular counterroll (OCR). With each repetition, participants increased their anticipatory pursuit gain, indicating a robust encoding of velocity memory. On the fifth trial, they rolled their heads to the opposite shoulder before pursuit, also inducing changes in ocular torsion. Consequently, for spatially accurate anticipatory pursuit, the velocity memory had to be updated across changes in head roll and ocular torsion. We tested how the velocity memory accounted for head roll and OCR by observing the effects of changes to these signals on anticipatory trajectories of the memory decoding (fifth) trials. We found that anticipatory pursuit was updated for changes in head roll; however, we observed no evidence of compensation for OCR, representing the absence of ocular torsion signals within the velocity memory. This indicated that the directional component of the memory must be coded retinally and updated to account for changes in head roll, but not OCR.

Conjugate Adaptation of Smooth Pursuit during Monocular Viewing in Strabismic Monkeys with Exotropia

Humans and monkeys are able to adapt their smooth pursuit output when challenged with consistent errors in foveal/parafoveal image motion during tracking. Visual motion information from the retina is known to be necessary for guiding smooth pursuit adaptation. The purpose of this study is to determine whether retinal motion signals delivered to one eye during smooth pursuit produce adaptation in the fellow eye. We tested smooth pursuit adaptation during monocular viewing in strabismic monkeys with exotropia. To induce smooth pursuit adaptation experimentally, we used a step-ramp tracking with two different velocities (adaptation paradigm), where the target begins moving at one speed (25°/s) for first 100 ms and then changes to a lower speed (5°/s) for the remainder of the trial. Typically, 100 to 200 trials were used to adapt the smooth pursuit response. Control trials employing single speed step-ramp target motion (ramp speed = 25°/s) were used before and after adaptation paradigm to estimate adaptation. The magnitude of adaptation as calculated by percentage change was not significantly different (P = 0.53) for the viewing (mean, 40.3% ± 5.9%) and the nonviewing (mean, 39.7% ± 6.2%) eyes during monocular viewing conditions, even in cases with large angle (18°-20°) strabismus. Our results indicate that animals with strabismus retain the ability to produce conjugate adaptation of smooth pursuit. Therefore, we suggest that a single central representation of retinal motion information in the viewing eye drives adaptation for both eyes equally.

Motion processing with two eyes in three dimensions

The movement of an object toward or away from the head is perhaps the most critical piece of information an organism can extract from its environment. Such 3D motion produces horizontally opposite motions on the two retinae. Little is known about how or where the visual system combines these two retinal motion signals, relative to the wealth of knowledge about the neural hierarchies involved in 2D motion processing and binocular vision. Canonical conceptions of primate visual processing assert that neurons early in the visual system combine monocular inputs into a single cyclopean stream (lacking eye-of-origin information) and extract 1D ("component") motions; later stages then extract 2D pattern motion from the cyclopean output of the earlier stage. Here, however, we show that 3D motion perception is in fact affected by the comparison of opposite 2D pattern motions between the two eyes. Three-dimensional motion sensitivity depends systematically on pattern motion direction when dichoptically viewing gratings and plaids-and a novel "dichoptic pseudoplaid" stimulus provides strong support for use of interocular pattern motion differences by precluding potential contributions from conventional disparity-based mechanisms. These results imply the existence of eye-of-origin information in later stages of motion processing and therefore motivate the incorporation of such eye-specific pattern-motion signals in models of motion processing and binocular integration.

Visuomotor Velocity Transformations for Smooth Pursuit Eye Movements

Smooth pursuit eye movements are driven by retinal motion signals. These retinal motion signals are converted into motor commands that obey Listing's law (i.e., no accumulation of ocular torsion). The fact that smooth pursuit follows Listing's law is often taken as evidence that no explicit reference frame transformation between the retinal velocity input and the head-centered motor command is required. Such eye-position-dependent reference frame transformations between eye- and head-centered coordinates have been well-described for saccades to static targets. Here we suggest that such an eye (and head)-position-dependent reference frame transformation is also required for target motion (i.e., velocity) driving smooth pursuit eye movements. Therefore we tested smooth pursuit initiation under different three-dimensional eye positions and compared human performance to model simulations. We specifically tested if the ocular rotation axis changed with vertical eye position, if the misalignment of the spatial and retinal axes during oblique fixations was taken into account, and if ocular torsion (due to head roll) was compensated for. If no eye-position-dependent velocity transformation was used, the pursuit initiation should follow the retinal direction, independently of eye position; in contrast, a correct visuomotor velocity transformation would result in spatially correct pursuit initiation. Overall subjects accounted for all three components of the visuomotor velocity transformation, but we did observe differences in the compensatory gains between individual subjects. We concluded that the brain does perform a visuomotor velocity transformation but that this transformation was prone to noise and inaccuracies of the internal model.

Sensitivity to retinal and extra-retinal motion signals as a function of age

Sensitivity to retinal and extra-retinal motion signals as a function of age

Eyes Pursue Moving Objects, Not Retinal Motion Signals

For smooth-pursuit eye movements, a moving target is necessary, but we show that it is not sufficient. Observers pursued targets that appeared to move in one direction even though they really moved in another. Changes in perceived direction did not disrupt pursuit eye movements, but motion-based failures in object parsing did.

Demonstration of an eye-movement-induced visual motion illusion (Filehne illusion) in Rhesus monkeys

During pursuit eye movements, the world around us remains perceptually stable despite the retinal-image slip induced by the eye movement. It is commonly held that this perceptual invariance is achieved by subtracting an internal reference signal, reflecting the eye movement, from the retinal motion signal. However, if the reference signal is too small or too large, a false eye-movement-induced motion of the external world, the Filehne illusion (FI), will be perceived. A reference signal of inadequate size can be simulated experimentally by asking human subjects to pursue a target across backgrounds with externally added motion that are perceived as moving. In the present study we asked if non-human primates respond to such manipulation in a way comparable to humans. Using psychophysical methods, we demonstrate that Rhesus monkeys do indeed experience a percept of pursuit-induced background motion. In this study we show that an FI can be predictably induced in Rhesus monkeys. The monkey FI shows dependencies on the size and direction of background movement, which is very similar to the ones characterizing the human FI. This congruence suggests that the perception of self-induced visual motion is based on similar inferential mechanisms in non-human and human primates.

Local motion inside an object affects pointing less than smooth pursuit

During smooth pursuit eye movements, briefly presented objects are mislocalized in the direction of motion. It has been proposed that the localization error is the sum of the pursuit signal and the retinal motion signal in a ~200 ms interval after flash onset. To evaluate contributions of retinal motion signals produced by the entire object (global motion) and elements within the object (local motion), we asked observers to reach to flashed Gabor patches (Gaussian-windowed sine-wave gratings). Global motion was manipulated by varying the duration of a stationary flash, and local motion was manipulated by varying the motion of the sine-wave. Our results confirm that global retinal motion reduces the localization error. The effect of local retinal motion on object localization was far smaller, even though local and global motion had equal effects on eye velocity. Thus, local retinal motion has differential access to manual and oculomotor control circuits. Further, we observed moderate correlations between smooth pursuit gain and localization error.