Gaze-centered Coordinates Research Articles

Gaze-specific motor memories for hand-reaching

Numerous studies have proposed that our adaptive motor behaviors depend on learning a map between sensory information and limb movement,1-3 called an "internal model." From this perspective, how the brain represents internal models is a critical issue in motor learning, especially regarding their association with spatial frames processed in motor planning.4,5 Extensive experimental evidence suggests that during planning stages for visually guided hand reaching, the brain transforms visual target representations in gaze-centered coordinates to motor commands in limb coordinates, via hand-target vectors in workspace coordinates.6-9 While numerous studies have intensively investigated whether the learning for reaching occurs in workspace or limb coordinates,10-20 the association of the learning with gaze coordinates still remains untested.21 Given the critical role of gaze-related spatial coding in reaching planning,22-26 the potential role of gaze states for learning is worth examining. Here, we show that motor memories for reaching are separately learned according to target location in gaze coordinates. Specifically, two opposing visuomotor rotations, which normally interfere with each other, can be simultaneously learned when each is associated with reaching to a foveal target and peripheral one. We also show that this gaze-dependent learning occurs in force-field adaptation. Furthermore, generalization of gaze-coupled reach adaptation is limited across central, right, and left visual fields. These results suggest that gaze states are available in the formation and recall of multiple internal models for reaching. Our findings provide novel evidence that a gaze-dependent spatial representation can provide a spatial coordinate framework for context-dependent motor learning.

Contributions of gaze-centered and object-centered coding in a double-step saccade task.

The position of a saccade target can be encoded in gaze-centered coordinates, that is, relative to the current gaze position, or in object-centered coordinates, that is, relative to an object in the environment. We tested the role of gaze-centered and object-centered coding in a double-step saccade task involving the Brentano version of the Müller-Lyer illusion. The two visual targets were presented either sequentially, requiring gaze-centered coding of the second saccade target, or simultaneously, thereby providing additional object-centered information about the location of the second target relative to the first. We found that the endpoint of the second saccade was affected by the illusion, irrespective of whether the targets were presented sequentially or simultaneously, suggesting that participants used a gaze-centered updating strategy. We found that variability in saccade endpoints was reduced when object-centered information was consistently available but not when its presence varied from trial to trial. Our results suggest that gaze-centered coding is dominant in the planning of sequential saccades, whereas object-centered information plays a relatively small role.

A Computational Model to Study the Dynamics of Updating of Remembered Visual Targets During Rapid and Slow Eye Movements

After an intervening eye movement, or saccade, humans and animals are able to localize previously perceived visual targets (spatial updating). Although efforts have been made to discover the mechanism underlying spatial updating, there are still many unanswered questions about the neuronal mechanism of this phenomenon. State space model is an effective method for modeling dynamical systems and it can represent the internal behaviour of these systems. Therefore, we developed a state space model for updating target-related spatial information in gaze-centered coordinates. We considered three types of input in our proposed model: 1) an efference copy signal, inspired by motor burst in SC, 2) an eye position signal, found in LIP, VIP, MT and MST areas and 3) visual topographic maps of visual stimuli, located in SC. To model the internal neuronal behaviour of the system, we developed a radial basis function neural network (RBFNN) which can be trained with an Extended Kalman filter method. This RBFNN represents the state space and we can obtain a topographic map of the remembered target in its hidden layer. From our proposed model, the output obtained is the decoded location of the remembered target. To explore the internal mechanism underlying the updating process, we trained this model on a double-step saccade-saccade or pursuit-saccade task. After training, the receptive fields of state-space units replicated both predictive remapping during saccades (Duhamel et al. Science 1992) and continuous eye-centered updating during smooth pursuit (Dash et al. Current Biology, in press). In addition, during trans-saccadic remapping, receptive fields also expanded (to our knowledge, this predicted expansion has not yet been reported in the published literature). In the future, we plan to incorporate this framework within a more comprehensive model of trans-saccadic integration of both spatial and feature information, and use this framework to construct a physiologically plausible model. Meeting abstract presented at VSS 2015. Language: en

Attentional load interferes with target localization across saccades.

The retinal positions of objects in the world change with each eye movement, but we seem to have little trouble keeping track of spatial information from one fixation to the next. We examined the role of attention in trans-saccadic localization by asking participants to localize targets while performing an attentionally demanding secondary task. In the first experiment, attentional load decreased localization precision for a remembered target, but only when a saccade intervened between target presentation and report. We then repeated the experiment and included a salient landmark that shifted on half the trials. The shifting landmark had a larger effect on localization under high load, indicating that observers rely more on landmarks to make localization judgments under high than under low attentional load. The results suggest that attention facilitates trans-saccadic localization judgments based on spatial updating of gaze-centered coordinates when visual landmarks are not available. The availability of reliable landmarks (present in most natural circumstances) can compensate for the effects of scarce attentional resources on trans-saccadic localization.

Parietal Oscillations Code Nonvisual Reach Targets Relative to Gaze and Body

Recent blood oxygenation level-dependent (BOLD) imaging work has suggested flexible coding frames for reach targets in human posterior parietal cortex, with a gaze-centered reference frame for visually guided reaches and a body-centered frame for proprioceptive reaches. However, BOLD activity, which reflects overall population activity, is insensitive to heterogeneous responses at the neuronal level and temporal dynamics between neurons. Neurons could synchronize in different frequency bands to form assemblies operating in different reference frames. Here we assessed the reference frames of oscillatory activity in parietal cortex during reach planning to nonvisible tactile stimuli. Under continuous recording of magneto-encephalographic data, subjects fixated either to the left or right of the body midline, while a tactile stimulus was presented to a nonvisible fingertip, located either to the left or right of gaze. After a delay, they had to reach toward the remembered stimulus location with the other hand. Our results show body-centered and gaze-centered reference frames underlying the power modulations in specific frequency bands. Whereas beta-band activity (18-30 Hz) in parietal regions showed body-centered spatial selectivity, the high gamma band (>60 Hz) demonstrated a transient remapping into gaze-centered coordinates in parietal and extrastriate visual areas. This gaze-centered coding was sustained in the low gamma (<60 Hz) and alpha (∼10 Hz) bands. Our results show that oscillating subpopulations encode remembered tactile targets for reaches relative to gaze, even though neither the sensory nor the motor output processes operate in this frame. We discuss these findings in the light of flexible control mechanisms across modalities and effectors.

Inactivation of the Parietal Reach Region Causes Optic Ataxia, Impairing Reaches but Not Saccades

Lesions in human posterior parietal cortex can cause optic ataxia (OA), in which reaches but not saccades to visual objects are impaired, suggesting separate visuomotor pathways for the two effectors. In monkeys, one potentially crucial area for reach control is the parietal reach region (PRR), in which neurons respond preferentially during reach planning as compared to saccade planning. However, direct causal evidence linking the monkey PRR to the deficits observed in OA is missing. We thus inactivated part of the macaque PRR, in the medial wall of the intraparietal sulcus, and produced the hallmarks of OA, misreaching for peripheral targets but unimpaired saccades. Furthermore, reach errors were larger for the targets preferred by the neural population local to the injection site. These results demonstrate that PRR is causally involved in reach-specific visuomotor pathways, and reach goal disruption in PRR can be a neural basis of OA.

Continuous updating of superior colliculus visuospatial memory responses during smooth pursuit eye movements.

Spatial updating is a process that enables us to constantly compute spatial relationships between ourselves and the surrounding environment during self-motion. Primates are able to spatially update saccade targets across intervening saccade or smooth pursuit (SP) eye movements. Various studies have demonstrated discrete remapping of visual responses, in several brain structures, before and after the saccade. However, no study to date has shown continuous updating of such responses during the eye movement, nor has anyone published a study of the neural mechanism of spatial updating for SP. In this study we recorded superior-colliculus unit activity from 2 monkeys. Animals were trained to spatially update the location of a saccade target across an intervening SP. Neurons were characterized (as visual, visuo-motor or motor neurons) and their visual / motor receptive fields (RF) were specified using a delayed saccade paradigm. After this each neuron was tested in the SP-saccade paradigm. Saccade target direction and pursuit ramp length were varied such that (across trials) the remembered saccade target passed approximately through the neuron’s RF at 2-5 different points on the SP ramp. Of the 50 neurons analyzed to date, every neuron that showed a visual response (n=47), whether it was a visual and visuomotor neuron, exhibited a clear and robust modulation in activity when the location of remembered target passed across the RF of the neuron during SP. In contrast, none of the motor neurons showed any updating related response, but instead only burst for an intended saccade. We conclude that 1) the superior colliculus visual memory response is continuously updated during SP eye movements in a gaze-centered coordinates, and 2) only visual responses (not motor) are involved in visuospatial memory. This may reflect a general mechanism for continuous updating during slow self-motion. Meeting abstract presented at VSS 2012

Spatial updating across saccades during manual interception

We studied the effect of intervening saccades on the manual interception of a moving target. Previous studies suggest that stationary reach goals are coded and updated across saccades in gaze-centered coordinates, but whether this generalizes to interception is unknown. Subjects (n = 9) reached to manually intercept a moving target after it was rendered invisible. Subjects either fixated throughout the trial or made a saccade before reaching (both fixation points were in the range of -10° to 10°). Consistent with previous findings and our control experiment with stationary targets, the interception errors depended on the direction of the remembered moving goal relative to the new eye position, as if the target is coded and updated across the saccade in gaze-centered coordinates. However, our results were also more variable in that the interception errors for more than half of our subjects also depended on the goal direction relative to the initial gaze direction. This suggests that the feedforward transformations for interception differ from those for stationary targets. Our analyses show that the interception errors reflect a combination of biases in the (gaze-centered) representation of target motion and in the transformation of goal information into body-centered coordinates for action.

Human Posterior Parietal Cortex Flexibly Determines Reference Frames for Reaching Based on Sensory Context

Current models of sensorimotor transformations emphasize the dominant role of gaze-centered representations for reach planning in the posterior parietal cortex (PPC). Here we exploit fMRI repetition suppression to test whether the sensory modality of a target determines the reference frame used to define the motor goal in the PPC and premotor cortex. We show that when targets are defined visually, the anterior precuneus selectively encodes the motor goal in gaze-centered coordinates, whereas the parieto-occipital junction, Brodman Area 5 (BA 5), and PMd use a mixed gaze- and body-centered representation. In contrast, when targets are defined by unseen proprioceptive cues, activity in these areas switches to represent the motor goal predominantly in body-centered coordinates. These results support computational models arguing for flexibility in reference frames for action according to sensory context. Critically, they provide neuroanatomical evidence that flexibility is achieved by exploiting a multiplicity of reference frames that can be expressed within individual areas.

Interactions between gaze-centered and allocentric representations of reach target location in the presence of spatial updating

Numerous studies have investigated the phenomenon of egocentric spatial updating in gaze-centered coordinates, and some have studied the use of allocentric cues in visually-guided movement, but it is not known how these two mechanisms interact. Here, we tested whether gaze-centered and allocentric information combine at the time of viewing the target, or if the brain waits until the last possible moment. To do this, we took advantage of the well-known fact that pointing and reaching movements show gaze-centered ‘retinal magnification’ errors (RME) that update across saccades. During gaze fixation, we found that visual landmarks, and hence allocentric information, reduces RME for targets in the left visual hemifield but not in the right. When a saccade was made between viewing and reaching, this landmark-induced reduction in RME only depended on gaze at reach, not at encoding. Based on this finding, we argue that egocentric–allocentric combination occurs after the intervening saccade. This is consistent with previous findings in healthy and brain damaged subjects suggesting that the brain updates early spatial representations during eye movement and combines them at the time of action.

Salient features in gaze-aligned recordings of human visual input during free exploration of natural environments

During free exploration, humans adjust their gaze by combining body, head, and eye movements. Laboratory experiments on the stimulus features driving gaze, however, typically focus on eye-in-head movements, use potentially biased stimuli, and restrict the field of view. Our novel wearable eye-tracking system (EyeSeeCam) overcomes these limitations. We recorded gaze- and head-centered videos of the visual input of observers freely exploring real-world environments (4 indoor, 8 outdoor), yielding approximately 10 h of data. Global power spectra reveal little difference between head- and gaze-centered recordings. Local stimulus features exhibit spatial biases in head-centered coordinates, which are environment-dependent, but consistent across observers. Eye-in-head movements center these biases in gaze-centered coordinates, leading to elevated "salient" features at center of gaze. This shows that central biases in image feature distributions in "natural" photographs are not a property of environments, but of stimuli already gaze-centered by the photographer. Further central biases in laboratory subjects' fixation distributions do not result from re-centering of the eyes but are an artifact of display restrictions. Hence, our findings demonstrate that the concept of feature "saliency" transfers from the laboratory to free exploration, but also highlight the importance of experiments with freely moving eyes, head, and body.

Influence of saccadic adaptation on spatial localization: Comparison of verbal and pointing reports

Under conditions of short-term saccadic adaptation, stimuli presented long before saccadic onset are perceptually mislocalized in space. Here we study whether saccadic adaptation can also affect localization of objects by pointing. We measured localization performance during fixation and after normal saccades and adapted saccades, for a bar presented well before a saccadic eye movement, for both pointing and verbal localization, under open-loop conditions generated by a transient dark period about 300 ms after the presentation of the bar. During fixation and normal saccade, localization performance for verbal report was veridical, while for pointing there was an overestimation of the target eccentricity respect to gaze, in agreement with the idea of separate representations of space for action and perception. During saccadic adaptation, there was a significant shift of both pointing and verbal report localization in the direction of adaptation with similar spatial selectivity for both tasks. These results indicate that saccadic adaptation induces a similar re-calibration of the action map as well as of the perceptual map, suggesting a common site of operation in the transformation from eye-centered to gaze-centered coordinates.

Influence of initial hand and target position on reach errors in optic ataxic and normal subjects

Recent neurophysiological studies suggest that reach planning areas in the posterior parietal cortex encode both target and initial hand position in gaze-centered coordinates, which could be used to calculate a desired movement vector. We tested how varying gaze, target position, and initial hand position affected reach errors in two left unilateral optic ataxia patients with right PPC damage and seven neurologically intact controls. Both controls' and patients' reaching errors revealed an influence of target position in gaze-centered coordinates; however, both patients' mean errors were offset toward the left, with greater errors when the target was in their left visual field, consistent with the damage to the right PPC. Control subjects also showed a large quasi-independent shoulder-centered influence of target position. This effect was much less present in patient C.F., who had more medial damage to the PPC. In contrast, for patient O.K., who had more lateral PPC damage, the shoulder-centered effect was larger and interacted with the gaze-centered influence of target position. All subjects' errors also revealed a shoulder-centered influence of the initial hand position, with larger influences on the patients' reaching errors. Both patients also showed an interactive influence of the shoulder-centered and gaze-centered initial hand positions. These results suggest that the target and the hand are compared at more than one level in the visuomotor pathway in multiple reference frames, and these comparisons are then integrated. Depending on the location of the damage within the PPC, these comparisons are disrupted, changing the relative influence of hand and target position in different reference frames on the final reaching movement.

Human Parietal “Reach Region” Primarily Encodes Intrinsic Visual Direction, Not Extrinsic Movement Direction, in a Visual–Motor Dissociation Task

Posterior parietal cortex (PPC) participates in the planning of visuospatial behaviors, including reach movements, in gaze-centered coordinates. It is not known if these representations encode the visual goal in retinal coordinates, or the movement direction relative to gaze. Here, by dissociating the intrinsic retinal stimulus from the extrinsic direction of movement, we show that PPC employs a visual code. Using delayed pointing and event-related functional magnetic resonance imaging, we identified a cluster of PPC regions whose activity was topographically (contralaterally) related to the direction of the planned movement. We then switched the normal visual-motor spatial relationship by adapting subjects to optical left/right reversing prisms. With prisms, movement-related PPC topography reversed, remaining tied to the retinal image. Thus, remarkably, the PPC region in each hemisphere now responded more for planned ipsilateral pointing movements. Other non-PPC regions showed the opposite world- or motor-fixed pattern. These findings suggest that PPC primarily encodes not motor commands but movement goals in visual coordinates.

Gaze-Centered Updating of Remembered Visual Space During Active Whole-Body Translations

Various cortical and sub-cortical brain structures update the gaze-centered coordinates of remembered stimuli to maintain an accurate representation of visual space across eyes rotations and to produce suitable motor plans. A major challenge for the computations by these structures is updating across eye translations. When the eyes translate, objects in front of and behind the eyes' fixation point shift in opposite directions on the retina due to motion parallax. It is not known if the brain uses gaze coordinates to compute parallax in the translational updating of remembered space or if it uses gaze-independent coordinates to maintain spatial constancy across translational motion. We tested this by having subjects view targets, flashed in darkness in front of or behind fixation, then translate their body sideways, and subsequently reach to the memorized target. Reach responses showed parallax-sensitive updating errors: errors increased with depth from fixation and reversed in lateral direction for targets presented at opposite depths from fixation. In a series of control experiments, we ruled out possible biasing factors such as the presence of a fixation light during the translation, the eyes accompanying the hand to the target, and the presence of visual feedback about hand position. Quantitative geometrical analysis confirmed that updating errors were better described by using gaze-centered than gaze-independent coordinates. We conclude that spatial updating for translational motion operates in gaze-centered coordinates. Neural network simulations are presented suggesting that the brain relies on ego-velocity signals and stereoscopic depth and direction information in spatial updating during self-motion.

Eye–Hand Coordination during Learning of a Novel Visuomotor Task

We investigated how gaze behavior and eye-hand coordination change when subjects learned a challenging visuomotor task that required acquisition of a novel mapping between bimanual actions and their visual sensory consequences. By applying isometric forces and torques to a rigid tool held freely between the two hands, subjects learned to control a cursor on a computer screen to hit successively displayed targets as quickly as possible. The learning occurred in stages that could be distinguished by changes in performance (target-hit rate) as well as by gaze behavior and eye-hand coordination. In a first exploratory stage, the hit rate was consistently low, the cursor position varied widely, and gaze typically pursued the cursor. In a second skill acquisition stage, the hit rate improved rapidly, and gaze fixations began to mark predictively desired cursor positions, indicating that subjects started to program spatially congruent eye and hand motor commands. In a third skill refinement stage, performance continued to improve gradually, and gaze shifted directly toward the target. We suggest that during the exploratory stage, the learner attempts to establish basic mapping rules between manual actions and eye-movement commands. In this process, subjects may establish correlations between hand motor commands and their visual sensory consequences, primarily in fovea-anchored, gaze-centered coordinates, and correlations between recent hand motor commands and eye motor commands. The established mapping rules are then implemented and refined in the skill acquisition and refinement stages.

Impairment of Gaze-centered Updating of Reach Targets in Bilateral Parietal–Occipital Damaged Patients

Recent studies have suggested that internal updating of visuospatial targets in humans occurs in gaze-centered coordinates and takes place in the parietal and extrastriate cortices. We explored how information for reaching is updated in two patients with bilateral lesions in these areas. Subjects performed two visuomotor tasks: (i) a fixation reaching task, which began with the appearance of one of five fixation positions (varying eye positions) followed by a central reaching target. Subjects reached to the target while fixating on the presented fixation position (relative to gaze the target was always presented in the periphery); and (ii) a saccade reaching task, in which subjects foveated on the central reaching target, then made a saccade to the presented fixation position before reaching to the central target. In both tasks, subjects reached to targets after a 500 or 5000 ms delay. Gaze-centered updating predicts similarities in reaching errors between fixation and saccade trials. Control subjects showed evidence for gaze-centered updating during both 500 and 5000 ms delay conditions. In contrast, patient AT, who had extensive occipital-parietal damage, only showed signs of gaze-centered representation after 5 s. Patient IG, with a more focal lesion in the parietal cortices, showed partial updating in gaze-centered coordinates when reaching with the small memory delay but recovered a complete gaze-centered representation after the longer delay. This suggests that patients with bilateral occipital-parietal lesions may rely on non-gaze-centered frames to store immediate target locations in reaching space but, given enough time, this information may be rerouted to access other gaze-centered motor cortical mechanisms.