Visuomotor Transformation Research Articles

Time Course of Changes in Brain Activity and Functional Connectivity Associated With Long-Term Adaptation to a Rotational Transformation

The purpose of this study was to examine the time course of changes in cerebral activity and functional connectivity during long-term adaptation to a visuomotor transformation. Positron emission tomography was used to measure changes in brain activity as subjects tracked a target under the influence of a rotational transformation that distorted visual feedback. The experiment was 1 week long and consisted of two scanning sessions (obtained on days 2 and 7), aimed at examining early and late stages of learning. On average, visuomotor adaptation was achieved within 3 days. During early stages of adaptation, better performance was associated with greater activity in brain areas related to attention including bilateral dorso- and ventrolateral prefrontal cortices, frontal eye fields, and the human homologue of area MT. However, as adaptation proceeded, improvements in performance were associated with greater activity in motor regions such as the left (contralateral) sensorimotor cortex, bilateral anterior cerebellum, left cingulate motor area, right putamen, and a nonmotor region within the middle temporal gyrus. This learning-specific shift in brain activity was associated with a progressive change in the functional connectivity of these regions toward the end of the first session. Interestingly, only the functional connections between the anterior cerebellum, left middle temporal gyrus, and left sensorimotor cortex remained strong once visuomotor adaptation was achieved. Our findings suggest that visuomotor adaptation is not only reflected in persistent changes in activity in motor-related regions, but also in the strengthening and maintenance of specific functional connections.

Human medial intraparietal cortex subserves visuomotor coordinate transformation

In the macaque, the posterior parietal cortex (PPC) integrates multimodal sensory information for planning and coordinating complex movements. In particular, the areas around the intraparietal sulcus (IPS) serve as an interface between the sensory and motor systems to allow for coordinated movements in space. Because recent imaging studies suggest a comparable functional and anatomical organization of human and monkey IPS, we hypothesized that in humans, as in macaques, the medial intraparietal cortex (area MIP) subserves visuomotor transformations. To test this hypothesis, changes of neural activity were measured using functional magnetic resonance imaging (fMRI) while healthy subjects performed a joystick paradigm similar to the ones previously employed in macaques for studying area MIP. As hypothesized, visuomotor coordinate transformation subserving goal-directed hand movements activated superior parietal cortex with the local maximum of increased neural activity lying in the medial wall of IPS. Compared to the respective visuomotor control conditions, goal-directed hand movements under predominantly proprioceptive control activated a more anterior part of medial IPS, whereas posterior medial IPS was more responsive to visually guided hand movements. Contrasting the two coordinate transformation conditions, changing the modality of movement guidance (visual/proprioceptive) did not significantly alter the BOLD signal within IPS but demonstrated differential recruitment of modality specific areas such as V5/MT and sensorimotor cortex/area 5, respectively. The data suggest that the human medial intraparietal cortex subserves visuomotor transformation processes to control goal-directed hand movements independently from the modality-specific processing of visual or proprioceptive information.

Distributed Population Mechanism for the 3-D Oculomotor Reference Frame Transformation

Human saccades require a nonlinear, eye orientation-dependent reference frame transformation to transform visual codes to the motor commands for eye muscles. Primate neurophysiology suggests that this transformation is performed between the superior colliculus and brain stem burst neurons, but provides little clues as to how this is done. To understand how the brain might accomplish this, we trained a 3-layer neural net to generate accurate commands for kinematically correct 3-D saccades. The inputs to the network were a 2-D, eye-centered, topographic map of Gaussian visual receptive fields and an efference copy of eye position in 6-dimensional, push-pull "neural integrator" coordinates. The output was an eye orientation displacement command in similar coordinates appropriate to drive brain stem burst neurons. The network learned to generate accurate, kinematically correct saccades, including the eye orientation-dependent tilts in saccade motor error commands required to match saccade trajectories to their visual input. Our analysis showed that the hidden units developed complex, eye-centered visual receptive fields, widely distributed fixed-vector motor commands, and "gain field"-like eye position sensitivities. The latter evoked subtle adjustments in the relative motor contributions of each hidden unit, thereby rotating the population motor vector into the correct correspondence with the visual target input for each eye orientation: a distributed population mechanism for the visuomotor reference frame transformation. These findings were robust; there was little variation across networks with between 9 and 49 hidden units. Because essentially the same observations have been reported in the visuomotor transformations of the real oculomotor system, as well as other visuomotor systems (although interpreted elsewhere in terms of other models) we suggest that the mechanism for visuomotor reference frame transformations identified here is the same solution used in the real brain.

Learning-related fMRI activation associated with a rotational visuo-motor transformation

The unique ability to learn transformed or altered visuo-motor relationships during motor learning (visuo-motor transformation learning) has engaged researchers for over a century. Compared to other forms of motor learning (e.g., sequence learning), little is known about plasticity in the cortical and/or subcortical systems involved. We used fMRI to isolate region-specific activation changes during the learning of a visuo-motor (joystick) task under a simple transformation (90° rotation of visual feedback). Distributed brain systems were engaged in the learning process. In particular, we found evidence of a learning-dependent transition from early activation of the posterior parietal cortex to later distributed cortico-subcortical–cerebellar responses (in the temporal and occipital cortices, basal ganglia, cerebellum and thalamus). The role of the posterior parietal cortex may relate specifically to the acquisition of the transformation, while that of the fusiform and superior temporal gyri may reflect higher level visual and visuo-spatial processing underlying consolidation. Learning-related increases in cerebellar responses are consistent with its proposed role in the acquisition of internal models of the motor apparatus. These learning-related changes suggest a role for interacting neural systems involving the co-operation of cortico-cortico, cortico-cerebellar and cortico-basal ganglia loops during visuo-motor transformation learning.

Acquisition and generalization of visuomotor transformations by nonhuman primates.

The kinematics of straight reaching movements can be specified vectorially by the direction of the movement and its extent. To explore the representation in the brain of these two properties, psychophysical studies have examined learning of visuomotor transformations of either rotation or gain and their generalization. However, the neuronal substrates of such complex learning are only beginning to be addressed. As an initial step in ensuring the validity of such investigations, it must be shown that monkeys indeed learn and generalize visuomotor transformations in the same manner as humans. Here, we analyze trajectories and velocities of movements as monkeys adapt to either rotational or gain transformations. We used rotations with different signs and magnitudes, and gains with different signs, and analyzed transfer of learning to untrained movements. The results show that monkeys can adapt to both types of transformation with a time course that resembles human learning. Analysis of the aftereffects reveals that rotation is learned locally and generalizes poorly to untrained directions, whereas gain is learned more globally and can be transferred to other amplitudes. The results lend additional support to the hypothesis that reaching movements are learned locally but can be easily rescaled to other magnitudes by scaling the peak velocity. The findings also indicate that reaching movements in monkeys are planned and executed very similarly to those in humans. This validates the underlying presumption that neuronal recordings in primates can help elucidate the mechanisms of motor learning in particular and motor planning in general.

Learning Multiple Visuomotor Transformations: Adaptation and Context-Dependent Recall

Recent motor control theories suggest that the brain uses internal models to plan and control accurate movements. An internal model is thought to represent how the biomechanics of the arm interacting with the outside world would respond to a motor command; therefore it can be seen as a predictive model of the reafference that helps the system plan ahead. Moreover, adaptation studies show that humans can learn multiple internal models. It is not clear, however, whether and how contextual cues are used to switch among competing internal models, which are required to compensate for altered environments. To investigate this question, we asked healthy participants to perform center-out pointing movements under normal and distorted visual feedback (0 degrees , 30 degrees counterclockwise, and 60 degrees clockwise rotation of hand-screen cursor relationships) conditions. The results suggest that humans can learn multiple environments simultaneously and can use contextual cues to facilitate adaptation and to recall the appropriate internal model of the visuomotor transformation.

Target-, limb-, and context-dependent neural activity in the cingulate and supplementary motor areas of the monkey.

Very little is known about the role of the cingulate motor area (CMA) in visually guided reaching compared to other cortical motor areas. To investigate the hierarchical role of the caudal CMA (CMAc) during reaching we recorded the activity of neurons in CMAc in comparison to the supplementary motor area proper (SMA) while a monkey performed an instructed delay task that required it to position a cursor over visual targets on a computer screen using two-dimensional (2D) joystick movements. The direction of the monkey's arm movement was dissociated from the direction of the visual target by periodically reversing the relationship between the direction of movement of the joystick and that of the cursor. Neurons that responded maximally with a particular limb movement direction regardless of target location were classified as limb-dependent, whereas neurons that responded maximally to a particular target direction regardless of the direction of limb movement were classified as target-dependent. Neurons whose activity was directional in one of the two visuomotor mapping conditions and non-directional or inactive in the other were categorized as context-dependent. Limb-dependent activity was observed more frequently than target-dependent activity in both CMAc and SMA proper during both the delay period (preparatory activity; CMAc, 17%; SMA, 31%) and during movement execution (CMAc, 49%, SMA, 48%). A modest percentage of neurons with preparatory activity were target-dependent in both CMAc (11%) and SMA proper (8%) and a similar percentage of neurons in both areas demonstrated target-dependent, movement activity (CMAc, 8%; SMA, 10%). The surprising finding was that a very large percentage of neurons in both areas displayed context-dependent activity either during the preparatory (CMAc, 72%; SMA, 61%) or movement (CMAc, 43%, SMA 42%) epochs of the task. These results show that neural activity in both CMAc and SMA can directly represent movement direction in either limb-centered or target-centered coordinates. The presence of target-dependent activity in CMAc, as well as SMA, suggests that both are involved in the transformation of visual target information into appropriate motor commands. Target-dependent activity has been found in the putamen, SMA, CMAc, dorsal and ventral premotor cortex, as well as primary motor cortex. This indicates that the visuomotor transformations required for visually guided reaching are carried out by a distributed network of interconnected motor areas. The large proportion of neurons with context-dependent activity suggests, however, that while both CMAc and SMA may play a role in the visuomotor transformation of target information into movement parameters, their activity is not solely coding parameters of movement, since their involvement in this process is highly condition-dependent.

1/f-type fluctuation in human visuomotor transformation.

In the absence of vision of the limb, movements toward a visual target exhibit substantial errors which are considered to originate mainly in the visuomotor transformation process. To determine the time-dependent property of human visuomotor transformation, we investigated the error sequences in movements toward visual target using scaling analyses. When subjects could see their controlling limb, the error sequences could not be distinguished from a random sequence. On the other hand, when the controlling limb was invisible, the error sequences were not random in order, but exhibited 1/f-type time correlation. This finding that the variation in human visuomotor transformation shows 1/f-type fluctuation provides a significant index for mathematical modeling and system identification in human visuomotor control.

Playing piano in the mind—an fMRI study on music imagery and performance in pianists

Reading of musical notes and playing piano is a very complex motor task which requires years of practice. In addition to motor skills, rapid and effective visuomotor transformation as well as processing of the different components of music like pitch, rhythm and musical texture are involved. The aim of the present study was the investigation of the cortical network which mediates music performance compared to music imagery in 12 music academy students playing the right hand part of a Bartok piece using functional magnetic resonance imaging (fMRI). In both conditions, fMRI activations of a bilateral frontoparietal network comprising the premotor areas, the precuneus and the medial part of Brodmann Area 40 were found. During music performance but not during imagery the contralateral primary motor cortex and posterior parietal cortex (PPC) bilaterally was active. This reflects the role of primary motor cortex for motor execution but not imagery and the higher visuomotor integration requirements during music performance compared to simulation. The notion that the same areas are involved in visuomotor transformation/motor planning and music processing emphasizes the multimodal properties of cortical areas involved in music and motor imagery in musicians.

Control of eye orientation: where does the brain's role end and the muscle's begin?

Our understanding of how the brain controls eye movements has benefited enormously from the comparison of neuronal activity with eye movements and the quantification of these relationships with mathematical models. Although these early studies focused on horizontal and vertical eye movements, recent behavioural and modelling studies have illustrated the importance, but also the complexity, of extending previous conclusions to the problems of controlling eye and head orientation in three dimensions (3-D). An important facet in understanding 3-D eye orientation and movement has been the discovery of mobile, soft-tissue sheaths or 'pulleys' in the orbit which might influence the pulling direction of extraocular muscles. Appropriately placed pulleys could generate the eye-position-dependent tilt of the ocular rotation axes which are characteristic for eye movements which follow Listing's law. Based on such pulley models of the oculomotor plant it has recently been proposed that a simple two-dimensional (2-D) neural controller would be sufficient to generate correct 3-D eye orientation and movement. In contrast to this apparent simplification in oculomotor control, multiple behavioural observations suggest that the visuo-motor transformations, as well as the premotor circuitry for saccades, pursuit eye movements and the vestibulo-ocular reflexes, must include a neural controller which operates in 3-D, even when considering an eye plant with pulleys. This review summarizes the most recent work and ideas on this controversy. In addition, by proposing directly testable hypotheses, we point out that, in analogy to the previously successful steps towards elucidating the neural control of horizontal eye movements, we need a quantitative characterization first of motoneuron and next of premotor neuron properties in 3-D before we can succeed in gaining further insight into the neural control of 3-D motor behaviours.

Left inferior parietal cortex integrates time and space during collision judgments

Left inferior parietal lobe lesions can cause perturbation of the space–time plans underlying skilled actions. But does the perceptual integration of spatiotemporal information use the same neural substrate or is the role of the left inferior parietal cortex restricted to visuomotor transformations? We use fMRI and a collision judgment paradigm to examine whether the left inferior parietal cortex integrates temporal and spatial variables in situations in which no complex action and no visuomotor transformation is required. We used a perceptual task in which healthy subjects indicated by simple button presses whether two moving objects (of the same or different size) would or would not collide with each other. This task of interest was contrasted with a control task that employed the same stimuli and identical motor responses but in which the size of the two moving objects had to be compared. To assess putative differential eye-movement effects both tasks were performed with and without central fixation. Analysis of the fMRI data (employing a random-effects model and SPM99) showed that collision judgments (relative to size judgments) provoked a significant increase in neural activity in the left inferior parietal cortex (supramarginal gyrus) only. These results show that left inferior parietal cortex is involved in the integration of perceptual spatiotemporal information and thus provide a neural correlate for the use of space–time plans (whose perturbation can lead to apraxia as originally hypothesized by Liepmann). Furthermore, the data suggest that the left supramarginal gyrus combines temporal and spatial variables more widely than previously supposed.

Optic ataxia as a deficit specific to the on-line control of actions

Optic ataxia is characterized by inaccuracies in body movements under visual control, and is a common consequence of damage to the posterior parietal lobes in humans. It is argued here that optic ataxia can be characterized as a deficit in the visual on-line guidance of actions, with action planning remaining relatively intact. This contrasts with the common view of optic ataxia as representing a deficit in the transformations that take place between visual inputs and motor outputs. Evidence in support of the planning-control view comes from the pattern of spared and disrupted behaviors in patients with optic ataxia. It is shown that spared behaviors are those that emphasize planning, whereas disrupted behaviors are those that emphasize control. In particular, recent studies have highlighted the inability of a patient with optic ataxia to make on-line adjustments to targets that change position during the movement. Taken in sum, the data from patients with optic ataxia is more consistent with the planning-control interpretation of optic ataxia than with the visuomotor transformation interpretation.

Neural basis for the processes that underlie visually guided and internally guided force control in humans.

Despite an intricate understanding of the neural mechanisms underlying visual and motor systems, it is not completely understood in which brain regions humans transfer visual information into motor commands. Furthermore, in the absence of visual information, the retrieval process for motor memory information remains unclear. We report an investigation where visuomotor and motor memory processes were separated from only visual and only motor activation. Subjects produced precision grip force during a functional MRI (fMRI) study that included four conditions: rest, grip force with visual feedback, grip force without visual feedback, and visual feedback only. Statistical and subtractive logic analyses segregated the functional process maps. There were three important observations. First, along with the well-established parietal and premotor cortical network, the anterior prefrontal cortex, putamen, ventral thalamus, lateral cerebellum, intermediate cerebellum, and the dentate nucleus were directly involved in the visuomotor transformation process. This activation occurred despite controlling for the visual input and motor output. Second, a detailed topographic orientation of visuomotor to motor/sensory activity was mapped for the premotor cortex, parietal cortex, and the cerebellum. Third, the retrieval of motor memory information was isolated in the dorsolateral prefrontal cortex, ventral prefrontal cortex, and anterior cingulate. The motor memory process did not extend to the supplementary motor area (SMA) and the basal ganglia. These findings provide evidence in humans for a model where a distributed network extends over cortical and subcortical regions to control the visuomotor transformation process used during visually guided tasks. In contrast, a localized network in the prefrontal cortex retrieves force output from memory during internally guided actions.

Impairments of manual tracking performance during spaceflight are associated with specific effects of microgravity on visuomotor transformations

In contrast to performance in cognitive tasks, tracking performance tends to deteriorate fairly consistently during spaceflight. We address the question whether this decrement results from specific effects of microgravity on motor control or from non-specific effects of the various other stressors present. In a case study we generalize the findings obtained with aiming movements, performed by the same cosmonaut with the same effectors as used for an unstable tracking task, to obtain hypotheses for specific changes of parameters of a simple model used to analyse tracking performance. Consistent with these hypotheses, we observed a reduction of limb stiffness in-flight, but a reduction of the tracking gain post-flight. The cross-task consistency of the observed changes does strongly suggest that the tracking impairment is at least partly caused by specific effects of microgravity on motor control, in particular by a mis-calibration of muscular forces which likely results from an underestimation of masses due to weightlessness.

Parietal lobe contribution to mental rotation demonstrated with rTMS.

A large number of imaging studies have identified a role for the posterior parietal lobe, in particular Brodmann's area 7 and the intraparietal sulcus (IPS), in mental rotation. Here we investigated whether neural activity in the posterior parietal lobe is essential for successful mental rotation performance by observing the effects of interrupting this activity during the execution of a mental rotation task. Repetitive transcranial magnetic stimulation (rTMS) was applied to posterior parietal locations estimated to overlie Brodmann's area 7 in the right and the left hemisphere, or to a posterior midline location (sham condition). In three separate experiments, rTMS (four pulses, 20 Hz) was delivered at these locations either 200-400, 400-600, or 600-800 msec after the onset of a mental rotation trial. Disrupting neural activity in the right parietal lobe interfered with task performance, but only when rTMS was delivered 400 to 600 msec after stimulus onset. Stimulation of the left parietal lobe did not reliably affect mental rotation performance at any of the time points investigated. The time-limited effect of rTMS was replicated in a fourth experiment that directly compared the effects of rTMS applied to the right parietal lobe either 200-400 or 400-600 msec into the mental rotation trial. The results indicate that the right superior posterior parietal lobe plays an essential role in mental rotation, consistent with its involvement in a variety of visuospatial and visuomotor transformations.

Graspable objects grab attention when the potential for action is recognized.

Visually guided grasping movements require a rapid transformation of visual representations into object-specific motor programs. Here we report that graspable objects may facilitate these visuomotor transformations by automatically grabbing visual spatial attention. Human subjects viewed two task-irrelevant objects--one was a 'tool', the other a 'non-tool'--while waiting for a target to be presented in one of the two object locations. Using event-related potentials (ERPs), we found that spatial attention was systematically drawn to tools in the right and lower visual fields, the hemifields that are dominant for visuomotor processing. Using event-related fMRI, we confirmed that tools grabbed spatial attention only when they also activated dorsal regions of premotor and prefrontal cortices, regions associated with visually guided actions and their planning. Although it is widely accepted that visual sensory gain aids perception, our results suggest that it may also have consequences for object-directed actions.

Hippocampal volumes analysis on magnetic resonance images in schizophrenic and non affected family members — genetic loading and impact of obstetric complications

Corticobasal syndrome (CBS) is a progressive neurodegenerative disorder with asymmetric presentation and course characterized by degeneration of basal ganglia and cortical structures. Limb apraxia is a commonly observed deficit in CBS. Few studies have examined comprehensively the nature of deficits in limb apraxia. The goal of our study was to investigate the severity of deficits in various conceptual and gesture production task modalities. CBS patients were divided in two groups based on the side of brain that was initially affected by the disease. Ten patients with right hemisphere presentation (RHP) and seven with left hemisphere presentation (LHP) were included. The results showed that while selective conceptual tasks deficits were present in both groups, the overall picture suggests preserved conceptual representations of tools and actions in CBS patients with either LHP or RHP. Both groups were impaired relative to controls on gesture production tasks. Performance on transitive gestures was more severely affected in both groups than intransitive gestures. Imitation was more severely affected than pantomime, suggesting deficits in visuomotor transformations. The addition of verbal cuing during concurrent imitation affected only the LHP patients, rendering them more impaired relative to controls in their imitation with verbal cuing as opposed to their imitation only performance. Imitation of non-representational gestures was least accurate and intransitive gestures were most accurate. Patients were more severely impaired relative to controls when holding the object and when they were shown pictures of tools to pantomime.

Ocular kinematics and eye-hand coordination

Eye-hand coordination is complicated by the fact that the eyes are constantly in motion relative to the head. This poses problems in interpreting the spatial information gathered from the retinas and using this to guide hand motion. In particular, eye-centered visual information must somehow be spatially updated across eye movements to be useful for future actions, and these representations must then be transformed into commands appropriate for arm motion. In this review, we present evidence that early visuomotor representations for arm movement are remapped relative to the gaze direction during each saccade. We find that this mechanism holds for targets in both far and near visual space. We then show how the brain incorporates the three dimensional, rotary geometry of the eyes when interpreting retinal images and transforming these into commands for arm movement. Next, we explore the possibility that hand-eye alignment is optimized for the eye with the best field of view. Finally, we describe how head orientation influences the linkage between oculocentric visual frames and bodycentric motor frames. These findings are framed in terms of our ‘conversion-on-demand’ model, in which only those representations selected for action are put through the complex visuomotor transformations required for interaction with objects in personal space, thus providing a virtual on-line map of visuomotor space.

When visuo-motor incongruence aids motor performance: the effect of perceiving motion structures during transformed visual feedback on bimanual coordination.

Two experiments are reported in which bimanual coordination tasks were performed under correct and transformed visual feedback conditions. Participants were to generate cyclical line-drawing patterns, with varying degrees of coordinative stability, while perceiving correct or transformed visual information of the trajectories on a screen. Visuo-motor transformations that dissociated the perceived movement direction from the actually generated direction, were applied to one or both limbs, resulting in varying degrees of perceptual grouping power. The transformed feedback did not influence the most stable coordination patterns (in-phase) whereas the accuracy and/or stability of the less stable coordination patterns (anti-phase and particularly orthogonal) benefited from particular visual feedback manipulations, i.e. when coherently grouped visual motion structures emerged, the quality of coordination improved significantly. These findings indicate that perceptual transformations aid the production of more complex coordination patterns, thereby underscoring the importance of perception-action coupling in bimanual coordination.

Engagement of gaze in capturing targets for future sequential manual actions.

We investigated the role of saccadic gaze fixations in encoding target locations for planning a future manual task consisting of a sequence of discrete target-oriented actions. We hypothesized that fixations of the individual targets are necessary for accurate encoding of target locations and that there is a transfer of sequence information from visual encoding to manual recall. Subjects viewed four targets presented at random positions on a screen. After various delays following target extinction, the subjects marked the remembered target locations on the screen with the tip of a hand-held stick. When the targets were presented simultaneously among distracting elements, the overall accuracy of marking increased with presentation time and total number of targets fixated because the subjects had to serially fixate the individual targets to locate them. Without distractors, the marking accuracy was similarly high regardless of duration of target presentation (0.25-8 s) and number of targets fixated; it was comparable to that with distractors when all four targets had been fixated. This indicates parallel encoding of target locations largely based on peripheral vision. Location memory was stable in these tasks over the delay periods investigated (0.5-8 s). With parallel encoding there was a "shrinkage" in the visuomotor transformation, i.e., the distances between the markings were systematically smaller than the corresponding inter-target distances. When the targets were presented sequentially without distractors, marking accuracy improved with the total number of targets fixated and shrinkage in the visuomotor transformation occurred only with parallel encoding, i.e., when subjects did not fixate the targets. In all experimental conditions for trials in which targets were fixated during encoding, there was little correspondence between the marking sequence and the sequence in which the targets were fixated. We conclude that subjects benefit from fixating targets for subsequent target-oriented manual actions when the targets are presented among distractors and when presented sequentially; when distinct targets are presented simultaneously against a blank background, they are efficiently encoded in parallel largely by peripheral vision.