Error-based Motor Learning Research Articles

Neural correlates of sensorimotor adaptation: thalamic contributions to learning from sensory prediction error

Understanding the neural mechanism of sensorimotor adaptation is essential to reveal how the brain learns from errors, a process driven by sensory prediction errors. While the previous literature has focused on cortical and cerebellar changes, the involvement of the thalamus has received less attention. This functional magnetic resonance imaging study aims to explore the neural substrates of learning from sensory prediction errors with an additional focus on the thalamus. Thirty participants adapted their goal-directed reaches to visual feedback rotations introduced in a step-wise manner, while reporting their predicted visual consequences of their movements intermittently. We found that adaptation initially engaged the cerebellum and fronto-parietal cortical regions, which persisted as adaptation progressed. By the end of adaptation, additional regions within the fronto-parietal cortex and medial pulvinar of the thalamus were recruited. Another finding was the involvement of bilateral medial dorsal nuclei, which showed a positive correlation with the level of motor adaptation. Notably, the gradual shift in the predicted hand movement consequences was associated with activity in the cerebellum, motor cortex and thalamus (ventral lateral, medial dorsal, and medial pulvinar). Our study presents clear evidence for an involvement of the thalamus, both classical ‘motor’ and higher-order nuclei, in error-based motor learning.

Explicit learning based on reward prediction error facilitates agile motor adaptations.

Error based motor learning can be driven by both sensory prediction error and reward prediction error. Learning based on sensory prediction error is termed sensorimotor adaptation, while learning based on reward prediction error is termed reward learning. To investigate the characteristics and differences between sensorimotor adaptation and reward learning, we adapted a visuomotor paradigm where subjects performed arm movements while presented with either the sensory prediction error, signed end-point error, or binary reward. Before each trial, perturbation indicators in the form of visual cues were presented to inform the subjects of the presence and direction of the perturbation. To analyse the interconnection between sensorimotor adaptation and reward learning, we designed a computational model that distinguishes between the two prediction errors. Our results indicate that subjects adapted to novel perturbations irrespective of the type of prediction error they received during learning, and they converged towards the same movement patterns. Sensorimotor adaptations led to a pronounced aftereffect, while adaptation based on reward consequences produced smaller aftereffects suggesting that reward learning does not alter the internal model to the same degree as sensorimotor adaptation. Even though all subjects had learned to counteract two different perturbations separately, only those who relied on explicit learning using reward prediction error could timely adapt to the randomly changing perturbation. The results from the computational model suggest that sensorimotor and reward learning operate through distinct adaptation processes and that only sensorimotor adaptation changes the internal model, whereas reward learning employs explicit strategies that do not result in aftereffects. Additionally, we demonstrate that when humans learn motor tasks, they utilize both learning processes to successfully adapt to the new environments.

Activity of the Substantia Nigra pars Reticulata During Saccade Adaptation.

When movements become inaccurate, the resultant error induces motor adaptation to improve accuracy. This error-based motor learning is regarded as a cerebellar function. However, the influence of the other brain areas on adaptation is poorly understood. During saccade adaptation, a type of error-based motor learning, the superior colliculus (SC) sends a post-saccadic error signal to the cerebellum to drive adaptation. Since the SC is directly inhibited by the substantia nigra pars reticulata (SNr), we hypothesized that the SNr might influence saccade adaptation by affecting the SC error signal. In fact, previous studies indicated that the SNr encodes motivation and motivation influences saccade adaptation. In this study, we first established that the SNr projects to the rostral SC, where small error signals are generated, in non-human primates. Then, we examined SNr activity while the animal underwent adaptation. SNr neurons paused their activity in association with the error. This pause was shallower and delayed compared to those of no-error trial saccades. The pause at the end of the adaptation was shallower and delayed compared to that at the beginning of the adaptation. The change in the inter-trial interval, an indicator of motivation, and adaptation speed had a positive correlation with the changes in the error-related pause. These results suggest that 1: the SNr exhibits a unique activity pattern during the error interval, 2: SNr activity increases during adaptation, consistent with the decrease in SC activity, and 3: motivational decay during the adaptation session might increase SNr activity and influence the adaptation speed.Significance StatementMovements impaired due to stroke, injury, or aging recover gradually. To repair an inaccurate movement, the brain measures the error of the movement and changes the movement to minimize the error. This process is called motor adaptation. Although it has been established that the cerebellum plays an essential role in this adaptation, the influence of the other brain areas on the adaptation process is not well defined. The observation that motivation levels influence adaptation led us to suspect that the basal ganglia may influence adaptation. Here, we reveal the substantia nigra pars reticulata activity changes during saccade adaptation in monkeys and suggest how the basal ganglia influence the error signal.

A triple distinction of cerebellar function for oculomotor learning and fatigue compensation.

The cerebellum implements error-based motor learning via synaptic gain adaptation of an inverse model, i.e. the mapping of a spatial movement goal onto a motor command. Recently, we modeled the motor and perceptual changes during learning of saccadic eye movements, showing that learning is actually a threefold process. Besides motor recalibration of (1) the inverse model, learning also comprises perceptual recalibration of (2) the visuospatial target map and (3) of a forward dynamics model that estimates the saccade size from corollary discharge. Yet, the site of perceptual recalibration remains unclear. Here we dissociate cerebellar contributions to the three stages of learning by modeling the learning data of eight cerebellar patients and eight healthy controls. Results showed that cerebellar pathology restrains short-term recalibration of the inverse model while the forward dynamics model is well informed about the reduced saccade change. Adaptation of the visuospatial target map trended in learning direction only in control subjects, yet without reaching significance. Moreover, some patients showed a tendency for uncompensated oculomotor fatigue caused by insufficient upregulation of saccade duration. According to our model, this could induce long-term perceptual compensation, consistent with the overestimation of target eccentricity found in the patients' baseline data. We conclude that the cerebellum mediates short-term adaptation of the inverse model, especially by control of saccade duration, while the forward dynamics model was not affected by cerebellar pathology.

Computational role of exploration noise in error-based de novo motor learning

The redundancy inherent to the human body is a central problem that must be solved by the brain when acquiring new motor skills. The problem of redundancy becomes particularly critical when learning a new motor policy from scratch in a novel environment and task (i.e., de novo learning). It has been proposed that motor variability could be leveraged to explore and identify task-potent motor commands, and recent results indicated a possible role of motor exploration in error-based motor learning, including in de novo learning tasks. However, the precise computational mechanisms underlying this role remain poorly understood. A new controller in a de novo motor task can potentially be learned by first using motor exploration to learn a sensitivity derivative, which can transform observed task errors into motor corrections, enabling the error-based learning of the controller. Although this approach has been discussed, the computational properties of exploration and how this mechanism can explain recent reports of motor exploration in error-based de-novo learning have not been thoroughly examined. Here, we used this approach to simulate the tasks used in several recent studies of human motor learning tasks in which motor exploration was observed, and replicating their main results. Analyses of the proposed learning mechanism using equations and simulations suggested that exploring the entire motor command space leads to the training of an efficient sensitivity derivative, enabling rapid learning of the controller, in visuomotor adaptation and de novo tasks. The successful replication of previous experimental results elucidated the role of motor exploration in motor learning.

Repeated adaptation and de-adaptation to the pelvis resistance force facilitate retention of motor learning in stroke survivors.

Locomotor adaptation to novel walking patterns induced by external perturbation has been tested to enhance motor learning for improving gait parameters in individuals poststroke. However, little is known regarding whether repeated adaptation and de-adaptation to the externally perturbed walking pattern may facilitate or degrade the retention of locomotor learning. In this study, we examined whether the intermittent adaptation to novel walking patterns elicited by external perturbation induces greater retention of the adapted locomotion in stroke survivors, compared with effects of the continuous adaptation. Fifteen individuals poststroke participated in two experimental conditions consisting of 1) treadmill walking with intermittent (i.e., interspersed 2 intervals of no perturbation) or continuous (no interval) adaptation to externally perturbed walking patterns and 2) overground walking before, immediately, and 10 min after treadmill walking. During the treadmill walking, we applied a laterally pulling force to the pelvis toward the nonparetic side during the stance phase of the paretic leg to disturb weight shifts toward the paretic side. Participants showed improved weight shift toward the paretic side and enhanced muscle activation of hip abductor/adductors immediately after the removal of the pelvis perturbation for both intermittent and continuous conditions (P < 0.05) and showed longer retention of the improved weight shift and enhanced muscle activation for the intermittent condition, which transferred from treadmill to overground walking (P < 0.05). In conclusion, repeated motor adaptation and de-adaptation to the pelvis resistance force during walking may promote the retention of error-based motor learning for improving weight shift toward the paretic side in individuals poststroke.NEW & NOTEWORTHY We examined whether the intermittent versus the continuous adaptation to external perturbation induces greater retention of the adapted locomotion in stroke survivors. We found that participants showed longer retention of the improved weight shift and enhanced muscle activation for the intermittent versus the continuous conditions, suggesting that repeated motor adaptation and de-adaptation to the pelvis perturbation may promote the retention of error-based motor learning for improving weight shift toward the paretic side in individuals poststroke.

Task-relevant and task-irrelevant variability causally shape error-based motor learning

Recent studies of motor learning show dissociable roles of reward- and sensory-prediction errors in updating motor commands by using typical adaptation paradigms where force or visual perturbations are imposed on hand movements. Such classic adaptation paradigms ignore a problem of redundancy inherently embedded in the motor pathways where the central nervous system has to find a unique solution in the high-dimensional motor command space. Computationally, a possible way of solving such a redundancy problem is exploring and updating motor commands based on the learned knowledge of the structures of both the motor pathways and the tasks. However, the effects of task-irrelevant motor command exploration in structure learning and its effects on reward-based and error-based learning have yet to be examined. Here, we used a redundant motor task where participants manipulated a cursor on a monitor screen with their hand gesture movements and then analyzed single-trial motor learning by fitting models consisting of reward-based and error-based learning contributions. We found that the error-based learning rate positively correlated with both task-relevant and task-irrelevant variability, likely reflecting the effect of motor exploration in structure learning. Further modeling results show that the effects of both task-relevant and task-irrelevant variability are simultaneous, and not mediated by one another. In contrast, the reward-based learning rate correlated with neither task-relevant nor task-irrelevant variability. Thus, although not having a direct influence on the task outcome, exploration in the task-irrelevant space late in training has a significant effect on the learning of a task structure used for error-based learning. This suggests that motor exploration, in both task-relevant and task-irrelevant spaces, has an essential role in error-based motor learning in a redundant motor mechanism.

The Substantia Nigra Pars Reticulata Modulates Error-Based Saccadic Learning in Monkeys.

The basal ganglia have long been considered crucial for associative learning, but whether they also are involved in another type of learning, error-based motor learning, is not clear. Error-based learning has been considered the province of the cerebellum. However, learning to use a robotic arm and saccade adaptation, which use error-based learning, are facilitated by motivation, which is a function of the basal ganglia. Additionally, patients with Parkinson’s disease, a basal ganglia deficit, show slower saccade adaptation than age matched controls. To further investigate whether the basal ganglia actually influence error-based learning, we reversibly inactivated the oculomotor portion of the substantia nigra pars reticulata (SNr) in two monkeys and tested saccade adaptation. Here, we show that nigral inactivation affected saccade adaptation. In particular, the inactivation facilitated the amplitude decrease adaptation of ipsiversive saccades. Consistent with previous studies, no effect was seen on the amplitude of the ipsiversive saccades when we did not induce adaptation. Therefore, the facilitated adaptation was not caused by inactivation directly modulating ipsiversive saccades. On the other hand, the kinematics of corrective saccades, which represent error processing, were changed after the inactivation. Thus, our data suggest that the oculomotor SNr assists saccade adaptation by strengthening the error signal. This effect indicates the basal ganglia influence error-based motor learning, a previously unrecognized function.

Dysmetria and Errors in Predictions: The Role of Internal Forward Model.

The terminology of cerebellar dysmetria embraces a ubiquitous symptom in motor deficits, oculomotor symptoms, and cognitive/emotional symptoms occurring in cerebellar ataxias. Patients with episodic ataxia exhibit recurrent episodes of ataxia, including motor dysmetria. Despite the consensus that cerebellar dysmetria is a cardinal symptom, there is still no agreement on its pathophysiological mechanisms to date since its first clinical description by Babinski. We argue that impairment in the predictive computation for voluntary movements explains a range of characteristics accompanied by dysmetria. Within this framework, the cerebellum acquires and maintains an internal forward model, which predicts current and future states of the body by integrating an estimate of the previous state and a given efference copy of motor commands. Two of our recent studies experimentally support the internal-forward-model hypothesis of the cerebellar circuitry. First, the cerebellar outputs (firing rates of dentate nucleus cells) contain predictive information for the future cerebellar inputs (firing rates of mossy fibers). Second, a component of movement kinematics is predictive for target motions in control subjects. In cerebellar patients, the predictive component lags behind a target motion and is compensated with a feedback component. Furthermore, a clinical analysis has examined kinematic and electromyography (EMG) features using a task of elbow flexion goal-directed movements, which mimics the finger-to-nose test. Consistent with the hypothesis of the internal forward model, the predictive activations in the triceps muscles are impaired, and the impaired predictive activations result in hypermetria (overshoot). Dysmetria stems from deficits in the predictive computation of the internal forward model in the cerebellum. Errors in this fundamental mechanism result in undershoot (hypometria) and overshoot during voluntary motor actions. The predictive computation of the forward model affords error-based motor learning, coordination of multiple degrees of freedom, and adequate timing of muscle activities. Both the timing and synergy theory fit with the internal forward model, microzones being the elemental computational unit, and the anatomical organization of converging inputs to the Purkinje neurons providing them the unique property of a perceptron in the brain. We propose that motor dysmetria observed in attacks of ataxia occurs as a result of impaired predictive computation of the internal forward model in the cerebellum.

Reinforcement Signaling Can Be Used to Reduce Elements of Cerebellar Reaching Ataxia.

Damage to the cerebellum causes a disabling movement disorder called ataxia, which is characterized by poorly coordinated movement. Arm ataxia causes dysmetria (over- or under-shooting of targets) with many corrective movements. As a result, people with cerebellar damage exhibit reaching movements with highly irregular and prolonged movement paths. Cerebellar patients are also impaired in error-based motor learning, which may impede rehabilitation interventions. However, we have recently shown that cerebellar patients can learn a simple reaching task using a binary reinforcement paradigm, in which feedback is based on participants' mean performance. Here, we present a pilot study that examined whether patients with cerebellar damage can use this reinforcement training to learn a more complex motor task-to decrease the path length of their reaches. We compared binary reinforcement training to a control condition of massed practice without reinforcement feedback. In both conditions, participants made target-directed reaches in 3-dimensional space while vision of their movement was occluded. In the reinforcement training condition, reaches with a path length below participants' mean were reinforced with an auditory stimulus at reach endpoint. We found that patients were able to use reinforcement signaling to significantly reduce their reach paths. Massed practice produced no systematic change in patients' reach performance. Overall, our results suggest that binary reinforcement training can improve reaching movements in patients with cerebellar damage and the benefit cannot be attributed solely to repetition or reduced visual control.

Use of a Zorb Bumper Ball in rehabilitation of a patient with ataxic multiple sclerosis: A case report

Multiple sclerosis affecting the cerebellum and associated pathways can result in debilitating ataxia. Recent research has shown that reinforcement and goal-based learning may be more effective in improving motor outcomes than error-based motor learning in patients with cerebellar deficits due to decreased intrinsic feedback mechanisms.

Basal ganglia contributions during the learning of a visuomotor rotation: Effect of dopamine, deep brain stimulation and reinforcement.

It is commonly thought that visuomotor adaptation is mediated by the cerebellum while reinforcement learning is mediated by the basal ganglia. In contrast to this strict dichotomy, we demonstrate a role for the basal ganglia in visuomotor adaptation (error-based motor learning) in patients with Parkinson's disease (PD) by comparing the degree of motor learning in the presence and absence of dopamine medication. We further show similar modulation of learning rates in the presence and absence of subthalamic deep brain stimulation. We also report that reinforcement is an essential component of visuomotor adaptation by demonstrating the lack of motor learning in patients with PD during the ON-dopamine state relative to the OFF-dopamine state in the absence of a reinforcement signal. Taken together, these results raise the possibility that the basal ganglia modulate the gain of visuomotor adaptation based on the reinforcement received at the end of the trial.

Eliminating Direction Specificity in Visuomotor Learning.

Motor learning is more useful if it generalizes to untrained scenarios when needed, especially for sports training and motor rehabilitation. However, as a form of motor learning, motor adaptation is typically direction specific. Here we first show that savings with motor adaptation, an index for model-free learning and explicit strategy learning in motor learning, is also direction specific. However, the participants' additional exposure to untrained directions via an irrelevant gain-learning task can enable the complete generalization of learning. Our findings challenge existing models of motor generalization and may have important implications for practical applications.

The dissociable effects of punishment and reward on motor learning.

A common assumption regarding error-based motor learning (motor adaptation) in humans is that its underlying mechanism is automatic and insensitive to reward- or punishment-based feedback. Contrary to this hypothesis, we show in a double dissociation that the two have independent effects on the learning and retention components of motor adaptation. Negative feedback, whether graded or binary, accelerated learning. While it was not necessary for the negative feedback to be coupled to monetary loss, it had to be clearly related to the actual performance on the preceding movement. Positive feedback did not speed up learning, but it increased retention of the motor memory when performance feedback was withdrawn. These findings reinforce the view that independent mechanisms underpin learning and retention in motor adaptation, reject the assumption that motor adaptation is independent of motivational feedback, and raise new questions regarding the neural basis of negative and positive motivational feedback in motor learning.

Redistribution of neural phase coherence reflects establishment of feedforward map in speech motor adaptation.

Despite recent progress in our understanding of sensorimotor integration in speech learning, a comprehensive framework to investigate its neural basis is lacking at behaviorally relevant timescales. Structural and functional imaging studies in humans have helped us identify brain networks that support speech but fail to capture the precise spatiotemporal coordination within the networks that takes place during speech learning. Here we use neuronal oscillations to investigate interactions within speech motor networks in a paradigm of speech motor adaptation under altered feedback with continuous recording of EEG in which subjects adapted to the real-time auditory perturbation of a target vowel sound. As subjects adapted to the task, concurrent changes were observed in the theta-gamma phase coherence during speech planning at several distinct scalp regions that is consistent with the establishment of a feedforward map. In particular, there was an increase in coherence over the central region and a decrease over the fronto-temporal regions, revealing a redistribution of coherence over an interacting network of brain regions that could be a general feature of error-based motor learning in general. Our findings have implications for understanding the neural basis of speech motor learning and could elucidate how transient breakdown of neuronal communication within speech networks relates to speech disorders.

Postural dependence of human locomotion during gait initiation.

The initiation of human walking involves postural motor actions for body orientation and balance stabilization that must be effectively integrated with locomotion to allow safe and efficient transport. Our ability to coordinately adapt these functions to environmental or bodily changes through error-based motor learning is essential to effective performance. Predictive compensations for postural perturbations through anticipatory postural adjustments (APAs) that stabilize mediolateral (ML) standing balance normally precede and accompany stepping. The temporal sequencing between these events may involve neural processes that suppress stepping until the expected stability conditions are achieved. If so, then an unexpected perturbation that disrupts the ML APAs should delay locomotion. This study investigated how the central nervous system (CNS) adapts posture and locomotion to perturbations of ML standing balance. Healthy human adults initiated locomotion while a resistance force was applied at the pelvis to perturb posture. In experiment 1, using random perturbations, step onset timing was delayed relative to the APA onset indicating that locomotion was withheld until expected stability conditions occurred. Furthermore, stepping parameters were adapted with the APAs indicating that motor prediction of the consequences of the postural changes likely modified the step motor command. In experiment 2, repetitive postural perturbations induced sustained locomotor aftereffects in some parameters (i.e., step height), immediate but rapidly readapted aftereffects in others, or had no aftereffects. These results indicated both rapid but transient reactive adaptations in the posture and gait assembly and more durable practice-dependent changes suggesting feedforward adaptation of locomotion in response to the prevailing postural conditions.

Neurocognitive mechanisms of error-based motor learning.

One mechanism for acquiring new motor skills is minimization of errors from one practice trial to the next. A substantial body of literature supports a role for cerebellar pathways in such adaptive motor error minimization processes. A region in the medial prefrontal cortex, including the anterior cingulate cortex, has been linked to performance monitoring and error detection processes for cognitive tasks. Recent findings support the notion that this region is also sensitive to the commission of motor errors. Furthermore, the basal ganglia nuclei also exhibit neural activity which varies with both errors and rewards. Here, we review the literature supporting a potential role for each of these networks in error-based motor learning, focusing on both feedback and feedforward control processes. We also speculate about the relative independence versus interactivity of their respective functions.

Rethinking Motor Learning and Savings in Adaptation Paradigms: Model-Free Memory for Successful Actions Combines with Internal Models

Although motor learning is likely to involve multiple processes, phenomena observed in error-based motor learning paradigms tend to be conceptualized in terms of only a single process: adaptation, which occurs through updating an internal model. Here we argue that fundamental phenomena like movement direction biases, savings (faster relearning), and interference do not relate to adaptation but instead are attributable to two additional learning processes that can be characterized as model-free: use-dependent plasticity and operant reinforcement. Although usually "hidden" behind adaptation, we demonstrate, with modified visuomotor rotation paradigms, that these distinct model-based and model-free processes combine to learn an error-based motor task. (1) Adaptation of an internal model channels movements toward successful error reduction in visual space. (2) Repetition of the newly adapted movement induces directional biases toward therepeated movement. (3) Operant reinforcement through association of the adapted movement with successful error reduction is responsible for savings.