Outgoing Motor Commands Research Articles

Did We Get Sensorimotor Adaptation Wrong? Implicit Adaptation as Direct Policy Updating Rather than Forward-Model-Based Learning.

The human motor system can rapidly adapt its motor output in response to errors. The prevailing theory of this process posits that the motor system adapts an internal forward model that predicts the consequences of outgoing motor commands and uses this forward model to plan future movements. However, despite clear evidence that adaptive forward models exist and are used to help track the state of the body, there is no definitive evidence that such models are used in movement planning. An alternative to the forward-model-based theory of adaptation is that movements are generated based on a learned policy that is adjusted over time by movement errors directly ("direct policy learning"). This learning mechanism could act in parallel with, but independent of, any updates to a predictive forward model. Forward-model-based learning and direct policy learning generate very similar predictions about behavior in conventional adaptation paradigms. However, across three experiments with human participants (N = 47, 26 female), we show that these mechanisms can be dissociated based on the properties of implicit adaptation under mirror-reversed visual feedback. Although mirror reversal is an extreme perturbation, it still elicits implicit adaptation; however, this adaptation acts to amplify rather than to reduce errors. We show that the pattern of this adaptation over time and across targets is consistent with direct policy learning but not forward-model-based learning. Our findings suggest that the forward-model-based theory of adaptation needs to be re-examined and that direct policy learning provides a more plausible explanation of implicit adaptation.SIGNIFICANCE STATEMENT The ability of our brain to adapt movements in response to error is one of the most widely studied phenomena in motor learning. Yet, we still do not know the process by which errors eventually result in adaptation. It is known that the brain maintains and updates an internal forward model, which predicts the consequences of motor commands, and the prevailing theory of motor adaptation posits that this updated forward model is responsible for trial-by-trial adaptive changes. Here, we question this view and show instead that adaptation is better explained by a simpler process whereby motor output is directly adjusted by task errors. Our findings cast doubt on long-held beliefs about adaptation.

Skilled reaching relies on a V2a propriospinal internal copy circuit

The precision of skilled forelimb movement has long been presumed to rely on rapid feedback corrections triggered by internally-directed copies of outgoing motor commands – but the functional relevance of inferred internal copy circuits has remained unclear. One class of spinal interneurons implicated in the control of mammalian forelimb movement, cervical propriospinal neurons (PNs), has the potential to convey an internal copy of pre-motor signals through dual innervation of forelimb-innervating motor neurons and pre-cerebellar neurons of the lateral reticular nucleus. We have examined whether the PN internal copy pathway functions in the control of goal-directed reaching. In mice, PNs include a genetically-accessible subpopulation of cervical V2a interneurons, and their targeted ablation perturbs reaching while leaving intact other elements of forelimb movement. Moreover, optogenetic activation of the PN internal copy branch recruits a rapid cerebellar feedback loop that modulates forelimb motor neuron activity and severely disrupts reaching kinematics. Our findings implicate V2a PNs as the focus of an internal copy pathway assigned to the rapid updating of motor output during reaching behavior.

Neural evidence for state feedback control of speaking

An important recent development in neuroscience has been the use of models based on state feedback control (SFC) to explain the role of the central nervous system in motor control. In SFC, control is based on internal feedback of an estimate of the dynamic state of the thing (e.g., arm) being controlled. Within the internal loop, the state is predicted from outgoing motor commands and corrected by comparing the feedback expected to result from this state with actual incoming sensory feedback. SFC has received scant attention in the speech community, but the indirect role it suggests for feedback can account for much of what is known about the role of feedback in speech motor control. Our lab has been investigating how well SFC also accounts for the neural correlates of auditory feedback processing during speaking. Our principal approach has used magnetoencephalography to record the cortical activity of speakers as they hear themselves speaking, but recently, we have also completed an auditory feedback study based on electrocorticography. Many of the results of these studies have supported the SFC model, but some have posed challenges for it, which will be discussed. [Work supported by NSF grant BCS-0926196 and NIH grant R01-DC010145.]

Implications of different classes of sensorimotor disturbance for cerebellar-based motor learning models

The exact role of the cerebellum in motor control and learning is not yet fully understood. The structure, connectivity and plasticity within cerebellar cortex has been extensively studied, but the patterns of connectivity and interaction with other brain structures, and the computational significance of these patterns, is less well known and a matter of debate. Two contrasting models of the role of the cerebellum in motor adaptation have previously been proposed. Most commonly, the cerebellum is employed in a purely feedforward pathway, with its output contributing directly to the outgoing motor command. The cerebellum must then learn an inverse model of the motor apparatus in order to achieve accurate control. More recently, Porrill et al. (Proc Biol Sci 271(1541):789-796, 2004) and Porrill et al. (PLoS Comput Biol 3:1935-1950, 2007a) and Porrill et al. (Neural Comput 19(1), 170-193, 2007b) have highlighted the potential importance of these recurrent connections by proposing an alternative architecture in which the cerebellum is embedded in a recurrent loop with brainstem control circuitry. In this framework, the feedforward connections are not necessary at all. The cerebellum must learn a forward model of the motor apparatus for accurate motor commands to be generated. We show here how these two models exhibit contrasting yet complimentary learning capabilities. Central to the differences in performance between architectures is that there are two distinct kinds of disturbance to which a motor system may need to adapt (1) changes in the relationship between the motor command and the observed outcome and (2) changes in the relationship between the stimulus and the desired outcome. The computational distinction between these two kinds of transformation is subtle and has therefore often been overlooked. However, the implications for learning turn out to be significant: learning with a feedforward architecture is robust following changes in the stimulus-desired outcome mapping but not necessarily the motor command-outcome mapping, while learning with a recurrent architecture is robust under changes in the motor command-outcome mapping but not necessarily the stimulus-desired outcome mapping. We first analyse these differences theoretically and through simulations in the vestibulo-ocular reflex (VOR), then illustrate how these same concepts apply more generally with a model of reaching movements.

Somatosensory temporal discrimination learning generalizes to motor interval production

The present study investigated whether a common timing mechanism underlies the ability to analyze incoming sensory information and control outgoing motor commands. Participants were presented with two pairs of air puffs on the ventral surface of the right forearm. One pair (the standard interval) was separated by 500 ms on every trial for half of the participants (“500 group”) and 800 ms on every trial for the other half (“800 group”). The duration of the comparison interval was always longer but varied adaptively to determine discrimination thresholds. Participants indicated which of the two intervals was longer. Both groups performed two motor interval production tasks (pressing a button twice in succession with the right thumb) before and again after somatosensory training. The target inter-press interval was 500 ms in one task and 800 ms in the other. A critical feature of the design was that only one of the motor tasks for each of the groups shared temporal properties with the somatosensory discrimination task. The results showed that somatosensory discrimination learning generalizes to motor interval production when the two tasks share temporal properties. Specifically, the 500 group showed a greater reduction in motor timing variability on the 500 ms task than the 800 ms task, whereas the 800 group showed a greater reduction in motor timing variability on the 800 ms task than the 500 ms task. The possible neural basis of temporal learning generalization is discussed.

Forward estimation of movement state in posterior parietal cortex

During goal-directed movements, primates are able to rapidly and accurately control an online trajectory despite substantial delay times incurred in the sensorimotor control loop. To address the problem of large delays, it has been proposed that the brain uses an internal forward model of the arm to estimate current and upcoming states of a movement, which are more useful for rapid online control. To study online control mechanisms in the posterior parietal cortex (PPC), we recorded from single neurons while monkeys performed a joystick task. Neurons encoded the static target direction and the dynamic movement angle of the cursor. The dynamic encoding properties of many movement angle neurons reflected a forward estimate of the state of the cursor that is neither directly available from passive sensory feedback nor compatible with outgoing motor commands and is consistent with PPC serving as a forward model for online sensorimotor control. In addition, we found that the space–time tuning functions of these neurons were largely separable in the angle–time plane, suggesting that they mostly encode straight and approximately instantaneous trajectories.

Human Visuospatial Updating After Passive Translations in Three-Dimensional Space

To maintain a stable representation of the visual environment as we move, the brain must update the locations of targets in space using extra-retinal signals. Humans can accurately update after intervening active whole-body translations. But can they also update for passive translations (i.e., without efference copy signals of an outgoing motor command)? We asked six head-fixed subjects to remember the location of a briefly flashed target (five possible targets were located at depths of 23, 33, 43, 63, and 150 cm in front of the cyclopean eye) as they moved 10 cm left, right, up, down, forward, or backward while fixating a head-fixed target at 53 cm. After the movement, the subjects made a saccade to the remembered location of the flash with a combination of version and vergence eye movements. We computed an updating ratio where 0 indicates no updating and 1 indicates perfect updating. For lateral and vertical whole-body motion, where updating performance is judged by the size of the version movement, the updating ratios were similar for leftward and rightward translations, averaging 0.84 +/- 0.28 (mean +/- SD) as compared with 0.51 +/- 0.33 for downward and 1.05 +/- 0.50 for upward translations. For forward/backward movements, where updating performance is judged by the size of the vergence movement, the average updating ratio was 1.12 +/- 0.45. Updating ratios tended to be larger for far targets than near targets, although both intra- and intersubject variabilities were smallest for near targets. Thus in addition to self-generated movements, extra-retinal signals involving otolith and proprioceptive cues can also be used for spatial constancy.

Afferent Input, Efference Copy, Signal Noise, and Biases in Perception of Joint Angle During Active Versus Passive Elbow Movements

Psychophysical studies have reported an overestimation of limb position in the direction of movement during the early part of active movements. The main hypothesis tested in this study is that the overestimation results from a process of forward prediction of limb state driven by an efference copy of the outgoing motor command. This hypothesis predicts that position overestimation should decrease or disappear during passive movements, for which there should be no efference copy. Seven subjects were asked to remember and to report the perceived angle of their elbow joint at different times during active and passive movements. They showed a highly velocity-dependent overestimation of the elbow joint angle near the beginning of the movement in both active and passive trials. Toward the end of the movement, subjects showed a relatively velocity-independent underestimation of their elbow angle in all trials. Contrary to the prediction of the efference copy hypothesis, the amplitude and the velocity-dependent slope of the elbow angle overestimation were both greater during the early part of passive movements than active movements. This indicates that psychophysical evidence of early overestimation of arm position on its own is not a sufficient proof of forward prediction based on an efference copy, at least under the conditions of this study. Decreased errors during active movements suggest that an efference copy can improve the accuracy of state estimation during active movements. Error patterns seem to parallel the likely level of sensorimotor noise, suggesting a probabilistic mechanism for position estimation.

Motor fluency deficits in the sequencing of actions in schizophrenia.

Many everyday activities depend on the capacity to organize and smoothly execute motor sequences. The authors tested the hypothesis that a sequencing deficit is associated with schizophrenia. They used a new method to distinguish between lower and higher order mechanisms for the impairment. The 1st task involved triggered sequences in which sensory information from 1 movement was the cue for initiation of the following movement. Results showed that the motor sequences were performed as fluently in patients as in controls. The 2nd and 3rd tasks involved sequences in which the entire movement sequence could be preplanned. Patients executed the sequences less fluently than controls but only under the condition where action sequences were required. Furthermore, the patients' fluency deficit increased with sequence complexity. The high discrimination power of Task 3 gave the authors a means to control for a potential psychometric confound involving differential discriminating power and to argue in favor of a specific higher order motor fluency deficit in patients with schizophrenia. It is suggested that basic lower order mechanisms that integrate sensory information with outgoing motor commands are preserved in schizophrenia, whereas higher order integrative mechanisms that are required for the smooth coordination of motor sequences are impaired.

Mechanisms and Measurement of Dyspnea in Chronic Obstructive Pulmonary Disease

Patients with chronic obstructive pulmonary disease (COPD) describe their breathlessness as related to the work and effort associated with breathing. Current evidence suggests that the perception of dyspnea is due to a "mismatch" between the outgoing motor command from the central nervous system and the corresponding afferent information from chemoreceptors and/or mechanoreceptors. To measure the severity of dyspnea the principles of psychophysics (stimulus --> response relationship) can be applied. One approach is to consider activities of daily living as a putative stimulus. Although this method relies on patient recall and description of daily tasks, ability to function, as well as time and effort to complete an activity, select clinical instruments have demonstrated appropriate measurement criteria in randomized clinical trials involving patients with COPD. Another approach is for a patient to report the intensity of dyspnea during exercise, and current practice is to provide ratings each minute "on cue" during the exercise test. A computerized system has been developed whereby the person can report ratings spontaneously and continuously by moving a computer mouse that adjusts a vertical bar adjacent to 0-10 category-ratio scale positioned on a monitor. With this continuous method the patient reports twice the number of dyspnea ratings during exercise compared with discrete ratings each minute. Patient-reported dyspnea based on activities of daily living and exercise testing provides distinct but complimentary information.

Awareness of muscular force during movement production: an fMRI study

Awareness of the muscular forces we produce during voluntary movement must be distinguished from awareness of motor outcome itself. Indeed, there is no univocal relationship between produced muscle force and movement outcome because of external forces. In the present study, we performed a functional magnetic resonance imaging study to investigate the neural bases underlying the awareness we can have of the muscular forces we put into our voluntary movements. In reference conditions, subjects made rhythmical hand movements and knew they had to reproduce, in a subsequent condition in which the resistance to the movement was increased, either their muscular forces or their kinematics. The idea behind this (well established) reproduction paradigm is that, after an explicit verbal instruction, subjects can only reproduce what they are aware off. The main contrast, that is, between the condition during which the subjects had to gain awareness of their muscular forces and that during which they had to gain awareness of their kinematics (conditions in which the actual motor output was similar), shows that gaining awareness about muscular forces exerted during movement execution makes much higher demands on many brain structures, in particular posterior insula, primary sensorimotor areas and associative somatosensory areas. This indicates the important role of somesthetic information processing in awareness of produced muscular force. Therefore, the often-heard presumption that muscle force sense might be based on the outgoing motor command is not confirmed by the present results.

Sensory prediction as a role for the cerebellum

We suggest that the cerebellum generates sensory or “state” estimates based on outgoing motor commands and sensory feedback. Thus, it is not a motor pattern generator (HOUK et al.) but a predictive system which is intimately involved in motor behavior. This theory may explain the sensitivity of the climbing fibers to both unexpected external events and motor errors (SIMPSON et al.), and we speculate that unusual biophysical properties of the inferior olive might allow the cerebellum to develop multiple asynchronous sensory estimates, [HOUK et al.; SIMPSON et al.; THACH]

Respiratory effort perception at rest and during carbon dioxide rebreathing in patients with dystrophia myotonica.

Breathlessness appears to be closely related to the perception of the outgoing motor command to breathe and should be increased in the presence of muscle weakness. However, breathlessness is not a common symptom in patients with chronic muscle disease who have weak respiratory muscles. The factors that determine the perception of respiratory effort in such patients have not been examined. The inspiratory effort sensation during resting breathing and progressive hypercapnia was investigated in 12 patients with dystrophia myotonica with weak respiratory muscles (nine men and three women of mean (SD) age 41.1 (10.5) years; maximum inspiratory pressure 43.1 (17.2) cm H2O) and an age and sex matched control group of normal subjects of mean age 39.6 (10.6) years and a maximum inspiratory pressure of 123 (15.2) cm H2O. During resting breathing with a mouthpiece no differences were seen in inspiratory effort sensation, mouth occlusion pressure, or tidal volume, but inspiratory time and cycle duration were significantly shorter in the patients with dystrophia. Minute ventilation (VE) was significantly higher in the patients (15.8 (4.0) l/min v 12.5 (2.6) l/min), while resting breathing was no more variable in the patients than in controls. The ventilatory response to carbon dioxide (VE/PCO2) was not significantly lower in the patients (14.9 (6.9) l/min/kPa) than in the controls (17.4 (4.3) l/min/kPa). Effort sensation responses to carbon dioxide driven breathing were similar in the control subjects and the patients. With regression analysis of pooled data neither maximum inspiratory pressure nor disease state contributed to perceived inspiratory effort during hypercapnia. Moderately severe global respiratory muscle weakness does not appear to influence the ventilatory response to rising carbon dioxide tension or the perception of inspiratory effort in patients with dystrophia myotonica.

The problem of redundancy in movement control: the adaptive model theory approach.

The problem of redundancy in movement control is encountered when one attempts to answer the question: How does the central nervous system (CNS) determine the pattern of neural activity required in some 5,000,000 descending motor fibres to control only 100-150 biomechanical degrees of freedom of movement? Mathematically this is equivalent to solving a set of simultaneous equations with many more unknowns than equations. This system of equations is redundant because it has an infinite number of possible solutions. The problem is solved by the neuronal circuitry hypothesized in Adaptive Model Theory (AMT). According to AMT, the CNS includes neuronal circuitry able to compute and maintain adaptively the accuracy of internal models of the reciprocal multivariable relationships between outgoing motor commands and their resulting sensory consequences. To identify these input-output relationships by means of regression analysis, correlations between the input signals have to be taken into account. For example, if the inputs are perfectly correlated, the model reduces to a virtual one-input system. In general, the number of inputs modelled equals the number of degrees of freedom encoded by the signals; that is, the number of independently varying (orthogonal) signals. The adaptive modelling circuitry proposed in AMT automatically tunes itself to extract independently varying sensory and motor signals before computing the dynamic relationships between them. Inverse models are employed during response execution to translate movements pre-planned as desired trajectories of these high-level sensory-feature signals into appropriately co-ordinated motor commands to send to the muscles. Since movement is pre-planned in terms of a number of orthogonalized sensory-feature signals equal to the number of degrees of freedom in the desired response, the problem of redundancy is solved and the correlation or co-ordination between motor-command signals is automatically introduced by the adaptive models.

Internal models and intermittency: a theoretical account of human tracking behavior.

This paper concerns the use of tracking studies to test a theoretical account of the information processing performed by the human CNS during control of movement. The theory provides a bridge between studies of reaction time and continuous tracking. It is proposed that the human CNS includes neuronal circuitry to compute inverse internal models of the multiple input, multiple output, dynamic, non-linear relationships between outgoing motor commands and their resulting perceptual consequences. The inverse internal models are employed during movement execution to transform preplanned trajectories of desired perceptual consequences into appropriate outgoing motor commands to achieve them. A finite interval of time is required by the CNS to preplan the desired perceptual consequences of a movement and it does not commence planning a new movement until planning of the old one has been completed. This behavior introduces intermittency into the planning of movements. In this paper we show that the gain and phase frequency response characteristics of the human operator in a visual pursuit tracking task can be derived theoretically from these assumptions. By incorporating the effects of internal model inaccuracy and of speed-accuracy trade-off in performance, it is shown that various aspects of experimentally measured tracking behavior can be accounted for.

Changes in the perception of inspiratory resistive loads during partial curarization.

1. The ability of normal subjects to estimate the magnitude of resistive loads and to detect resistive loads was determined using standard psychophysical techniques. The experiments were repeated during maintained partial neuromuscular blockade with D-tubocurarine. 2. During curarization the ability to detect the presence of a small inspiratory resistive load was not significantly impaired. This finding is consistent with the view that changes in the usual breath-by-breath relation between pressure and flow mediate detection. 3. The size of resistive loads was over-estimated during partial curarization. Subjects thus placed more reliance on sensing the increased motor command required during curarization than on alternative peripheral signals related to pressure, flow or resistance. 4. The exponent for the power function relating the perceived magnitude of a resistive load to its actual size (Stevens, 1957) was reduced during partial neuromuscular blockade. 5. These results suggest that detection of resistive loads depends upon sensing apparent information arising from an unexpected pressure-flow disturbance but that estimation of the size of an added load depends, in part, on sensing the outgoing motor command.