Internal Forward Model Research Articles

Cerebellum as a forward but not inverse model in visuomotor adaptation task: a tDCS-based and modeling study.

Despite several pieces of evidence, which suggest that the human brain employs internal models for motor control and learning, the location of these models in the brain is not yet clear. In this study, we used transcranial direct current stimulation (tDCS) to manipulate right cerebellar function, while subjects adapt to a visuomotor task. We investigated the effect of this manipulation on the internal forward and inverse models by measuring two kinds of behavior: generalization of training in one direction to neighboring directions (as a proxy for inverse models) and localization of the hand position after movement without visual feedback (as a proxy for forward model). The experimental results showed no effect of cerebellar tDCS on generalization, but significant effect on localization. These observations support the idea that the cerebellum is a possible brain region for internal forward, but not inverse model formation. We also used a realistic human head model to calculate current density distribution in the brain. The result of this model confirmed the passage of current through the cerebellum. Moreover, to further explain some observed experimental results, we modeled the visuomotor adaptation process with the help of a biologically inspired method known as population coding. The effect of tDCS was also incorporated in the model. The results of this modeling study closely match our experimental data and provide further evidence in line with the idea that tDCS manipulates FM's function in the cerebellum.

The role of internal forward models and proprioception in hand position estimation.

Our ability to properly move and react in different situations is largely dependent on our perception of our limbs' position. At least three sources- vision, proprioception, and internal forward models (FMs)- seem to contribute to this perception. To the best of our knowledge, the effect of each source has not been studied individually. Specifically, role of FM has been ignored in some previous studies. We hypothesized that FM has a critical role in subjects' perception which needs to be considered in the relevant studies to obtain more reliable results. Therefore, we designed an experiment with the goal of investigating FM and proprioception role in subjects' perception of their hand's position. Three groups of subjects were recruited in the study. Based on the experiment design, it was supposed that subjects in different groups relied on proprioception, FM, and both of them for estimating their unseen hand's position. Comparing the results of three groups revealed significant difference between their estimation' errors. FM provided minimum estimation error, while proprioception had a bias error in the tested region. Integrating proprioception with FM decreased this error. Integration of two Gaussian functions, fitted to the error distribution of FM and proprioception groups, was simulated and created a mean error value almost similar to the experimental observation. These results suggest that FM role needs to be considered when studying the perceived position of the limbs. This can lead to gain better insights into the mechanisms underlying the perception of our limbs' position which might have potential clinical and rehabilitation applications, e.g., in the postural control of elderly which are at high risk of falls and injury because of deterioration of their perception with age.

The intentional stance as structure learning: a computational perspective on mindreading.

Recent theories of mindreading explain the recognition of action, intention, and belief of other agents in terms of generative architectures that model the causal relations between observables (e.g., observed movements) and their hidden causes (e.g., action goals and beliefs). Two kinds of probabilistic generative schemes have been proposed in cognitive science and robotics that link to a "theory theory" and "simulation theory" of mindreading, respectively. The former compares perceived actions to optimal plans derived from rationality principles and conceptual theories of others' minds. The latter reuses one's own internal (inverse and forward) models for action execution to perform a look-ahead mental simulation of perceived actions. Both theories, however, leave one question unanswered: how are the generative models - including task structure and parameters - learned in the first place? We start from Dennett's "intentional stance" proposal and characterize it within generative theories of action and intention recognition. We propose that humans use an intentional stance as a learning bias that sidesteps the (hard) structure learning problem and bootstraps the acquisition of generative models for others' actions. The intentional stance corresponds to a candidate structure in the generative scheme, which encodes a simplified belief-desire folk psychology and a hierarchical intention-to-action organization of behavior. This simple structure can be used as a proxy for the "true" generative structure of others' actions and intentions and is continuously grown and refined - via state and parameter learning - during interactions. In turn - as our computational simulations show - this can help solve mindreading problems and bootstrap the acquisition of useful causal models of both one's own and others' goal-directed actions.

The Errors of Our Ways: Understanding Error Representations in Cerebellar-Dependent Motor Learning

The cerebellum is essential for error-driven motor learning and is strongly implicated in detecting and correcting for motor errors. Therefore, elucidating how motor errors are represented in the cerebellum is essential in understanding cerebellar function, in general, and its role in motor learning, in particular. This review examines how motor errors are encoded in the cerebellar cortex in the context of a forward internal model that generates predictions about the upcoming movement and drives learning and adaptation. In this framework, sensory prediction errors, defined as the discrepancy between the predicted consequences of motor commands and the sensory feedback, are crucial for both on-line movement control and motor learning. While many studies support the dominant view that motor errors are encoded in the complex spike discharge of Purkinje cells, others have failed to relate complex spike activity with errors. Given these limitations, we review recent findings in the monkey showing that complex spike modulation is not necessarily required for motor learning or for simple spike adaptation. Also, new results demonstrate that the simple spike discharge provides continuous error signals that both lead and lag the actual movements in time, suggesting errors are encoded as both an internal prediction of motor commands and the actual sensory feedback. These dual error representations have opposing effects on simple spike discharge, consistent with the signals needed to generate sensory prediction errors used to update a forward internal model.

Action sounds update the mental representation of arm dimension: contributions of kinaesthesia and agency.

Auditory feedback accompanies almost all our actions, but its contribution to body-representation is understudied. Recently it has been shown that the auditory distance of action sounds recalibrates perceived tactile distances on one’s arm, suggesting that action sounds can change the mental representation of arm length. However, the question remains open of what factors play a role in this recalibration. In this study we investigate two of these factors, kinaesthesia, and sense of agency. Across two experiments, we asked participants to tap with their arm on a surface while extending their arm. We manipulated the tapping sounds to originate at double the distance to the tapping locations, as well as their synchrony to the action, which is known to affect feelings of agency over the sounds. Kinaesthetic cues were manipulated by having additional conditions in which participants did not displace their arm but kept tapping either close (Experiment 1) or far (Experiment 2) from their body torso. Results show that both the feelings of agency over the action sounds and kinaesthetic cues signaling arm displacement when displacement of the sound source occurs are necessary to observe changes in perceived tactile distance on the arm. In particular, these cues resulted in the perceived tactile distances on the arm being felt smaller, as compared to distances on a reference location. Moreover, our results provide the first evidence of consciously perceived changes in arm-representation evoked by action sounds and suggest that the observed changes in perceived tactile distance relate to experienced arm elongation. We discuss the observed effects in the context of forward internal models of sensorimotor integration. Our results add to these models by showing that predictions related to action sounds must fit with kinaesthetic cues in order for auditory inputs to change body-representation.

A cerebellar role in performance monitoring – Evidence from EEG and voxel-based morphometry in patients with cerebellar degenerative disease

The cerebellum applies an internal forward-model to predict the sensory consequences of actions. This forward-model is updated based on on-line performance monitoring. A previous study has shown that performance monitoring is altered in patients with focal vascular cerebellar lesions, but altered neural responses are not paralleled by impaired behaviour, and the critical cerebellar sites have yet to be identified. The present study investigated if saccadic performance monitoring is more severely altered in patients with cerebellar degenerative disease relative to the previously examined patients with focal vascular cerebellar lesions, and which cerebellar regions support performance monitoring. 16 patients and 16 healthy controls performed an antisaccade task while an electroencephalogram (EEG) was recorded. Error rates were increased, and the error-related negativity (ERN), an event-related potential (ERP) component associated with error processing/performance monitoring, was reduced while the error positivity (Pe), a later ERP component related to more conscious aspects of error processing, was preserved in patients. Thus, performance monitoring is altered in patients with cerebellar degeneration, confirming a critical role of the cerebellum for fast classification of saccadic accuracy. In contrast to patients with focal lesions, post-acute functional reorganization and compensation presumably is hampered by disease progression, resulting in altered neural processing and impaired behavioural performance. Voxel-based morphometry (VBM) indicated the strongest effects for behavioural performance, with correlations between gray matter volume reduction in bilateral posterolateral regions (left Crus II and right lobule VI) and increased error rates. Moreover, somewhat smaller correlations were found for volume loss in left lobule VIIb/VIIIa and right lobule V and ERN amplitude, and in right Crus I and Pe amplitude. The present findings are consistent with involvement of posterolateral cerebellar regions in motor and cognitive functions.

Formation of internal forward model with sensory and reward prediction errors : A behavioral confirmation

Formation of internal forward model with sensory and reward prediction errors : A behavioral confirmation

Active Sampling supported Comparison of Causal Inference Models for Agency Attribution in Goal-Directed Actions

Perception of own actions is influenced by visual information and predictions from internal forward models [1]. Integrating these information sources depends critically on whether visual consequences are associated with one's own action (sense of agency) or with changes in the external world unrelated to the action [2] and the accuracy of integrated signals [3]. Attribution of percepts to consequences of own actions depends thus on the consistency between internally predicted and actual visual signals. However, is the attribution of agency rather a binary decision ('I did, or did not cause the visual consequences of the action' [4]), or is this process based on a more gradual attribution of the degree of agency? Both alternatives result in different behaviors of causal inference models, which we try to distinguish by model comparison. METHODS. We used a virtual-reality setup to manipulate the consistency between pointing movements and their visual consequences. We investigated the influence of this manipulation on self-action perception. We compared two Bayesian causal inference models to the experimental data, one with a binary latent agency variable [2], and one with a continuous latent agency variable [4]. Here, subject-specific regions for stimulus conditions that maximally differentiate between the two models were identified online using Active Sampling methods [6] to evaluate relative model evidences with a small number of samples. RESULTS/CONCLUSION. Both models correctly predict the data, and specifically empirical agency ratings showing high attribution of agency for small deviations between sensory and predicted feedback. Some participants show signatures of a binary internal representation of agency. In addition, relationships with other inference models [5] are discussed. [1] Wolpert et al.,Science,269,1995. [2] Körding et al.,PLoS ONE,2(9),2007. Shams&Beierholm,TiCS,14,2010. [3] Burge et al.,JVis,8(4),2008. [4] Beck et al.,Jvis,13(9),2013. [5] Marko et al.,JNPhys,108,2012. Ernst,Jvis,7(5),2007. [6] MacKay,NeuralComp,4(4),1992. Paninski,NeuralComp,17(7),2005. Meeting abstract presented at VSS 2014

Is Action-Perception Coupling Improved with Delay in Patients with Focal Cerebellar Lesions?

It has been proposed that the cerebellum uses internal forward models to predict sensory consequences of particular actions ([Wolpert and Flanagan, 2001][1]). This allows movements to be corrected when necessary. In addition to its paramount role in the control of movements, the cerebellum is also

Making and monitoring errors based on altered auditory feedback.

Previous research has demonstrated that altered auditory feedback (AAF) disrupts music performance and causes disruptions in both action planning and the perception of feedback events. It has been proposed that this disruption occurs because of interference within a shared representation for perception and action (Pfordresher, 2006). Studies reported here address this claim from the standpoint of error monitoring. In Experiment 1 participants performed short melodies on a keyboard while hearing no auditory feedback, normal auditory feedback, or alterations to feedback pitch on some subset of events. Participants overestimated error frequency when AAF was present but not for normal feedback. Experiment 2 introduced a concurrent load task to determine whether error monitoring requires executive resources. Although the concurrent task enhanced the effect of AAF, it did not alter participants’ tendency to overestimate errors when AAF was present. A third correlational study addressed whether effects of AAF are reduced for a subset of the population who may lack the kind of perception/action associations that lead to AAF disruption: poor-pitch singers. Effects of manipulations similar to those presented in Experiments 1 and 2 were reduced for these individuals. We propose that these results are consistent with the notion that AAF interference is based on associations between perception and action within a forward internal model of auditory-motor relationships.

Motor Intention Determines Sensory Attenuation of Brain Responses to Self-initiated Sounds

One of the functions of the brain is to predict sensory consequences of our own actions. In auditory processing, self-initiated sounds evoke a smaller brain response than passive sound exposure of the same sound sequence. Previous work suggests that this response attenuation reflects a predictive mechanism to differentiate the sensory consequences of one's own actions from other sensory input, which seems to form the basis for the sense of agency (recognizing oneself as the agent of the movement). This study addresses the question whether attenuation of brain responses to self-initiated sounds can be explained by brain activity involved in movement planning rather than movement execution. We recorded ERPs in response to sounds initiated by button presses. In one condition, participants moved a finger to press the button voluntarily, whereas in another condition, we initiated a similar, but involuntary, finger movement by stimulating the corresponding region of the primary motor cortex with TMS. For involuntary movements, no movement intention (and no feeling of agency) could be formed; thus, no motor plans were available to the forward model. A portion of the brain response evoked by the sounds, the N1-P2 complex, was reduced in amplitude following voluntary, self-initiated movements, but not following movements initiated by motor cortex stimulation. Our findings demonstrate that movement intention and the corresponding feeling of agency determine sensory attenuation of brain responses to self-initiated sounds. The present results support the assumptions of a predictive internal forward model account operating before primary motor cortex activation.

Visually guided gait modifications for stepping over an obstacle: a bio-inspired approach

There is an increasing interest in conceiving robotic systems that are able to move and act in an unstructured and not predefined environment, for which autonomy and adaptability are crucial features. In nature, animals are autonomous biological systems, which often serve as bio-inspiration models, not only for their physical and mechanical properties, but also their control structures that enable adaptability and autonomy-for which learning is (at least) partially responsible. This work proposes a system which seeks to enable a quadruped robot to online learn to detect and to avoid stumbling on an obstacle in its path. The detection relies in a forward internal model that estimates the robot's perceptive information by exploring the locomotion repetitive nature. The system adapts the locomotion in order to place the robot optimally before attempting to step over the obstacle, avoiding any stumbling. Locomotion adaptation is achieved by changing control parameters of a central pattern generator (CPG)-based locomotion controller. The mechanism learns the necessary alterations to the stride length in order to adapt the locomotion by changing the required CPG parameter. Both learning tasks occur online and together define a sensorimotor map, which enables the robot to learn to step over the obstacle in its path. Simulation results show the feasibility of the proposed approach.

Sensorial suppression of self-generated sounds and its dependence on attention

The auditory processing of self-generated sounds is characterized by an attenuated vertex N1-component of the event-related potential (ERP) compared to the responses elicited by externally generated sounds. Typically, a motor condition where sounds are actively produced by button presses is compared with a passive listening condition. While this effect is usually interpreted as reflection of an internal forward model system, the impact of attention and arousal on the so called self-generation effect has not been systematically controlled in these studies: Is the auditory stimulation more attended during the active task compared to passive listening, e.g., caused by a higher arousal level? Or is it rather attended less and attention is drawn away from the task-irrelevant stimulation to the motor task? Accordingly, the self-generation effects reported in the literature can easily be over- or underestimated. In the present study we disentangled attention from the self-generation effect by introducing an active listening condition, in which attention is focused to the same feature as in the self-generation condition — the stimulus onset-to-onset interval. We observed a classical ‘self-generation effect’, i.e. attenuated amplitudes for self-generated compared to passive listened sounds at frontocentral electrodes. As expected this effect was overlapped by attention effects in space and time. However, topographical and tomographical analyses allowed us to clearly disentangle both effects. Our results argue for the existence of a genuine self-generation effect, but emphasize the problem of possible over- or underestimation caused by attentional confounds.

Are fast/slow process in motor adaptation and forward/inverse internal model two sides of the same coin?

Motor adaptation is tuning of motor commands to compensate the disturbances in the outside environment and/or in the sensory-motor system. In spite of various theoretical and empirical studies, mechanism by which the brain learns to adapt has not been clearly understood. Among different computational models, two lines of researches are of interest in this study: first, the models which assume two adaptive processes, i.e. fast and slow, for motor learning, and second, the computational frameworks which assume two types of internal models in the central nervous system (CNS), i.e., forward and inverse models. They explain motor learning by modifying these internal models. Here, we present a hypothesis for a possible relationship between these two viewpoints according to the computational and physiological findings. This hypothesis suggests a direct relationship between the forward (inverse) internal model and the fast (slow) adaptive process. That is, the forward (inverse) model and fast (slow) adaptive process can be two sides of the same coin. Further evaluation of this hypothesis is helpful to achieve a better understanding of motor adaptation mechanism in the brain and also it lends itself to be used in designing therapeutic programs for rehabilitation of certain movement disorders.

Me - Not Me - Or In Between? Comparison of Causal Inference Models for Agency attribution in goal-directed actions

Perception of own actions is influenced by visual information and predictions from internal forward models [1]. Integrating these information sources depends critically on whether visual consequences are associated with one's own action (sense of agency) or with changes in the external world unrelated to the action [2,3] and the accuracy of integrated signals [4,5]. Attribution of percepts to consequences of own actions should thus depend on the consistency between internally predicted and actual visual signals, but what does the data support: binary (me vs not me)[6] or continuous (partially me) attribution? METHODS. To examine this question, we used a virtual-reality setup to manipulate the consistency between pointing movements and their visual consequences and investigated the influence of this manipulation on self-action perception with visual stimuli of varying precision. In previous work we showed that a Bayesian causal inference model, assuming a binary latent agency variable controlling the attributed influence of the self-action on perceptual consequences [2,3], accounted for the empirical data [6]. Here, new models assuming a continuous variable for attribution of the visual feedback to own action are presented and their performance predicting the empirical data evaluated and compared to the binary model [2,3]. The models assume both visual feedback and internal estimate are directly caused by the (unobserved) real motor state. RESULTS AND CONCLUSION. The models correctly predict empirical agency ratings, showing attribution of visual signals to self-action for small, and stronger reliance on internal information for large deviations. We discuss the performance of these causal inference models, applying methods for model comparison. [1] Wolpert et al., Science, 269, 1995. [2] Körding et al., PLoS ONE, 2(9), 2007. [3] Shams & Beierholm, TiCS, 14, 2010. [4] Alais & Burr, CurBio, 14, 2004. [5] Burge et al., JVis, 8(4), 2008. [6] Beck et al., JVis, 11(11): 955, 2011. Meeting abstract presented at VSS 2013

Silent articulation modulates auditory and audiovisual speech perception

The concept of an internal forward model that internally simulates the sensory consequences of an action is a central idea in speech motor control. Consistent with this hypothesis, silent articulation has been shown to modulate activity of the auditory cortex and to improve the auditory identification of concordant speech sounds, when embedded in white noise. In the present study, we replicated and extended this behavioral finding by showing that silently articulating a syllable in synchrony with the presentation of a concordant auditory and/or visually ambiguous speech stimulus improves its identification. Our results further demonstrate that, even in the case of perfect perceptual identification, concurrent mouthing of a syllable speeds up the perceptual processing of a concordant speech stimulus. These results reflect multisensory-motor interactions during speech perception and provide new behavioral arguments for internally generated sensory predictions during silent speech production.

Purkinje Cell Simple Spike Discharge Encodes Error Signals Consistent with a Forward Internal Model

Processing motor errors is essential for online control of goal-directed movements and motor learning. Evidence from psychophysical and imaging studies supports the long-standing view that error processing is central to cerebellar function. The dominant view is that error-related signals are encoded in the complex spike discharge of Purkinje cells. However, the findings are inconsistent on whether complex spike activity correlates with motor errors. Recently, we examined if simple spike firing carries error signals in monkeys trained to manually track a randomly moving target. The task requires continuous processing of motor errors characterized by the relative movements between the hand-driven cursor and the target center. Linear regression models show that error parameters are robustly represented in the simple spike activity of most Purkinje cells. At the single cell level, the error signals are encoded independently and integrated with kinematic signals. In a large majority of Purkinje cells, correlation strengths between the simple spike discharge and an error parameter have bimodal profiles with respect to time, exhibiting a local maxima corresponding to firing leading the behavior and another one corresponding to firing lagging behavior. The bimodal temporal profiles suggest that individual error parameters are dually encoded as both an internal prediction used for feedback-independent, compensatory movements and the actual sensory feedback used to monitor performance. Approximately 75% of the dual representations have opposing modulations of the simple spike activity, one increasing firing and the other depressing firing, as reflected by the reversed signs of the regression coefficients corresponding to the local maxima of the R (2) profile. These dual representations of individual parameters with opposing modulation of the simple spike firing are consistent with the signals needed to generate sensory prediction errors used to update an internal model.

Neural control and adaptive neural forward models for insect-like, energy-efficient, and adaptable locomotion of walking machines.

Living creatures, like walking animals, have found fascinating solutions for the problem of locomotion control. Their movements show the impression of elegance including versatile, energy-efficient, and adaptable locomotion. During the last few decades, roboticists have tried to imitate such natural properties with artificial legged locomotion systems by using different approaches including machine learning algorithms, classical engineering control techniques, and biologically-inspired control mechanisms. However, their levels of performance are still far from the natural ones. By contrast, animal locomotion mechanisms seem to largely depend not only on central mechanisms (central pattern generators, CPGs) and sensory feedback (afferent-based control) but also on internal forward models (efference copies). They are used to a different degree in different animals. Generally, CPGs organize basic rhythmic motions which are shaped by sensory feedback while internal models are used for sensory prediction and state estimations. According to this concept, we present here adaptive neural locomotion control consisting of a CPG mechanism with neuromodulation and local leg control mechanisms based on sensory feedback and adaptive neural forward models with efference copies. This neural closed-loop controller enables a walking machine to perform a multitude of different walking patterns including insect-like leg movements and gaits as well as energy-efficient locomotion. In addition, the forward models allow the machine to autonomously adapt its locomotion to deal with a change of terrain, losing of ground contact during stance phase, stepping on or hitting an obstacle during swing phase, leg damage, and even to promote cockroach-like climbing behavior. Thus, the results presented here show that the employed embodied neural closed-loop system can be a powerful way for developing robust and adaptable machines.

A New Proposal on How Motor Memory Is Consolidated

A New Proposal on How Motor Memory Is Consolidated

Cerebellar contribution to the prediction of self-initiated sounds

In everyday life we frequently make the fundamental distinction between sensory input resulting from our own actions and sensory input that is externally-produced. It has been speculated that making this distinction involves the use of an internal forward-model, which enables the brain to adjust its response to self-produced sensory input. In the auditory domain, this idea has been supported by event-related potential and evoked-magnetic field studies revealing that self-initiated sounds elicit a suppressed N100/M100 brain response compared to externally-produced sounds. Moreover, a recent study reveals that patients with cerebellar lesions do not show a significant N100-suppression effect. This result supports the theory that the cerebellum is essential for generating internal forward predictions. However, all except one study compared self-initiated and externally-produced auditory stimuli in separate conditions. Such a setup prevents an unambiguous interpretation of the N100-suppression effect when distinguishing self- and externally-produced sensory stimuli: the N100-suppression can also be explained by differences in the allocation of attention in different conditions. In the current electroencephalography (EEG)-study we investigated the N100-suppression effect in an altered design comparing (i) self-initiated sounds to externally-produced sounds that occurred intermixed with these self-initiated sounds (i.e., both sound types occurred in the same condition) or (ii) self-initiated sounds to externally-produced sounds that occurred in separate conditions. Results reveal that the cerebellum generates selective predictions in response to self-initiated sounds independent of condition type: cerebellar patients, in contrast to healthy controls, do not display an N100-suppression effect in response to self-initiated sounds when intermixed with externally-produced sounds. Furthermore, the effect is not influenced by the temporal proximity of externally-produced sounds to self-produced sounds. Controls and patients showed a P200-reduction in response to self-initiated sounds. This suggests the existence of an additional and probably more conscious mechanism for identifying self-generated sounds that does not functionally depend on the cerebellum.