Cerebellar damage limits reinforcement learning.

Abstract

This scientific commentary refers to ‘Effective reinforcement learning following cerebellar damage requires a balance between exploration and motor noise’, by Therrien et al. (doi:10.1093/brain/awv329) . An exciting challenge for research in motor learning is to disentangle the multiple processes involved, and to tie these down to distinct neural systems. About 17 years ago, Doya proposed that the cerebellum, basal ganglia and cerebral cortex were separately responsible for supervised learning, reinforcement learning, and unsupervised learning, respectively (Doya, 1999, 2000). Supervised learning is driven, unsurprisingly, by signals provided by a ‘supervisor’ and is typically equated with error-based learning: after an action, an error in performance is processed, and subsequent actions are adjusted to try to minimize the error. Reinforcement learning is driven by rewards and punishments: exploratory actions are tried out and each action’s outcome is evaluated; learning aims to maximize the value of future action choices. Unsupervised learning occurs in the face of repeated experience of the environment, and generates a mapping of its statistical regularities: it can be driven by Hebbian learning so that, for example, similar sensory events become associated with one another. In the motor domain this can equate to yet another form of learning, use-dependent learning, where there is a bias to produce actions more similar to previous ones. For many years, these learning processes were thought of as functionally and anatomically independent. However, huge efforts are now being made to understand how these various processes interact. In this issue of Brain , Therrien, Wolpert and Bastian have added to these efforts by testing and modelling how patients with cerebellar ataxia differ from healthy controls in performing error- and reinforcement-learning tasks (Therrien et al. , 2015). Using a mechanistic model, they show that optimal learning with reinforcement feedback requires subjects to balance the variability …