Learning from Sensory and Reward Prediction Errors during Motor Adaptation

Jun Izawa,Reza Shadmehr,Konrad Körding

doi:10.1371/journal.pcbi.1002012

Abstract

Voluntary motor commands produce two kinds of consequences. Initially, a sensory consequence is observed in terms of activity in our primary sensory organs (e.g., vision, proprioception). Subsequently, the brain evaluates the sensory feedback and produces a subjective measure of utility or usefulness of the motor commands (e.g., reward). As a result, comparisons between predicted and observed consequences of motor commands produce two forms of prediction error. How do these errors contribute to changes in motor commands? Here, we considered a reach adaptation protocol and found that when high quality sensory feedback was available, adaptation of motor commands was driven almost exclusively by sensory prediction errors. This form of learning had a distinct signature: as motor commands adapted, the subjects altered their predictions regarding sensory consequences of motor commands, and generalized this learning broadly to neighboring motor commands. In contrast, as the quality of the sensory feedback degraded, adaptation of motor commands became more dependent on reward prediction errors. Reward prediction errors produced comparable changes in the motor commands, but produced no change in the predicted sensory consequences of motor commands, and generalized only locally. Because we found that there was a within subject correlation between generalization patterns and sensory remapping, it is plausible that during adaptation an individual's relative reliance on sensory vs. reward prediction errors could be inferred. We suggest that while motor commands change because of sensory and reward prediction errors, only sensory prediction errors produce a change in the neural system that predicts sensory consequences of motor commands.

Highlights

Our motor commands generally produce two kinds of consequences: a sensory consequence in terms of activity in our primary sensory organs, and a rewarding consequence in terms of forming a subjective measure of utility or usefulness of these sensations
We focus on a simple motor adaptation task and consider a mathematical framework in which both reward and sensory prediction errors could contribute to the trial-to-trial change in the motor commands
It is thought that motor adaptation relies on sensory prediction errors to form an estimate of the perturbation

Summary

Introduction

Our motor commands generally produce two kinds of consequences: a sensory consequence in terms of activity in our primary sensory organs (e.g., vision, proprioception), and a rewarding consequence in terms of forming a subjective measure of utility or usefulness of these sensations (e.g., release of dopamine). While dancing, the motor commands that move our body produce proprioceptive feedback, while internal evaluation of that feedback indicates a pleasurable experience. These two consequences of the motor command form the basis for two kinds of prediction error: a sensory prediction error, and a reward prediction error. Motor adaptation studies often focus on learning from sensory prediction error [1,2,3,4,5,6,7], despite the fact that people are rewarded for each movement. Studies that focus on learning from reward prediction error (e.g., decision making tasks) often do not consider potential sensory prediction errors [8,9,10]. We attempt to ask whether learning from these two distinct signals can be behaviorally dissociated

Objectives

Methods

Discussion

Conclusion