Reward prediction error does not explain movement selectivity in DMS-projecting dopamine neurons.

Rachel S Lee,Nathan F Parker,Marcelo G Mattar,Ilana B Witten,Nathaniel D Daw

doi:10.7554/elife.42992

Abstract

Although midbrain dopamine (DA) neurons have been thought to primarily encode reward prediction error (RPE), recent studies have also found movement-related DAergic signals. For example, we recently reported that DA neurons in mice projecting to dorsomedial striatum are modulated by choices contralateral to the recording side. Here, we introduce, and ultimately reject, a candidate resolution for the puzzling RPE vs movement dichotomy, by showing how seemingly movement-related activity might be explained by an action-specific RPE. By considering both choice and RPE on a trial-by-trial basis, we find that DA signals are modulated by contralateral choice in a manner that is distinct from RPE, implying that choice encoding is better explained by movement direction. This fundamental separation between RPE and movement encoding may help shed light on the diversity of functions and dysfunctions of the DA system.

Highlights

A central feature of dopamine (DA) is its association with two apparently distinct functions: reward and movement (Niv et al 2007; Berke 2018)
We recently reported that whereas dopamine neurons projecting to ventral striatum showed classic reward prediction error” (RPE) signals, a subset of midbrain DA neurons that project to the dorsomedial striatum (DMS) were selective for a mouse’s choice of action (Parker et al 2016)
We fitted their choices using a standard trial-by-trial Q-learning model that predicted the probability of the animal's choice at each trial of the task (Figure 1c, Table 1). These choices were driven by a pair of decision variables putatively reflecting the animal’s valuation of each option. As mice performed this task, we recorded activity from either the terminals or cell bodies of DA neurons that project to DMS (VTA/SN::DMS) using fiber photometry to measure the fluorescence of the calcium indicator GCaMP6f (Figure 1d,e; Figure 1-Figure Supplement 1a,b)

Summary

Introduction

A central feature of dopamine (DA) is its association with two apparently distinct functions: reward and movement (Niv et al 2007; Berke 2018). Barto 1995; Cohen et al 2012; Coddington and Dudman 2018; Soares, Atallah, and Paton 2016; Hart et al 2014) In this classic framework, rather than explicitly encoding movement, DA neurons influence movements indirectly, by determining which movements are learned, and/or the general motivation to engage in a movement (Niv et al 2007; Collins and Frank 2014; Berke 2018). We recently reported that whereas dopamine neurons projecting to ventral striatum showed classic RPE signals, a subset of midbrain DA neurons that project to the dorsomedial striatum (DMS) were selective for a mouse’s choice of action (Parker et al 2016). Other studies have shown that DA neurons may have other forms of apparently non-RPE signals, such as signals related to novel or aversive stimuli (Menegas et al 2017; Horvitz 2000; Ungless, Magill, and Bolam 2004; Matsumoto and Hikosaka 2009; Lammel et al 2011)

Methods

Results

Conclusion