Cue-Evoked Dopamine Neuron Activity Helps Maintain but Does Not Encode Expected Value

Jesse A Mendoza,Christopher K Lafferty,Angela K Yang,Jonathan P Britt

doi:10.1016/j.celrep.2019.09.077

Jesse A Mendoza, Christopher K Lafferty + Show 2 more

Open Access

https://doi.org/10.1016/j.celrep.2019.09.077

Copy DOI

Journal: Cell Reports	Publication Date: Nov 1, 2019
Citations: 2	License type: cc-by

Affiliation: Concordia University, McGill University

Abstract

Cue-evoked midbrain dopamine (DA) neuron activity reflects expected value, but its influence on reward assessment is unclear. In mice performing a trial-based operant task, we test if bidirectional manipulations of cue or operant-associated DA neuron activity drive learning as a result of under- or overexpectation of reward value. We target optogenetic manipulations to different components of forced trials, when only one lever is presented, and assess lever biases on choice trials in the absence of photomanipulation. Although lever biases are demonstrated to be flexible and sensitive to changes in expected value, augmentation of cue or operant-associated DA signaling does not significantly alter choice behavior, and blunting DA signaling during any component of the forced trials reduces choice trial responses on the associated lever. These data suggest cue-evoked DA helps maintain cue-value associations but does not encode expected value as to set the benchmark against which received reward is judged.

Highlights

Reinforcement learning is driven by discrepancies between what is expected and what happens (Sutton and Barto, 1998)
Phasic DA Neuron Activity Occurs during Lever Presentations, Lever Pressing, and Reward Consumption To confirm that lever presentations during a trial-based operant task serve as a reward-predictive cue to elicit DA neuron activity in mice, we targeted GCaMP7s expression to ventral tegmental area (VTA) DA neurons and used fiber photometry to measure calcium-sensitive fluorescence (Figure 1; Reed et al, 2018)
When we reduced the value of the reward associated with a given lever, behavioral responding on that lever decreased relative to the non-devalued lever across forced and choice trials (Figures 2A and 2D)

Summary

Introduction

Reinforcement learning is driven by discrepancies between what is expected and what happens (Sutton and Barto, 1998) These reward prediction errors (RPEs) are used to update the expected value of preceding cues in order to improve the accuracy of future predictions. The presentation of reward-predictive cues elicits an RPE that is equivalent to the expected value of the upcoming reward. In light of this equivalence, it has been suggested that cue-evoked RPE signaling in the brain may simultaneously encode expected value, such that the outcome of any value prediction is judged in relation to this signal (Pultorak et al, 2018; Schelp et al, 2017). Augmentation of cue-evoked DA has been demonstrated to reduce reward seeking behavior, even though it transiently invigorates behavioral responding (Schelp et al, 2017)

Objectives

Results

Discussion

Conclusion