Optogenetic Mimicry of the Transient Activation of Dopamine Neurons by Natural Reward Is Sufficient for Operant Reinforcement

Kyung Man Kim,Michael V Baratta,Doheon Lee,Aimei Yang,Edward S Boyden,Christopher D Fiorillo,Xiaoxi Zhuang

doi:10.1371/journal.pone.0033612

Abstract

Activation of dopamine receptors in forebrain regions, for minutes or longer, is known to be sufficient for positive reinforcement of stimuli and actions. However, the firing rate of dopamine neurons is increased for only about 200 milliseconds following natural reward events that are better than expected, a response which has been described as a “reward prediction error” (RPE). Although RPE drives reinforcement learning (RL) in computational models, it has not been possible to directly test whether the transient dopamine signal actually drives RL. Here we have performed optical stimulation of genetically targeted ventral tegmental area (VTA) dopamine neurons expressing Channelrhodopsin-2 (ChR2) in mice. We mimicked the transient activation of dopamine neurons that occurs in response to natural reward by applying a light pulse of 200 ms in VTA. When a single light pulse followed each self-initiated nose poke, it was sufficient in itself to cause operant reinforcement. Furthermore, when optical stimulation was delivered in separate sessions according to a predetermined pattern, it increased locomotion and contralateral rotations, behaviors that are known to result from activation of dopamine neurons. All three of the optically induced operant and locomotor behaviors were tightly correlated with the number of VTA dopamine neurons that expressed ChR2, providing additional evidence that the behavioral responses were caused by activation of dopamine neurons. These results provide strong evidence that the transient activation of dopamine neurons provides a functional reward signal that drives learning, in support of RL theories of dopamine function.

Highlights

There is extensive evidence, from pharmacological studies, that dopamine can cause positive reinforcement of stimuli and actions [1,2]
Channelrhodopsin-2 (ChR2) was introduced into the right ventral tegmental area (VTA) of dopamine transporter (DAT) IRES-Cre mutant mice [33] via injection of a Cre-dependent AAV-FLEX vector, which flanks the reversed ChR2 with sets of incompatible lox sites so that expression is localized to Cre-expressing cells [34]
Given the strong correlations that we observed between ChR2 expression and all three behavioral responses (Fig. 5), the present results provide compelling evidence that the brief, naturally occurring activation of dopamine neurons is sufficient for positive reinforcement of preceding actions, as hypothesized by reinforcement learning (RL) models of dopamine function [13,14]

Summary

Introduction

There is extensive evidence, from pharmacological studies, that dopamine can cause positive reinforcement of stimuli and actions [1,2]. A large body of evidence indicates that this transient response is aptly described as a ‘‘reward prediction error’’ (RPE), since firing rate increases in response to events that are better than predicted, decreases in response to events that are worse than predicted, and undergoes little change in response to events that meet expectations [3,4,5,6,7,8,9,10]. RPE drives reinforcement in computational models of RL [11,12], and it has been proposed that dopamine could serve an analogous reinforcing function in the brain [13,14]. The RL theory of dopamine function has been highly influential, and if correct, it would unify a large body of research. Compelling evidence in favor of the theory has been lacking, and the theory has been criticized on a variety of grounds

Objectives

Methods

Results

Conclusion