A Neural Correlate of Predicted and Actual Reward-Value Information in Monkey Pedunculopontine Tegmental and Dorsal Raphe Nucleus during Saccade Tasks

Ken-Ichi Okada,Kae Nakamura,Yasushi Kobayashi

doi:10.1155/2011/579840

Abstract

Dopamine, acetylcholine, and serotonin, the main modulators of the central nervous system, have been proposed to play important roles in the execution of movement, control of several forms of attentional behavior, and reinforcement learning. While the response pattern of midbrain dopaminergic neurons and its specific role in reinforcement learning have been revealed, the role of the other neuromodulators remains rather elusive. Here, we review our recent studies using extracellular recording from neurons in the pedunculopontine tegmental nucleus, where many cholinergic neurons exist, and the dorsal raphe nucleus, where many serotonergic neurons exist, while monkeys performed eye movement tasks to obtain different reward values. The firing patterns of these neurons are often tonic throughout the task period, while dopaminergic neurons exhibited a phasic activity pattern to the task event. The different modulation patterns, together with the activity of dopaminergic neurons, reveal dynamic information processing between these different neuromodulator systems.

Highlights

Reinforcement learning algorithms, originally proposed in the machine learning field, successfully explain various types of adaptive behavioral changes, including the simple classical and operant conditioning of animals [1,2,3,4,5,6] as well as the complex social and economic behavior of humans [7]
Numerous neurophysiological studies have shown that midbrain dopaminergic neurons, located in the substantia nigra pars compacta (SNc) and ventral tegmental area (VTA), encode the reward prediction error signal [1, 9,10,11,12]
Dopaminergic neurons exhibit phasic burst firing in response to external stimuli and rewards, and the response magnitude alters throughout the course of learning to match the reward prediction error signal [8]

Summary

Introduction

Reinforcement learning algorithms, originally proposed in the machine learning field, successfully explain various types of adaptive behavioral changes, including the simple classical and operant conditioning of animals [1,2,3,4,5,6] as well as the complex social and economic behavior of humans [7]. The noradrenergic locus coeruleus (LC) has widely distributed ascending projections to the neocortex [40] The neurons for these different neuromodulators are plausible candidates as the source of input to dopaminergic neurons and play an important role in the reinforcement process in parallel with dopaminergic neurons; their activity during motivated behavioral tasks remains rather elusive. We first summarize the growing literature on the PPTN/DRN in relation to the dopaminergic system (Section 2), we discuss our recent single-unit recording studies from the PPTN/DRN in behaving monkeys (Section 3), and assess the possible mechanisms for reward prediction error computation and its interaction with the motivational signal (Section 4). The reward prediction signals of PPTN/DRN neurons are intermingled with the signals for task motivation

Anatomy

20 Lt large

Findings

Circuit Mechanisms for the Computation of the Reward Prediction Error Signal