Abstract
Compared to our understanding of positive prediction error signals occurring due to unexpected reward outcomes, less is known about the neural circuitry in humans that drives negative prediction errors during omission of expected rewards. While classical learning theories such as Rescorla–Wagner or temporal difference learning suggest that both types of prediction errors result from a simple subtraction, there has been recent evidence suggesting that different brain regions provide input to dopamine neurons which contributes to specific components of this prediction error computation. Here, we focus on the brain regions responding to negative prediction error signals, which has been well-established in animal studies to involve a distinct pathway through the lateral habenula. We examine the activity of this pathway in humans, using a conditioned inhibition paradigm with high-resolution functional MRI. First, participants learned to associate a sensory stimulus with reward delivery. Then, reward delivery was omitted whenever this stimulus was presented simultaneously with a different sensory stimulus, the conditioned inhibitor (CI). Both reward presentation and the reward-predictive cue activated midbrain dopamine regions, insula and orbitofrontal cortex. While we found significant activity at an uncorrected threshold for the CI in the habenula, consistent with our predictions, it did not survive correction for multiple comparisons and awaits further replication. Additionally, the pallidum and putamen regions of the basal ganglia showed modulations of activity for the inhibitor that did not survive the corrected threshold.
Highlights
While the field of reinforcement learning has generally focused on the role of reward prediction errors in training reward expectations, the mechanisms involved in learning about omission of expected reward delivery are less well understood
We examined if the same signals that have been reported for reward omissions in monkey studies, an increase in lateral habenula activity accompanied by a reduction in the firing of dopamine neurons, could be observed in human fMRI
We found that the ratings for the conditioned stimulus (CS)+ when presented concurrently with the Inhibitor were significantly lower than the ratings for the CS+ alone [t(18) = 7.07, p < 0.001], indicating that the conditioned inhibition procedure had significantly reduced reward expectations, demonstrating conditioned inhibition
Summary
While the field of reinforcement learning has generally focused on the role of reward prediction errors in training reward expectations, the mechanisms involved in learning about omission of expected reward delivery are less well understood. We focus on the latter computation, which has been well-established in animal studies, showing that neurons in the lateral habenula respond both to aversive outcomes and the omission of an expected reward, and further drive an inhibition of dopamine neurons, leading to the “dip” component of prediction error encoding how much worse something was than expected (Matsumoto and Hikosaka, 2009b). In appetitive Pavlovian conditioning, individuals learn expectations about stimuli that are reliably paired with rewards This conditioning procedure causes the previously neutral cue to drive a conditioned response. By learning theories like Rescorla–Wagner, if another sensory stimulus is reliably present during these unexpected omissions, the accumulation of negative prediction errors causes the CI to acquire negative value. This results in inhibitory conditioning, and a reduction of the conditioned response. The broken kettle becomes a CI because it reliably predicts the omission of tea
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call