Two spatiotemporally distinct value systems shape reward-based learning in the human brain.

Elsa Fouragnan,Marios G Philiastides,Chris Retzler,Karen Mullinger

doi:10.1038/ncomms9107

Abstract

Avoiding repeated mistakes and learning to reinforce rewarding decisions is critical for human survival and adaptive actions. Yet, the neural underpinnings of the value systems that encode different decision-outcomes remain elusive. Here coupling single-trial electroencephalography with simultaneously acquired functional magnetic resonance imaging, we uncover the spatiotemporal dynamics of two separate but interacting value systems encoding decision-outcomes. Consistent with a role in regulating alertness and switching behaviours, an early system is activated only by negative outcomes and engages arousal-related and motor-preparatory brain structures. Consistent with a role in reward-based learning, a later system differentially suppresses or activates regions of the human reward network in response to negative and positive outcomes, respectively. Following negative outcomes, the early system interacts and downregulates the late system, through a thalamic interaction with the ventral striatum. Critically, the strength of this coupling predicts participants' switching behaviour and avoidance learning, directly implicating the thalamostriatal pathway in reward-based learning.

Highlights

Avoiding repeated mistakes and learning to reinforce rewarding decisions is critical for human survival and adaptive actions
We designed the psychophysiological interaction analysis (PPI) analysis to identify brain areas in the Late system that increase their connectivity with the thalamus following negative outcomes. This connectivity analysis revealed a significant inverse coupling between the thalamus and the ventral STR cluster we found in the Late system, which corresponds to the nucleus accumbens (NAcc), a known projection site of the dopaminergic
Only the centromedial thalamus (CM-THAL) showed a significant inverse coupling with the Late system as discussed above. These findings suggest that following negative outcomes, the CM-THAL interacts both with structures controlling early autonomic responses, as well as those activated later to update value information, acting as a major hub between the Early and Late systems[37,38]

Summary

Introduction

Avoiding repeated mistakes and learning to reinforce rewarding decisions is critical for human survival and adaptive actions. Coupling single-trial electroencephalography with simultaneously acquired functional magnetic resonance imaging, we uncover the spatiotemporal dynamics of two separate but interacting value systems encoding decision-outcomes. The reinforcement sensitivity theory (RST) introduced by Jeffrey Gray[1] in the 1970’s was the first to describe two distinct decision-outcome value systems that trigger avoidance behaviour and orchestrate learning. In line with RST, recent human electroencephalography (EEG) data revealed two temporally distinct processing stages of outcome value; an early valence-sensitive process thought to be driven by an automatic alertness response to negative outcomes[2,3,4] and a later, more deliberate, assessment of the value information required for learning and updating reward expectations[3]. Our hypothesis is that endogenous trial-to-trial variability in the two temporally distinct EEG components can be used to form separate BOLD predictors (rather than using a categorical predictor representing outcome valence) to tease apart the cortical and subcortical networks associated with each system

Methods

Results

Conclusion