Robust and distributed neural representation of action values.

Eun Ju Shin,Yunsil Jang,Jung Hoon Sul,Yeonseung Chung,Sung-Hyun Lee,Daeyeol Lee,Hyunjung Lee,Xinying Cai,Hoseok Kim,Min Whan Jung,Soyoun Kim

doi:10.7554/elife.53045

Abstract

Studies in rats, monkeys, and humans have found action-value signals in multiple regions of the brain. These findings suggest that action-value signals encoded in these brain structures bias choices toward higher expected rewards. However, previous estimates of action-value signals might have been inflated by serial correlations in neural activity and also by activity related to other decision variables. Here, we applied several statistical tests based on permutation and surrogate data to analyze neural activity recorded from the striatum, frontal cortex, and hippocampus. The results show that previously identified action-value signals in these brain areas cannot be entirely accounted for by concurrent serial correlations in neural activity and action value. We also found that neural activity related to action value is intermixed with signals related to other decision variables. Our findings provide strong evidence for broadly distributed neural signals related to action value throughout the brain.

Highlights

The reinforcement learning theory provides a general theoretical framework for understanding the neural basis of value-based decision making (Corrado and Doya, 2007; Dayan and Niv, 2008; Glimcher, 2011; Lee et al, 2012a; Mars et al, 2012; O’Doherty et al, 2007)
Temporally discounted values (DVs) of alternative choices were randomized across trials, so that all decision variables were devoid of temporal correlation
Neural signals related to action value have been found in widespread regions of the brain, especially in the frontal cortex-basal ganglia loop (Chase et al, 2015; Ito and Doya, 2011; Lee, 2006; Lee et al, 2012a; Rushworth et al, 2009), suggesting the involvement of multiple brain structures in value-based decision making

Summary

Introduction

The reinforcement learning theory provides a general theoretical framework for understanding the neural basis of value-based decision making (Corrado and Doya, 2007; Dayan and Niv, 2008; Glimcher, 2011; Lee et al, 2012a; Mars et al, 2012; O’Doherty et al, 2007). In algorithms based on this theory, an agent selects an action based on a set of action values (i.e., values associated with potential actions) in a given state (Sutton and Barto, 1998). A large body of studies in rats, monkeys, and humans have found neural or hemodynamic signals correlated with action value in multiple regions of the brain, especially in the frontal cortex-basal ganglia loop

Methods

Results

Conclusion