Dissociable neural correlates of uncertainty underlie different exploration strategies

Samuel J Gershman,Momchil S Tomov,Van Q Truong,Rohan A Hundia

doi:10.1038/s41467-020-15766-z

Samuel J Gershman, Momchil S Tomov + Show 2 more

Open Access

https://doi.org/10.1038/s41467-020-15766-z

Copy DOI

Abstract

Most real-world decisions involve a delicate balance between exploring unfamiliar alternatives and committing to the best known option. Previous work has shown that humans rely on different forms of uncertainty to negotiate this "explore-exploit” trade-off, yet the neural basis of the underlying computations remains unclear. Using fMRI (n = 31), we find that relative uncertainty is represented in right rostrolateral prefrontal cortex and drives directed exploration, while total uncertainty is represented in right dorsolateral prefrontal cortex and drives random exploration. The decision value signal combining relative and total uncertainty to compute choice is reflected in motor cortex activity. The variance of this signal scales with total uncertainty, consistent with a sampling mechanism for random exploration. Overall, these results are consistent with a hybrid computational architecture in which different uncertainty computations are performed separately and then combined by downstream decision circuits to compute choice.

Highlights

Most real-world decisions involve a delicate balance between exploring unfamiliar alternatives and committing to the best known option
We find that relative uncertainty is reflected in right rostrolateral prefrontal cortex (RLPFC), and total uncertainty is reflected in right dorsolateral prefrontal cortex (DLPFC), replicating findings reported by Badre et al.[26]
We find that the linear combination of relative and total uncertainty with value is reflected in motor cortex, suggesting that these upstream estimates are integrated by motor circuits in order to compute the categorical decision

Summary

Introduction

Most real-world decisions involve a delicate balance between exploring unfamiliar alternatives and committing to the best known option. While earlier studies favored value-based random exploration strategies such as softmax exploration, later work[8,14] has shown that random exploration in people is sensitive to the total uncertainty of the available options, increasing choice stochasticity when option values are more uncertain. This can cause choice variability to track payoff variability, a phenomenon sometimes referred to as the payoff variability effect[18,19,20]

Objectives

Results

Conclusion