Learning in Visual Regions as Support for the Bias in Future Value-Driven Choice.

Jan Theeuwes,Tomas Knapen,Sara Jahfari

doi:10.1093/cercor/bhz218

Abstract

Reinforcement learning can bias decision-making toward the option with the highest expected outcome. Cognitive learning theories associate this bias with the constant tracking of stimulus values and the evaluation of choice outcomes in the striatum and prefrontal cortex. Decisions however first require processing of sensory input, and to date, we know far less about the interplay between learning and perception. This functional magnetic resonance imaging study (N = 43) relates visual blood oxygen level–dependent (BOLD) responses to value beliefs during choice and signed prediction errors after outcomes. To understand these relationships, which co-occurred in the striatum, we sought relevance by evaluating the prediction of future value-based decisions in a separate transfer phase where learning was already established. We decoded choice outcomes with a 70% accuracy with a supervised machine learning algorithm that was given trial-by-trial BOLD from visual regions alongside more traditional motor, prefrontal, and striatal regions. Importantly, this decoding of future value-driven choice outcomes again highlighted an important role for visual activity. These results raise the intriguing possibility that the tracking of value in visual cortex is supportive for the striatal bias toward the more valued option in future choice.

Highlights

In decision-making, our value beliefs bias future choices
By focusing on how participants learn, we find blood oxygen level–dependent (BOLD) in visual regions to change with trial-by-trial adaptations in value beliefs about the faces presented and to be subsequently scaled by the signed reward prediction errors (RPEs) after feedback
The relevance of these observed value and feedback modulations was sought by exploring the prediction of future value-driven choice outcomes in a followup transfer phase where feedback was omitted

Summary

Introduction

In decision-making, our value beliefs bias future choices. This bias is shaped by the outcomes of similar decisions made in the past where the action, or stimulus chosen, becomes associated with a positive or negative outcome (“value beliefs”). To underset the bias in action selection, midbrain dopamine neurons are thought to send a teaching signal toward the striatum and prefrontal cortex after an outcome (Montague et al 1996; Schultz et al 1997; Tobler et al 2005). The dorsal and ventral parts of the striatum are known to receive differential, and overlapping, inputs from midbrain neurons (O’Doherty et al 2004; Atallah et al 2007). Ventral and dorsal striatum have been ascribed a differential role during learning by reinforcement learning

Methods

Results

Conclusion