Striatal activity consistent with model-based, rather than model-free prediction errors

Abstract

Event Abstract Back to Event Striatal activity consistent with model-based, rather than model-free prediction errors Shawn C. Green1*, Peng Zhang1, Nathaniel Daw2, Daniel Kersten1, Sheng He1 and Paul Schrater1 1 University of Minnesota, United States 2 New York University , United States Over the past several decades there has been considerable interest in uncovering the neural underpinnings of human decision-making. One particularly fruitful vein of research has focused on similarities between processing done in certain brain areas and computations that must be performed by model-free reinforcement learning algorithms. Of specific note is the repeatedly replicated correlation seen between activity in the ventral striatum and the size of trial-by-trial model-free reinforcement learning prediction errors. Striatal activity is high when the difference between the expected value of reward and the actual reward is large, while striatal activity is at a minimum when the amount of reward received is very close to the predicted value. Other brain areas such as the ventromedial prefrontal cortex, anterior cingulate, and amygdala have been similarly linked to model-free reinforcement learning parameters. In contrast, we have recently shown behaviorally that human choices are more consistent with a model-based, rather than a model-free learning algorithm. To demonstrate this, several versions of the standard sequential binary choice task were employed. Each of the versions contained contextual cues suggestive of a certain generative process for outcomes (i.e. Ð whether the outcomes were temporally independent, coupled, etc); importantly however, the actual generative process (and thus the observed outcome statistics) was identical in all versions. While a model-free system would be insensitive to such contextual cues (as such algorithms simply compute the expected values of states), human behavior was instead altered by these cues in a manner that was well predicted given the set of beliefs about the generative process promoted by the contextual cues. Our question is whether the activity in those brain regions associated with prediction error would be modulated by contextual manipulations. If these areas are truly calculating model-free values, they should be insensitive to irrelevant contextual cues as the actual observed outcome statistics are identical across conditions. If, on the other hand, activity depends on beliefs about the process generating those statistics, predictable differences should be observed. The data favored the latter hypothesis. In particular, when contextual cues were provided that were inconsistent with the true generative process, activity in the ventral striatum was correlated with model-free prediction error values. However, when contextual cues were provided that were consistent with the true generative process, the correlation between prediction errors and ventral striatal activity disappeared. We interpret these findings as being consistent with the notion that reward computations in the ventral striatum reflect the predictions of an internal generative model of the task. Conference: Computational and Systems Neuroscience 2010, Salt Lake City, UT, United States, 25 Feb - 2 Mar, 2010. Presentation Type: Poster Presentation Topic: Poster session I Citation: Green SC, Zhang P, Daw N, Kersten D, He S and Schrater P (2010). Striatal activity consistent with model-based, rather than model-free prediction errors. Front. Neurosci. Conference Abstract: Computational and Systems Neuroscience 2010. doi: 10.3389/conf.fnins.2010.03.00034 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 18 Feb 2010; Published Online: 18 Feb 2010. * Correspondence: Shawn C Green, University of Minnesota, Minneapolis, United States, csgreen@umn.edu Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Shawn C Green Peng Zhang Nathaniel Daw Daniel Kersten Sheng He Paul Schrater Google Shawn C Green Peng Zhang Nathaniel Daw Daniel Kersten Sheng He Paul Schrater Google Scholar Shawn C Green Peng Zhang Nathaniel Daw Daniel Kersten Sheng He Paul Schrater PubMed Shawn C Green Peng Zhang Nathaniel Daw Daniel Kersten Sheng He Paul Schrater Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.