Dopamine Enhances Model-Free Credit Assignment Through Boosting of Retrospective Model-Based Inference

Lorenz Deserno,Rani Moran,Ying Lee,Jochen Michely,Peter Dayan,Raymond Dolan

doi:10.1016/j.biopsych.2021.02.246

Abstract

Dopamine is implicated in signalling model-free (MF) reward prediction errors and various aspects of model-based (MB) planning and choice. Recently, we showed that cooperative interactions between MB and MF systems include guidance of MF credit assignment by MB inference. Here, we test a novel hypothesis that enhancing dopamine levels boosts the guidance of MF credit assignment via MB inference.

Highlights

Dual system theories of reinforcement learning (RL) propose that behaviour is controlled by a prospective, model-b ased (MB), planning system and a retrospective, model-free (MF), value-c aching system (Daw and Dayan, 2014; Daw et al, 2005; Dolan and Dayan, 2013)
Levodopa enhances guidance of preferential MFCA (PMFCA) by retrospective MB inference on uncertainty trials To test our key hypothesis, that guidance of PMFCA by retrospective MB inference on uncertainty trials is enhanced by levodopa, we focused on the four computational parameters that pertaining to MFCA on uncertainty trials (‍cMnoFm,info, cMrejF,info, cMnoFm,noninfo, cMrejF,noninfo‍, Figure 5B and C)
We show that enhancing DA boosted the guidance of MFCA by retrospective MB inference

Summary

Introduction

Dual system theories of reinforcement learning (RL) propose that behaviour is controlled by a prospective, model-b ased (MB), planning system and a retrospective, model-free (MF), value-c aching system (Daw and Dayan, 2014; Daw et al, 2005; Dolan and Dayan, 2013). Studies, motivated by classical psychological dual control distinctions regarding goal and habit (Balleine and Dickinson, 1998; Dickinson, 1997; Dickinson, 1985 Dolan and Dayan, 2013), focused on how these systems operated separately, with subsequent work indicating their influences being combined only close to the point of choice (Daw et al, 2011) Even in the former and other experiments (involving the well-d escribed two-s tep task), human functional neuroimaging showed that activity in the ventral striatum and the prefrontal cortex reflected MB values as well as MF RPEs (Daw et al, 2011; for replication, see Deserno et al, 2015b; Deserno et al, 2015a). More recent evidence has raised the possibility that this may occur during rest periods (Antonov et al, 2021; Liu et al, 2021a)

Objectives

Methods

Results

Conclusion