Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis

Quentin Jm Huys,Ryan Bogdan,Diego A Pizzagalli,Peter Dayan

doi:10.1186/2045-5380-3-12

Abstract

BackgroundDepression is characterised partly by blunted reactions to reward. However, tasks probing this deficiency have not distinguished insensitivity to reward from insensitivity to the prediction errors for reward that determine learning and are putatively reported by the phasic activity of dopamine neurons. We attempted to disentangle these factors with respect to anhedonia in the context of stress, Major Depressive Disorder (MDD), Bipolar Disorder (BPD) and a dopaminergic challenge.MethodsSix behavioural datasets involving 392 experimental sessions were subjected to a model-based, Bayesian meta-analysis. Participants across all six studies performed a probabilistic reward task that used an asymmetric reinforcement schedule to assess reward learning. Healthy controls were tested under baseline conditions, stress or after receiving the dopamine D2 agonist pramipexole. In addition, participants with current or past MDD or BPD were evaluated. Reinforcement learning models isolated the contributions of variation in reward sensitivity and learning rate.ResultsMDD and anhedonia reduced reward sensitivity more than they affected the learning rate, while a low dose of the dopamine D2 agonist pramipexole showed the opposite pattern. Stress led to a pattern consistent with a mixed effect on reward sensitivity and learning rate.ConclusionReward-related learning reflected at least two partially separable contributions. The first related to phasic prediction error signalling, and was preferentially modulated by a low dose of the dopamine agonist pramipexole. The second related directly to reward sensitivity, and was preferentially reduced in MDD and anhedonia. Stress altered both components. Collectively, these findings highlight the contribution of model-based reinforcement learning meta-analysis for dissecting anhedonic behavior.

Highlights

Depression is characterised partly by blunted reactions to reward
Qt(at, st) depends on four factors: the binary sequence rt up to that point in time, which indicates whether a reward was delivered or not, an initial Q0 value, the learning rate and the subjective effect size of a reward ρ, which we identify with reward sensitivity
Categorical comparisons We examined how learning rate and reward sensitivity were affected by the factors explored in each of the individual datasets

Summary

Introduction

Depression is characterised partly by blunted reactions to reward. tasks probing this deficiency have not distinguished insensitivity to reward from insensitivity to the prediction errors for reward that determine learning and are putatively reported by the phasic activity of dopamine neurons. Anhedonia is one of the cardinal symptoms for a clinical diagnosis of major depressive disorder (MDD; [1,2,3]) and refers to an inability to experience pleasure or a diminished reactivity to pleasurable stimuli. It is typically measured by verbal reports. This is possibly the closest behavioural equivalent to the notion of a reduction in consummatory pleasure Instruments measuring anhedonia, such as the relevant subscores of the Beck Depression Inventory [16] or the Mood and Anxiety Symptom Questionnaire (MASQ) [17] typically focus on this factor [18]. It is most important to separate these factors, since they are likely to be associated with radically different ætiologies and therapeutic routes

Objectives

Methods

Results

Discussion

Conclusion