Spontaneous eye blink rate predicts individual differences in exploration and exploitation during reinforcement learning

Joanne C Van Slooten,Sara Jahfari,Jan Theeuwes

doi:10.1038/s41598-019-53805-y

Abstract

Spontaneous eye blink rate (sEBR) has been linked to striatal dopamine function and to how individuals make value-based choices after a period of reinforcement learning (RL). While sEBR is thought to reflect how individuals learn from the negative outcomes of their choices, this idea has not been tested explicitly. This study assessed how individual differences in sEBR relate to learning by focusing on the cognitive processes that drive RL. Using Bayesian latent mixture modelling to quantify the mapping between RL behaviour and its underlying cognitive processes, we were able to differentiate low and high sEBR individuals at the level of these cognitive processes. Further inspection of these cognitive processes indicated that sEBR uniquely indexed explore-exploit tendencies during RL: lower sEBR predicted exploitative choices for high valued options, whereas higher sEBR predicted exploration of lower value options. This relationship was additionally supported by a network analysis where, notably, no link was observed between sEBR and how individuals learned from negative outcomes. Our findings challenge the notion that sEBR predicts learning from negative outcomes during RL, and suggest that sEBR predicts individual explore-exploit tendencies. These then influence value sensitivity during choices to support successful performance when facing uncertain reward.

Highlights

Spontaneous eye blink rate has been linked to striatal dopamine function and to how individuals make value-based choices after a period of reinforcement learning (RL)
Formal learning theories posit that different cognitive processes contribute to learning[1]: the learning rate determines the magnitude by which individuals update their beliefs about the environment after positive or negative outcomes, and their explore-exploit tendency describes the sensitivity to exploit actions that likely result in reward
We evaluate how variability in each underlying cognitive process uniquely relates to individual differences in Spontaneous eye blink rate (sEBR), thereby controlling for the effects of all other variables with a network approach

Summary

Introduction

Spontaneous eye blink rate (sEBR) has been linked to striatal dopamine function and to how individuals make value-based choices after a period of reinforcement learning (RL). Formal learning theories posit that different cognitive processes contribute to learning[1]: the learning rate determines the magnitude by which individuals update their beliefs about the environment after positive or negative outcomes, and their explore-exploit tendency describes the sensitivity to exploit actions that likely result in reward. These different processes can have similar effects on final learned behaviour. Even more so as recent literature suggests very different dopaminergic mechanisms in using value to make decisions (explore-exploit) and updating values (learning)[23,24,25]

Methods

Results

Conclusion