Cardiac Concomitants of Feedback and Prediction Error Processing in Reinforcement Learning.

Lucas Kastner,Arno Villringer,Jana Kube,Jane Neumann

doi:10.3389/fnins.2017.00598

Abstract

Successful learning hinges on the evaluation of positive and negative feedback. We assessed differential learning from reward and punishment in a monetary reinforcement learning paradigm, together with cardiac concomitants of positive and negative feedback processing. On the behavioral level, learning from reward resulted in more advantageous behavior than learning from punishment, suggesting a differential impact of reward and punishment on successful feedback-based learning. On the autonomic level, learning and feedback processing were closely mirrored by phasic cardiac responses on a trial-by-trial basis: (1) Negative feedback was accompanied by faster and prolonged heart rate deceleration compared to positive feedback. (2) Cardiac responses shifted from feedback presentation at the beginning of learning to stimulus presentation later on. (3) Most importantly, the strength of phasic cardiac responses to the presentation of feedback correlated with the strength of prediction error signals that alert the learner to the necessity for behavioral adaptation. Considering participants' weight status and gender revealed obesity-related deficits in learning to avoid negative consequences and less consistent behavioral adaptation in women compared to men. In sum, our results provide strong new evidence for the notion that during learning phasic cardiac responses reflect an internal value and feedback monitoring system that is sensitive to the violation of performance-based expectations. Moreover, inter-individual differences in weight status and gender may affect both behavioral and autonomic responses in reinforcement-based learning.

Highlights

Reinforcement learning describes the process of adapting behavior according to the consequences of actions
We wished to address three research questions: (1) Can we observe systematic differences in leaning from reward and learning from punishment in reinforcement learning? (2) How is feedback processing and learning reflected in phasic cardiac responses during reinforcement learning? (3) How do gender and weight status impact on behavioral measures and cardiac concomitants of reinforcement learning? we applied an experimental design that comprised independent reward and punishment conditions
Using a probabilistic learning task, we investigated the cardiac concomitants of reinforcement-based learning and the impact of weight status and gender on learning performance

Summary

Introduction

Reinforcement learning describes the process of adapting behavior according to the consequences of actions. In reinforcement learning positive and negative feedback provide the learner with the necessary information for successful behavioral adaptation. We wished to address three research questions: (1) Can we observe systematic differences in leaning from reward and learning from punishment in reinforcement learning? (2) How is feedback processing and learning reflected in phasic cardiac responses during reinforcement learning? (3) How do gender and weight status impact on behavioral measures and cardiac concomitants of reinforcement learning? We first introduce the main concepts of reinforcement learning and phasic cardiac responses and derive hypotheses for our research questions, followed by the presentation of our experimental task, measurement techniques and analysis methods. We present our results in the order of our research question, i.e., first regarding the behavioral level, second regarding the autonomic level, and regarding the effects of weight status and gender on both behavioral and autonomic measures

Objectives

Methods

Results

Discussion

Conclusion