Agency rescues competition for credit assignment among predictive cues from adverse learning conditions

Mihwa Kang,Fahd H Alhazmi,Ingrid Reverte,Inmaculada Marquez,Keith Kaufman,Anna Matarazzo,Mihaela D Iordanova,Guillem R Esber,Stephen Volz,Salvatore Fevola

doi:10.1038/s41598-021-95541-2

Abstract

A fundamental assumption of learning theories is that the credit assigned to predictive cues is not simply determined by their probability of reinforcement, but by their ability to compete with other cues present during learning. This assumption has guided behavioral and neural science research for decades, and tremendous empirical and theoretical advances have been made identifying the mechanisms of cue competition. However, when learning conditions are not optimal (e.g., when training is massed), cue competition is attenuated. This failure of the learning system exposes the individual’s vulnerability to form spurious associations in the real world. Here, we uncover that cue competition in rats can be rescued when conditions are suboptimal provided that the individual has agency over the learning experience. Our findings reveal a new effect of agency over learning on credit assignment among predictive cues, and open new avenues of investigation into the underlying mechanisms.

Highlights

A fundamental assumption of learning theories is that the credit assigned to predictive cues is not determined by their probability of reinforcement, but by their ability to compete with other cues present during learning
One such condition is experiencing the trials in massed fashion; that is, separated by an intertrial interval (ITI) that is typically shorter than twice the duration of the cue[23], as reported in rats24,25, pigeons[26], and humans[27]
Competitive credit assignment among environmental cues is the backbone of associative and reinforcement learning models of Pavlovian conditioning, to the point that an inability to account for cue-competition phenomena renders a model obsolete[37]

Summary

Introduction

A fundamental assumption of learning theories is that the credit assigned to predictive cues is not determined by their probability of reinforcement, but by their ability to compete with other cues present during learning. It has long been known that cue competition is not ubiquitous[17,18,19] and can be disrupted across multiple learning conditions[20,21,22] One such condition is experiencing the trials in massed fashion; that is, separated by an intertrial interval (ITI) that is typically shorter than twice the duration of the cue[23], as reported in rats24,25, pigeons[26], and humans[27]. Using a variety of well-established and novel cue competition tasks, we found robust evidence of cue competition only in animals that had agency over learning This effect cannot be explained by differential levels of engagement, general discrimination competence, or ability to process compounded stimuli concurrently. Our data provide the first demonstration of a critical role for agency in how credit is apportioned among predictive cues and open up new lines of neural and theoretical inquiry

Methods

Results

Conclusion