Reinforcement learning across the rat estrous cycle

Jeroen P.H Verharen,Jiska Kentrop,Louk J.M.J Vanderschuren,Roger A.H Adan

doi:10.1016/j.psyneuen.2018.09.016

Jeroen P.H Verharen, Jiska Kentrop + Show 2 more

Open Access

https://doi.org/10.1016/j.psyneuen.2018.09.016

Copy DOI

Abstract

Reinforcement learning, the process by which an organism flexibly adapts behavior in response to reward and punishment, is vital for the proper execution of everyday behaviors, and its dysfunction has been implicated in a wide variety of mental disorders. Here, we use computational trial-by-trial analysis of data of female rats performing a probabilistic reward learning task and demonstrate that core computational processes underlying value-based decision making fluctuate across the estrous cycle, providing a neuroendocrine substrate by which gonadal hormones may influence adaptive behavior.

Highlights

Reinforcement learning is an essential mechanism for organisms to adapt to a dynamic environment, by allowing flexible alterations in behavior in response to positive and negative feedback, for example during foraging and social encounters (Sutton and Barto, 1998)
We tested a cohort of female rats on a probabilistic reversal learning paradigm (Bari et al, 2010; Verharen et al, 2018), used computational modeling to extract the subcomponents of value-based decision making, and assessed how these components were affected by the estrous cycle
To gain insight into whether these underlying processes were modulated by the cycle, we fit the trial-by-trial data in the session to a computational reinforcement learning model (Gershman, 2016), and used Bayesian hierarchical estimation (Daw, 2009) to determine the parameter values that best described the behavior of the animals (Fig. 1d)

Summary

Introduction

Reinforcement learning is an essential mechanism for organisms to adapt to a dynamic environment, by allowing flexible alterations in behavior in response to positive and negative feedback, for example during foraging and social encounters (Sutton and Barto, 1998). Deficits in reinforcement learning have been implicated in several psychiatric conditions, including addiction and schizophrenia (Maia and Frank, 2011). Given the large gender differences in the prevalence of mental disorders, and the existence of cyclic changes in the severity of schizophrenia and sensitivity to drugs in women (Hendrick et al, 1996), we sought to determine how the estrous cycle of females affects the computational processes that underlie reinforcement learning. To this aim, we tested a cohort of female rats on a probabilistic reversal learning paradigm (Bari et al, 2010; Verharen et al, 2018), used computational modeling to extract the subcomponents of value-based decision making, and assessed how these components were affected by the estrous cycle

Methods

Results

Conclusion