Computational Models Of Reinforcement Learning Research Articles

Reinforcement learning of altruistic punishment differs between cultures and across the lifespan.

Altruistic punishment is key to establishing cooperation and maintaining social order, yet its developmental trends across cultures remain unclear. Using computational reinforcement learning models, we provided the first evidence of how social feedback dynamically influences group-biased altruistic punishment across cultures and the lifespan. Study 1 (n = 371) found that Chinese participants exhibited higher learning rates than Americans when socially incentivized to punish unfair allocations. Additionally, Chinese adults showed slower learning and less exploration when punishing ingroups than outgroups, a pattern absent in American counterparts, potentially reflecting a tendency towards ingroup favoritism that may contribute to reinforcing collectivist values. Study 2 (n = 430, aged 12-52) further showed that such ingroup favoritism develops with age. Chinese participants' learning rates for ingroup punishment decreased from adolescence into adulthood, while outgroup rates stayed constant, implying a process of cultural learning. Our findings highlight cultural and age-related variations in altruistic punishment learning, with implications for social reinforcement learning and culturally sensitive educational practices promoting fairness and altruism.

Neurocomputational Mechanisms Underlying Differential Reinforcement Learning From Wins and Losses in Obesity With and Without Binge Eating

BackgroundBinge-eating disorder (BED) is thought of as a disorder of cognitive control, but evidence regarding its neurocognitive mechanisms is inconclusive. Key limitations of previous research include a lack of consistent separation between effects of BED and obesity and a disregard for self-report evidence suggesting that neurocognitive alterations may emerge primarily in loss- or harm-avoidance contexts. MethodsTo address these gaps, in this longitudinal study we investigated behavioral flexibility and its underlying neurocomputational processes in reward-seeking and loss-avoidance contexts. Obese participants with BED, obese participants without BED, and healthy normal-weight participants (n = 96) performed a probabilistic reversal learning task during functional imaging, with different blocks focused on obtaining wins or avoiding losses. They were reinvited for a 6-month follow-up assessment. ResultsAnalyses informed by computational models of reinforcement learning showed that unlike obese participants with BED, obese participants without BED performed worse in the win than in the loss condition. Computationally, this was explained by differential learning sensitivities in the win versus loss conditions in the groups. In the brain, this was echoed in differential neural learning signals in the ventromedial prefrontal cortex per condition. The differences were subtle but scaled with BED symptoms, such that more severe BED symptoms were associated with increasing bias toward improved learning from wins versus losses. Across conditions, obese participants with BED switched more between choice options than healthy normal-weight participants. This was reflected in diminished representation of choice certainty in the ventromedial prefrontal cortex. ConclusionsOur study highlights the importance of distinguishing between obesity with and without BED to identify unique neurocomputational alterations underlying different styles of maladaptive eating behavior.

Impairment in acquisition of conditioned fear in people with depressive symptoms.

Depression is one of the primary global public health issues, and there has been a dramatic increase in depression levels among young people over the past decade. The neuroplasticity theory of depression postulates that a malfunction in neural plasticity, which is responsible for learning, memory, and adaptive behavior, is the primary source of the disorder's clinical manifestations. Nevertheless, the impact of depression symptoms on associative learning remains underexplored. We used the differential fear conditioning paradigm to investigate the effects of depressive symptoms on fear acquisition and extinction learning. Skin conductance response (SCR) is an objective evaluation indicator, and ratings of nervousness, likeability, and unconditioned stimuli (US) expectancy are subjective evaluation indicators. In addition, we used associability generated by a computational reinforcement learning model to characterize the skin conductance response. The findings indicate that individuals with depressive symptoms exhibited significant impairment in fear acquisition learning compared to those without depressive symptoms based on the results of the skin conductance response. Moreover, in the discrimination fear learning task, the skin conductance response was positively correlated with associability, as estimated by the hybrid model in the group without depressive symptoms. Additionally, the likeability rating scores improved post-extinction learning in the group without depressive symptoms, and no such increase was observed in the group with depressive symptoms. The study highlights that individuals with pronounced depressive symptoms exhibit impaired fear acquisition and extinction learning, suggesting a possible deficit in associative learning. Employing the hybrid model to analyze the learning process offers a deeper insight into the associative learning processes of humans, thus allowing for improved comprehension and treatment of these mental health problems.

Reduction of Aversive Learning Rates in Pavlovian Conditioning by Angiotensin II Antagonist Losartan: A Randomized Controlled Trial

BackgroundAngiotensin receptor blockade has been linked to aspects of aversive learning and memory formation and to the prevention of posttraumatic stress disorder symptom development. MethodsWe investigated the influence of the angiotensin receptor blocker losartan on aversive Pavlovian conditioning using a probabilistic learning paradigm. In a double-blind, randomized, placebo-controlled design, we tested 45 (18 female) healthy volunteers during a baseline session, after application of losartan or placebo (drug session), and during a follow-up session. During each session, participants engaged in a task in which they had to predict the probability of an electrical stimulation on every trial while the true shock contingencies switched repeatedly between phases of high and low shock threat. Computational reinforcement learning models were used to investigate learning dynamics. ResultsAcute administration of losartan significantly reduced participants’ adjustment during both low-to-high and high-to-low threat changes. This was driven by reduced aversive learning rates in the losartan group during the drug session compared with baseline. The 50-mg drug dose did not induce reduction of blood pressure or change in reaction times, ruling out a general reduction in attention and engagement. Decreased adjustment of aversive expectations was maintained at a follow-up session 24 hours later. ConclusionsThis study shows that losartan acutely reduces Pavlovian learning in aversive environments, thereby highlighting a potential role of the renin-angiotensin system in anxiety development.

Postnatal Phencyclidine-Induced Deficits in Decision Making Are Ameliorated by Optogenetic Inhibition of Ventromedial Orbitofrontal Cortical Glutamate Neurons

BackgroundThe orbitofrontal cortex (OFC) is essential for decision making, and functional disruptions within the OFC are evident in schizophrenia. Postnatal phencyclidine (PCP) administration in rats is a neurodevelopmental manipulation that induces schizophrenia-relevant cognitive impairments. We aimed to determine whether manipulating OFC glutamate cell activity could ameliorate postnatal PCP-induced deficits in decision making. MethodsMale and female Wistar rats (n = 110) were administered saline or PCP on postnatal days 7, 9, and 11. In adulthood, we expressed YFP (yellow fluorescent protein) (control), ChR2 (channelrhodopsin-2) (activation), or eNpHR 3.0 (enhanced halorhodopsin) (inhibition) in glutamate neurons within the ventromedial OFC (vmOFC). Rats were tested on the probabilistic reversal learning task once daily for 20 days while we manipulated the activity of vmOFC glutamate cells. Behavioral performance was analyzed using a Q-learning computational model of reinforcement learning. ResultsCompared with saline-treated rats expressing YFP, PCP-treated rats expressing YFP completed fewer reversals, made fewer win-stay responses, and had lower learning rates. We induced similar performance impairments in saline-treated rats by activating vmOFC glutamate cells (ChR2). Strikingly, PCP-induced performance deficits were ameliorated when the activity of vmOFC glutamate cells was inhibited (halorhodopsin). ConclusionsPostnatal PCP-induced deficits in decision making are associated with hyperactivity of vmOFC glutamate cells. Thus, normalizing vmOFC activity may represent a potential therapeutic target for decision-making deficits in patients with schizophrenia.

L-DOPA and oxytocin influence the neurocomputational mechanisms of self-benefitting and prosocial reinforcement learning

Humans learn through reinforcement, particularly when outcomes are unexpected. Recent research suggests similar mechanisms drive how we learn to benefit other people, that is, how we learn to be prosocial. Yet the neurochemical mechanisms underlying such prosocial computations remain poorly understood. Here, we investigated whether pharmacological manipulation of oxytocin and dopamine influence the neurocomputational mechanisms underlying self-benefitting and prosocial reinforcement learning. Using a double-blind placebo-controlled cross-over design, we administered intranasal oxytocin (24 IU), dopamine precursor l-DOPA (100 mg + 25 mg carbidopa), or placebo over three sessions. Participants performed a probabilistic reinforcement learning task with potential rewards for themselves, another participant, or no one, during functional magnetic resonance imaging. Computational models of reinforcement learning were used to calculate prediction errors (PEs) and learning rates. Participants behavior was best explained by a model with different learning rates for each recipient, but these were unaffected by either drug. On the neural level, however, both drugs blunted PE signaling in the ventral striatum and led to negative signaling of PEs in the anterior mid-cingulate cortex, dorsolateral prefrontal cortex, inferior parietal gyrus, and precentral gyrus, compared to placebo, and regardless of recipient. Oxytocin (versus placebo) administration was additionally associated with opposing tracking of self-benefitting versus prosocial PEs in dorsal anterior cingulate cortex, insula and superior temporal gyrus. These findings suggest that both l-DOPA and oxytocin induce a context-independent shift from positive towards negative tracking of PEs during learning. Moreover, oxytocin may have opposing effects on PE signaling when learning to benefit oneself versus another.

Effect of lysergic acid diethylamide (LSD) on reinforcement learning in humans.

The non-selective serotonin 2A (5-HT2A) receptor agonist lysergic acid diethylamide (LSD) holds promise as a treatment for some psychiatric disorders. Psychedelic drugs such as LSD have been suggested to have therapeutic actions through their effects on learning. The behavioural effects of LSD in humans, however, remain incompletely understood. Here we examined how LSD affects probabilistic reversal learning (PRL) in healthy humans. Healthy volunteers received intravenous LSD (75 μg in 10 mL saline) or placebo (10 mL saline) in a within-subjects design and completed a PRL task. Participants had to learn through trial and error which of three stimuli was rewarded most of the time, and these contingencies switched in a reversal phase. Computational models of reinforcement learning (RL) were fitted to the behavioural data to assess how LSD affected the updating ('learning rates') and deployment of value representations ('reinforcement sensitivity') during choice, as well as 'stimulus stickiness' (choice repetition irrespective of reinforcement history). Raw data measures assessing sensitivity to immediate feedback ('win-stay' and 'lose-shift' probabilities) were unaffected, whereas LSD increased the impact of the strength of initial learning on perseveration. Computational modelling revealed that the most pronounced effect of LSD was the enhancement of the reward learning rate. The punishment learning rate was also elevated. Stimulus stickiness was decreased by LSD, reflecting heightened exploration. Reinforcement sensitivity differed by phase. Increased RL rates suggest LSD induced a state of heightened plasticity. These results indicate a potential mechanism through which revision of maladaptive associations could occur in the clinical application of LSD.

Action suppression reveals opponent parallel control via striatal circuits.

The direct and indirect pathways of the basal ganglia are classically thought to promote and suppress action, respectively1. However, the observed co-activation of striatal direct and indirect medium spiny neurons2 (dMSNs and iMSNs, respectively) has challenged this view. Here we study these circuits in mice performing an interval categorization task that requires a series of self-initiated and cued actions and, critically, a sustained period of dynamic action suppression. Although movement produced the co-activation of iMSNs and dMSNs in the sensorimotor, dorsolateral striatum (DLS), fibre photometry and photo-identified electrophysiological recordings revealed signatures of functional opponency between the two pathways during action suppression. Notably, optogenetic inhibition showed that DLS circuits were largely engaged to suppress-and not promote-action. Specifically, iMSNs on a given hemisphere were dynamically engaged to suppress tempting contralateral action. To understand how such regionally specific circuit function arose, we constructed a computational reinforcement learning model that reproduced key features of behaviour, neural activity and optogenetic inhibition. The model predicted that parallel striatal circuits outside the DLS learned the action-promoting functions, generating the temptation to act. Consistent with this, optogenetic inhibition experiments revealed that dMSNs in the associative, dorsomedial striatum, in contrast to those in the DLS, promote contralateral actions. These data highlight how opponent interactions between multiple circuit- and region-specific basal ganglia processes can lead to behavioural control, and establish a critical role for the sensorimotor indirect pathway in the proactive suppression of tempting actions.

Oxytocin modulates neurocomputational mechanisms underlying prosocial reinforcement learning.

Humans often act in the best interests of others. However, how we learn which actions result in good outcomes for other people and the neurochemical systems that support this 'prosocial learning' remain poorly understood. Using computational models of reinforcement learning, functional magnetic resonance imaging and dynamic causal modelling, we examined how different doses of intranasal oxytocin, a neuropeptide linked to social cognition, impact how people learn to benefit others (prosocial learning) and whether this influence could be dissociated from how we learn to benefit ourselves (self-oriented learning). We show that a low dose of oxytocin prevented decreases in prosocial performance over time, despite no impact on self-oriented learning. Critically, oxytocin produced dose-dependent changes in the encoding of prediction errors (PE) in the midbrain-subgenual anterior cingulate cortex (sgACC) pathway specifically during prosocial learning. Our findings reveal a new role of oxytocin in prosocial learning by modulating computations of PEs in the midbrain-sgACC pathway.

Modeling the Social Reinforcement of Misinformation Dissemination on Social Media

Despite the salience of misinformation and its consequences, there still lies a tremendous gap in research on the broader tendencies in collective cognition that compels individuals to spread misinformation so excessively. This study examined social learning as an antecedent of engaging with misinformation online. Using data released by Twitter for academic research in 2018, Tweets that included URL news links of both known misinformation and reliable domains were analyzed. Lindström’s computational reinforcement learning model was adapted as an expression of social learning, where a Twitter user’s posting frequency of news links is dependent on the relative engagement they receive in consequence. The research found that those who shared misinformation were highly sensitive to social reward. Inflation of positive social feedback was associated with a decrease in posting latency, indicating that users that posted misinformation were strongly influenced by social learning. However, the posting frequency of authentic news sharers remained fixed, even after receiving an increase in relative and absolute engagement. The results identified social learning is a contributor to the spread of misinformation online. In addition, behavior driven by social validation suggests a positive correlation between posting frequency, gratification received from posting, and a growing mental health dependency on social media. Developing interventions for spreading misinformation online may profit by assessing which online environments amplify social learning, particularly the conditions under which misinformation proliferates.

Altered Reinforcement Learning from Reward and Punishment in Anorexia Nervosa: Evidence from Computational Modeling.

Anorexia nervosa (AN) is associated with altered sensitivity to reward and punishment. Few studies have investigated whether this results in aberrant learning. The ability to learn from rewarding and aversive experiences is essential for flexibly adapting to changing environments, yet individuals with AN tend to demonstrate cognitive inflexibility, difficulty set-shifting and altered decision-making. Deficient reinforcement learning may contribute to repeated engagement in maladaptive behavior. This study investigated learning in AN using a probabilistic associative learning task that separated learning of stimuli via reward from learning via punishment. Forty-two individuals with Diagnostic and Statistical Manual of Mental Disorders (DSM)-5 restricting-type AN were compared to 38 healthy controls (HCs). We applied computational models of reinforcement learning to assess group differences in learning, thought to be driven by violations in expectations, or prediction errors (PEs). Linear regression analyses examined whether learning parameters predicted BMI at discharge. AN had lower learning rates than HC following both positive and negative PE (p<.02), and were less likely to exploit what they had learned. Negative PE on punishment trials predicted lower discharge BMI (p<.001), suggesting individuals with more negative expectancies about avoiding punishment had the poorest outcome. This is the first study to show lower rates of learning in AN following both positive and negative outcomes, with worse punishment learning predicting less weight gain. An inability to modify expectations about avoiding punishment might explain persistence of restricted eating despite negative consequences, and suggests that treatments that modify negative expectancy might be effective in reducing food avoidance in AN.

Ageing is associated with disrupted reinforcement learning whilst learning to help others is preserved

Reinforcement learning is a fundamental mechanism displayed by many species. However, adaptive behaviour depends not only on learning about actions and outcomes that affect ourselves, but also those that affect others. Using computational reinforcement learning models, we tested whether young (age 18–36) and older (age 60–80, total n = 152) adults learn to gain rewards for themselves, another person (prosocial), or neither individual (control). Detailed model comparison showed that a model with separate learning rates for each recipient best explained behaviour. Young adults learned faster when their actions benefitted themselves, compared to others. Compared to young adults, older adults showed reduced self-relevant learning rates but preserved prosocial learning. Moreover, levels of subclinical self-reported psychopathic traits (including lack of concern for others) were lower in older adults and the core affective-interpersonal component of this measure negatively correlated with prosocial learning. These findings suggest learning to benefit others is preserved across the lifespan with implications for reinforcement learning and theories of healthy ageing.

Modeling changes in probabilistic reinforcement learning during adolescence.

In the real world, many relationships between events are uncertain and probabilistic. Uncertainty is also likely to be a more common feature of daily experience for youth because they have less experience to draw from than adults. Some studies suggest probabilistic learning may be inefficient in youths compared to adults, while others suggest it may be more efficient in youths in mid adolescence. Here we used a probabilistic reinforcement learning task to test how youth age 8-17 (N = 187) and adults age 18-30 (N = 110) learn about stable probabilistic contingencies. Performance increased with age through early-twenties, then stabilized. Using hierarchical Bayesian methods to fit computational reinforcement learning models, we show that all participants’ performance was better explained by models in which negative outcomes had minimal to no impact on learning. The performance increase over age was driven by 1) an increase in learning rate (i.e. decrease in integration time scale); 2) a decrease in noisy/exploratory choices. In mid-adolescence age 13-15, salivary testosterone and learning rate were positively related. We discuss our findings in the context of other studies and hypotheses about adolescent brain development.

Reward and fictive prediction error signals in ventral striatum: asymmetry between factual and counterfactual processing.

Reward prediction error, the difference between the expected and obtained reward, is known to act as a reinforcement learning neural signal. In the current study, we propose a model fitting approach that combines behavioral and neural data to fit computational models of reinforcement learning. Briefly, we penalized subject-specific fitted parameters that moved away too far from the group median, except when that deviation led to an improvement in the model's fit to neural responses. By means of a probabilistic monetary learning task and fMRI, we compared our approach with standard model fitting methods. Q-learning outperformed actor-critic at both behavioral and neural level, although the inclusion of neuroimaging data into model fitting improved the fit of actor-critic models. We observed both action-value and state-value prediction error signals in the striatum, while standard model fitting approaches failed to capture state-value signals. Finally, left ventral striatum correlated with reward prediction error while right ventral striatum with fictive prediction error, suggesting a functional hemispheric asymmetry regarding prediction-error driven learning.

Beyond the average: The role of variable reward sensitivity in eating disorders

Eating disorders are often characterized by episodes of overeating and undereating. To date, most theories have explained the liability for such episodes by differences in traits such as reward sensitivity or cognitive control. Here, we review the evidence for a more parsimonious account of the waxing and waning in food intake by linking it to state-like variability of alleged traits such as reward sensitivity. To formally demonstrate that our variability model of eating disorders could explain a wide range of observed reward-related behavior, we conducted simulations of value-based choices and learning. These simulations based on well-established computational models of reinforcement learning and Bayesian sequential updating show how variability may arise and manifest in eating behavior. We argue that by reconceptualizing stable traits as distributions over likely states promoting adaptation, our proposed model integrates disparate findings and leads to novel predictions in a quantitative framework. Collectively, these emerging results call for a stronger emphasis on within-person variability to improve mechanistic insights into eating disorders.

Stimulation of the vagus nerve reduces learning in a go/no-go reinforcement learning task

When facing decisions to approach rewards or to avoid punishments, we often figuratively go with our gut, and the impact of metabolic states such as hunger on motivation are well documented. However, whether and how vagal feedback signals from the gut influence instrumental actions is unknown. Here, we investigated the effect of non-invasive transcutaneous auricular vagus nerve stimulation (taVNS) vs. sham (randomized cross-over design) on approach and avoidance behavior using an established go/no-go reinforcement learning paradigm in 39 healthy human participants (23 female) after an overnight fast. First, mixed-effects logistic regression analysis of choice accuracy showed that taVNS acutely impaired decision-making, p = .041. Computational reinforcement learning models identified the cause of this as a reduction in the learning rate through taVNS (∆α = −0.092, pboot = .002), particularly after punishment (∆αPun = −0.081, pboot = .012 vs. ∆αRew =−0.031, pboot = .22). However, taVNS had no effect on go biases, Pavlovian response biases or response time. Hence, taVNS appeared to influence learning rather than action execution. These results highlight a novel role of vagal afferent input in modulating reinforcement learning by tuning the learning rate according to homeostatic needs.

Altered Reinforcement Learning From Reward and Punishment in Anorexia Nervosa: Evidence From Computational Modeling

OBJECTIVES Anorexia nervosa (AN) is associated with altered sensitivity to reward and punishment. Few studies have investigated whether this results in aberrant learning. The ability to learn from rewarding and aversive experiences is essential for flexibly adapting to changing environments, yet individuals with AN tend to demonstrate cognitive inflexibility, difficulty set-shifting and altered decision-making. Deficient reinforcement learning may contribute to repeated engagement in maladaptive behavior. METHODS This study investigated learning in AN using a probabilistic associative learning task that separated learning of stimuli via reward from learning via punishment. Forty-two individuals with Diagnostic and Statistical Manual of Mental Disorders (DSM)-5 restricting-type AN were compared to 38 healthy controls (HCs). We applied computational models of reinforcement learning to assess group differences in learning, thought to be driven by violations in expectations, or prediction errors (PEs). Linear regression analyses examined whether learning parameters predicted BMI at discharge. RESULTS AN had lower learning rates than HC following both positive and negative PE (p < .02), and were less likely to exploit what they had learned. Negative PE on punishment trials predicted lower discharge BMI (p < .001), suggesting individuals with more negative expectancies about avoiding punishment had the poorest outcome. CONCLUSIONS This is the first study to show lower rates of learning in AN following both positive and negative outcomes, with worse punishment learning predicting less weight gain. An inability to modify expectations about avoiding punishment might explain persistence of restricted eating despite negative consequences, and suggests that treatments that modify negative expectancy might be effective in reducing food avoidance in AN.

Cognitive inflexibility specificity for individuals with high levels of obsessive-compulsive symptoms

Abstract Obsessive and compulsive symptoms are only partially explained by current cognitive-behavioral models. A complementary proposal, behavioral dysregulation, considers cognitive inflexibility (a malfunction of the capacity of switching between mental processes in order to generate appropriate behavioral responses) as an endophenotype of OCD. However, it is unclear whether deficits in cognitive flexibility are specific to individuals with OCD, or whether cognitive underperformance in OCD should instead be explained by nonspecific factors. Here, we probed cognitive flexibility by using symptom-related stimuli in a probabilistic reversal learning task. We compared performance of individuals with two closely related OCD-related phenomena: OC symptoms and “Not Just Right Experiences.” Data were analyzed through computational models of reinforcement learning. Academic achievement was used for comparison as an unrelated learning task. Our results support the inflexibility specificity hypothesis, but only for the processing of symptom-related triggers. Clinical implications are discussed in relation to contemporary CBT models.

Enhanced Go and NoGo Learning in Individuals With Obesity.

Overeating in individuals with obesity is hypothesized to be partly caused by automatic action tendencies to food cues that have the potential to override goal-directed dietary restriction. Individuals with obesity are often characterized by alterations in the processing of such rewarding food, but also of non-food stimuli, and previous research has suggested a stronger impact on the execution of goal-directed actions in obesity. Here, we investigated whether Pavlovian cues can also corrupt the learning of new approach or withdrawal behavior in individuals with obesity. We employed a probabilistic Pavlovian-instrumental learning paradigm in which participants (29 normal-weight and 29 obese) learned to actively respond (Go learning) or withhold a response (NoGo learning) in order to gain monetary rewards or avoid losses. Participants were better at learning active approach responses (Go) in the light of anticipated rewards and at learning to withhold a response (NoGo) in the light of imminent punishments. Importantly, there was no evidence for a stronger corruption of instrumental learning in individuals with obesity. Instead, they showed better learning across conditions than normal-weight participants. Using a computational reinforcement learning model, we additionally found an increased learning rate in individuals with obesity. Previous studies have mostly reported a lower reinforcement learning performance in individuals with obesity. Our results contradict this and suggest that their performance is not universally impaired: Instead, while previous studies found reduced stimulus-value learning, individuals with obesity may show better action-value learning. Our findings highlight the need for a broader investigation of behavioral adaptation in obesity across different task designs and types of reinforcement learning.

Dose-dependent effects of estrogen on prediction error related neural activity in the nucleus accumbens of healthy young women.

Whereas the effect of the sex steroid 17-beta-estradiol (E2) on dopaminergic (DA) transmission in the nucleus accumbens (NAc) is well evidenced in female rats, studies in humans are inconsistent. Moreover, linear and inverted u-shaped dose response curves have been observed for E2's effects on hippocampal plasticity, but the shape of dose response curves for E2's effects on the NAc is much less characterized. Investigation of dose response curves for E2's effects on DA-related neural activity in the human NAc. Placebo or E2 valerate in doses of 2, 4, 6 or 12 mg was orally administered to 125 naturally cycling young women during the low-hormone menstruation phase on two consecutive days using a randomized, double-blinded design. The E2 treatment regimen induced a wide range of E2 levels, from physiological (2- and 4-mg groups; equivalent to cycle peak) to supraphysiological levels (6- and 12-mg groups; equivalent to early pregnancy). This made it possible to study different dose response functions for E2's effects on NAc activity. During E2 peak, participants performed a well-established reversal learning paradigm. We used trial-wise prediction errors (PE) estimated via a computational reinforcement learning model as a proxy for dopaminergic activity. Linear and quadratic regression analyses predicting PE-related NAc activity from salivary E2 levels were calculated. There was a positive linear relationship between PE-associated NAc activity and salivary E2 increases. The randomized, placebo-controlled elevation of E2 levels stimulates NAc activity in the human brain, likely mediated by dopaminergic processes.