Probabilistic Reversal Learning Research Articles

Behavioral Interventions Can Improve Brain Injury-Induced Deficits in Behavioral Flexibility and Impulsivity Linked to Impaired Reward-Feedback Beta Oscillations.

Traumatic brain injury (TBI) affects a large population, resulting in severe cognitive impairments. Although cognitive rehabilitation is an accepted treatment for some deficits, studies in patients are limited in ability to probe physiological and behavioral mechanisms. Therefore, animal models are needed to optimize strategies. Frontal TBI in a rat model results in robust and replicable cognitive deficits, making this an ideal candidate for investigating various behavioral interventions. In this study, we report three distinct frontal TBI experiments assessing behavior well into the chronic post-injury period using male Long-Evans rats. First, we evaluated the impact of frontal injury on local field potentials recorded simultaneously from 12 brain regions during a probabilistic reversal learning (PbR) task. Next, a set of rats were tested on a similar PbR task or an impulsivity task (differential reinforcement of low-rate behavior [DRL]) and half received salient cues associated with reinforcement contingencies to encourage engagement in the target behavior. After intervention on the PbR task, brains were stained for markers of activity. On the DRL task, cue relevance was decoupled from outcomes to determine if beneficial effects persisted on impulsive behavior. TBI decreased the ability to detect reinforced outcomes; this was evident in task performance and reward-feedback signals occurring at beta frequencies in lateral orbitofrontal cortex (OFC) and associated frontostriatal regions. The behavioral intervention improved flexibility and increased OFC activity. Intervention also reduced impulsivity, even after cues were decoupled, which was partially mediated by improvements in timing behavior. The current study established a platform to begin investigating cognitive rehabilitation in rats and identified a strong role for dysfunctional OFC signaling in probabilistic learning after frontal TBI.

Belief Updating, Childhood Maltreatment, and Paranoia in Schizophrenia-Spectrum Disorders.

Exposure to childhood maltreatment-a risk factor for psychosis is associated with paranoia-may impact one's beliefs about the world and how beliefs are updated. We hypothesized that increased exposure to childhood maltreatment is related to volatility-related belief updating, specifically higher expectations of volatility, and that these relationships are strongest for threat-related maltreatment. Additionally, we tested whether belief updating mediates the relationship between maltreatment and paranoia. Belief updating was measured in 75 patients with schizophrenia-spectrum disorders and 76 nonpsychiatric controls using a 3-option probabilistic reversal learning (3PRL) task. A Hierarchical Gaussian Filter (HGF) was used to estimate computational parameters of belief updating, including prior expectations of volatility (μ03). The Childhood Trauma Questionnaire (CTQ) was used to assess cumulative maltreatment, threat, and deprivation exposure. Paranoia was measured using the Positive and Negative Syndrome Scale (PANSS) and the revised Green et al. Paranoid Thoughts Scale (R-GPTS). Greater exposure to childhood maltreatment is associated with higher prior expectations of volatility in the whole sample and in individuals with schizophrenia-spectrum disorders. This was specific to threat-related maltreatment, rather than deprivation, in schizophrenia-spectrum disorders. Paranoia was associated with both exposure to childhood maltreatment and volatility priors, but we did not observe a significant indirect effect of volatility priors on the relationship between maltreatment and paranoia. Our study suggests that individuals with schizophrenia-spectrum disorders who were exposed to threatening experiences during childhood expect their environment to be more volatile, potentially facilitating aberrant belief updating and conferring risk for paranoia.

Enhanced cognitive flexibility and phasic striatal dopamine dynamics in a mouse model of low striatal tonic dopamine.

The catecholamine neuromodulators dopamine and norepinephrine are implicated in motor function, motivation, and cognition. Although roles for striatal dopamine in these aspects of behavior are well established, the specific roles for cortical catecholamines in regulating striatal dopamine dynamics and behavior are less clear. We recently showed that elevating cortical dopamine but not norepinephrine suppresses hyperactivity in dopamine transporter knockout (DAT-KO) mice, which have elevated striatal dopamine levels. In contrast, norepinephrine transporter knockout (NET-KO) mice have a phenotype distinct from DAT-KO mice, as they show elevated extracellular cortical catecholamines but reduced baseline striatal dopamine levels. Here we evaluated the consequences of altered catecholamine levels in NET-KO mice on cognitive flexibility and striatal dopamine dynamics. In a probabilistic reversal learning task, NET-KO mice showed enhanced reversal learning, which was consistent with larger phasic dopamine transients (dLight) in the dorsomedial striatum (DMS) during reward delivery and reward omission, compared to WT controls. Selective depletion of dorsal medial prefrontal cortex (mPFC) norepinephrine in WT mice did not alter performance on the reversal learning task but reduced nestlet shredding. Surprisingly, NET-KO mice did not show altered breakpoints in a progressive ratio task, suggesting intact food motivation. Collectively, these studies show novel roles of cortical catecholamines in the regulation of tonic and phasic striatal dopamine dynamics and cognitive flexibility, updating our current views on dopamine regulation and informing future therapeutic strategies to counter multiple psychiatric disorders.

Non-motor symptoms associated with progressive loss of dopaminergic neurons in a mouse model of Parkinson's disease.

Parkinson's disease (PD) is characterized by three main motor symptoms: bradykinesia, rigidity and tremor. PD is also associated with diverse non-motor symptoms that may develop in parallel or precede motor dysfunctions, ranging from autonomic system dysfunctions and impaired sensory perception to cognitive deficits and depression. Here, we examine the role of the progressive loss of dopaminergic transmission in behaviors related to the non-motor symptoms of PD in a mouse model of the disease (the TIF-IADATCreERT2 strain). We found that in the period from 5 to 12 weeks after the induction of a gradual loss of dopaminergic neurons, mild motor symptoms became detectable, including changes in the distance between paws while standing as well as the swing speed and step sequence. Male mutant mice showed no apparent changes in olfactory acuity, no anhedonia-like behaviors, and normal learning in an instrumental task; however, a pronounced increase in the number of operant responses performed was noted. Similarly, female mice with progressive dopaminergic neuron degeneration showed normal learning in the probabilistic reversal learning task and no loss of sweet-taste preference, but again, a robustly higher number of choices were performed in the task. In both males and females, the higher number of instrumental responses did not affect the accuracy or the fraction of rewarded responses. Taken together, these data reveal discrete, dopamine-dependent non-motor symptoms that emerge in the early stages of dopaminergic neuron degeneration.

Different learning aberrations relate to delusion-like beliefs with different contents.

The prediction error account of delusions has had success. However, its explanation of delusions with different contents has been lacking. Persecutory delusions and paranoia are the common unfounded beliefs that others have harmful intentions towards us. Other delusions include believing that one's thoughts or actions are under external control, or that events in the world have specific personal meaning. We compare learning on two different cognitive tasks, probabilistic reversal learning (PRL) and Kamin blocking, that have relationships to paranoid and non-paranoid delusion-like beliefs, respectively. We find that Clinical High-Risk status alone does not result in different behavioral results on the PRL task but that an individual's level of paranoia is associated with excessive switching behavior. During the Kamin blocking task, paranoid individuals learned inappropriately about the blocked cue. However, they also had decreased learning about the control cue, suggesting more general learning impairments. Non-paranoid delusion-like belief conviction (but not paranoia) was associated with aberrant learning about the blocked cue but intact learning about the control cue, suggesting specific impairments in learning related to cue combination. We fit task-specific computational models separately to behavioral data to explore how latent parameters vary within individuals between tasks, and how they can explain symptom-specific effects. We find that paranoia is associated with low learning rates on the PRL task as well as the blocking task. Non-paranoid delusion-like belief conviction was instead related to parameters controlling the degree and direction of similarity between cue updating during simultaneous cue presentation. These results suggest that paranoia and other delusion-like beliefs involve dissociable deficits in learning and belief updating, which - given the transdiagnostic status of paranoia - may have differential utility in predicting psychosis.

Characterisation of behaviours relevant to apathy syndrome in the aged male rat

Apathy is a complex psychiatric syndrome characterised by motivational deficit, emotional blunting and cognitive changes. It occurs alongside a broad range of neurological disorders, but also occurs in otherwise healthy ageing. Despite its clinical prevalence, apathy does not yet have a designated treatment strategy. Generation of a translational animal model of apathy syndrome would facilitate the development of novel treatments. Given the multidimensional nature of apathy, a model cannot be achieved with a single behavioural test. Using a battery of behavioural tests we investigated whether aged rats exhibit behavioural deficits across different domains relevant to apathy. Using the effort for reward and progressive ratio tasks we found that aged male rats (21–27 months) show intact reward motivation. Using the novelty supressed feeding test and position-based object exploration we found aged rats showed increased anxiety-like behaviour inconsistent with emotional blunting. The sucrose preference test and reward learning assay showed intact reward sensitivity and reward-related cognition in aged rats. However, using a bowl-digging version of the probabilistic reversal learning task, we found a deficit in cognitive flexibility in aged rats that did not translate across to a touchscreen version of the task. While these data reveal important changes in cognitive flexibility and anxiety associated with ageing, aged rats do not show deficits across other behavioural domains relevant to apathy. This suggests that aged rats are not a suitable model for age-related apathy syndrome. These findings contrast with previous work in mice, revealing important species differences in behaviours relevant to apathy syndrome in ageing.

Mild exogenous inflammation blunts neural signatures of bounded evidence accumulation and reward prediction error processing in healthy male participants

BackgroundAltered neural haemodynamic activity during decision making and learning has been linked to the effects of inflammation on mood and motivated behaviours. So far, it has been reported that blunted mesolimbic dopamine reward signals are associated with inflammation-induced anhedonia and apathy. Nonetheless, it is still unclear whether inflammation impacts neural activity underpinning decision dynamics. The process of decision making involves integration of noisy evidence from the environment until a critical threshold of evidence is reached. There is growing empirical evidence that such process, which is usually referred to as bounded accumulation of decision evidence, is affected in the context of mental illness. MethodsIn a randomised, placebo-controlled, crossover study, 19 healthy male participants were allocated to placebo and typhoid vaccination. Three to four hours post-injection, participants performed a probabilistic reversal-learning task during functional magnetic resonance imaging. To capture the hidden neurocognitive operations underpinning decision-making, we devised a hybrid sequential sampling and reinforcement learning computational model. We conducted whole brain analyses informed by the modelling results to investigate the effects of inflammation on the efficiency of decision dynamics and reward learning. ResultsWe found that during the decision phase of the task, typhoid vaccination attenuated neural signatures of bounded evidence accumulation in the dorsomedial prefrontal cortex, only for decisions requiring short integration time. Consistent with prior work, we showed that, in the outcome phase, mild acute inflammation blunted the reward prediction error in the bilateral ventral striatum and amygdala. ConclusionsOur study extends current insights into the effects of inflammation on the neural mechanisms of decision making and shows that exogenous inflammation alters neural activity indexing efficiency of evidence integration, as a function of choice discriminability. Moreover, we replicate previous findings that inflammation blunts striatal reward prediction error signals.

Lasting dynamic effects of the psychedelic 2,5-dimethoxy-4-iodoamphetamine ((±)-DOI) on cognitive flexibility.

Psychedelic drugs can aid fast and lasting remission from various neuropsychiatric disorders, though the underlying mechanisms remain unclear. Preclinical studies suggest serotonergic psychedelics enhance neuronal plasticity, but whether neuroplastic changes can also be seen at cognitive and behavioural levels is unexplored. Here we show that a single dose of the psychedelic 2,5-dimethoxy-4-iodoamphetamine ((±)-DOI) affects structural brain plasticity and cognitive flexibility in young adult mice beyond the acute drug experience. Using ex vivo magnetic resonance imaging, we show increased volumes of several sensory and association areas one day after systemic administration of 2 mgkg-1 (±)-DOI. We then demonstrate lasting effects of (±)-DOI on cognitive flexibility in a two-step probabilistic reversal learning task where 2 mgkg-1 (±)-DOI improved the rate of adaptation to a novel reversal in task structure occurring one-week post-treatment. Strikingly, (±)-DOI-treated mice started learning from reward omissions, a unique strategy not typically seen in mice in this task, suggesting heightened sensitivity to previously overlooked cues. Crucially, further experiments revealed that (±)-DOI's effects on cognitive flexibility were contingent on the timing between drug treatment and the novel reversal, as well as on the nature of the intervening experience. (±)-DOI's facilitation of both cognitive adaptation and novel thinking strategies may contribute to the clinical benefits of psychedelic-assisted therapy, particularly in cases of perseverative behaviours and a resistance to change seen in depression, anxiety, or addiction. Furthermore, our findings highlight the crucial role of time-dependent neuroplasticity and the influence of experiential factors in shaping the therapeutic potential of psychedelic interventions for impaired cognitive flexibility.

Prospective associations between cognitive flexibility and eating disorder symptoms in anorexia nervosa and bulimia nervosa

This study investigated concurrent and prospective associations between measures of reversal learning and attentional set-shifting and eating disorder symptoms at baseline, 3 months, and 6 months among individuals with anorexia nervosa restricting subtype (AN-R, n = 26), AN binge eating/purging subtype (AN-BP, n = 22), bulimia nervosa (BN, n = 35), and healthy controls (n = 27), and explored whether these associations differed by diagnosis. At baseline, participants completed diagnostic interviews, height/weight measurements, and measures of set-shifting (the Intradimensional/Extradimensional shift task) and reversal learning (a probabilistic reversal learning task). At 3- and 6-month follow-up, participants with eating disorders completed assessments of weight and eating disorder symptoms. A one-way analysis of variance found no evidence that baseline reversal learning and attentional set-shifting differed across diagnostic groups. Multilevel modeling analyses indicated that perseverative errors (an index of reversal learning) predicted an increase in purging over time for individuals with AN-BP and BN. Set-shifting errors differentially predicted frequency of loss of control eating for individuals with AN-BP and BN; however, set-shifting was not related to loss of control eating when examined separately in AN-BP and BN. These findings suggest that disentangling facets of cognitive flexibility may help understand change in eating disorder symptoms.

Impaired flexible reward learning in ADHD patients is associated with blunted reinforcement sensitivity and neural signals in ventral striatum and parietal cortex

Reward-based learning and decision-making are prime candidates to understand symptoms of attention deficit hyperactivity disorder (ADHD). However, only limited evidence is available regarding the neurocomputational underpinnings of the alterations seen in ADHD. This concerns flexible behavioral adaption in dynamically changing environments, which is challenging for individuals with ADHD. One previous study points to elevated choice switching in adolescent ADHD, which was accompanied by disrupted learning signals in medial prefrontal cortex.Here, we investigated young adults with ADHD (n = 17) as compared to age- and sex-matched controls (n = 17) using a probabilistic reversal learning experiment during functional magnetic resonance imaging (fMRI). The task requires continuous learning to guide flexible behavioral adaptation to changing reward contingencies. To disentangle the neurocomputational underpinnings of the behavioral data, we used reinforcement learning (RL) models, which informed the analysis of fMRI data.ADHD patients performed worse than controls particularly in trials before reversals, i.e., when reward contingencies were stable. This pattern resulted from ‘noisy’ choice switching regardless of previous feedback. RL modelling showed decreased reinforcement sensitivity and enhanced learning rates for negative feedback in ADHD patients. At the neural level, this was reflected in a diminished representation of choice probability in the left posterior parietal cortex in ADHD. Moreover, modelling showed a marginal reduction of learning about the unchosen option, which was paralleled by a marginal reduction in learning signals incorporating the unchosen option in the left ventral striatum.Taken together, we show that impaired flexible behavior in ADHD is due to excessive choice switching (‘hyper-flexibility’), which can be detrimental or beneficial depending on the learning environment. Computationally, this resulted from blunted sensitivity to reinforcement of which we detected neural correlates in the attention-control network, specifically in the parietal cortex. These neurocomputational findings remain preliminary due to the relatively small sample size.

Computational modelling of reinforcement learning and functional neuroimaging of probabilistic reversal for dissociating compulsive behaviours in gambling and cocaine use disorders.

Individuals with cocaine use disorder or gambling disorder demonstrate impairments in cognitive flexibility: the ability to adapt to changes in the environment. Flexibility is commonly assessed in a laboratory setting using probabilistic reversal learning, which involves reinforcement learning, the process by which feedback from the environment is used to adjust behavior. It is poorly understood whether impairments in flexibility differ between individuals with cocaine use and gambling disorders, and how this is instantiated by the brain. We applied computational modelling methods to gain a deeper mechanistic explanation of the latent processes underlying cognitive flexibility across two disorders of compulsivity. We present a re-analysis of probabilistic reversal data from individuals with either gambling disorder (n = 18) or cocaine use disorder (n = 20) and control participants (n = 18), using a hierarchical Bayesian approach. Furthermore, we relate behavioural findings to their underlying neural substrates through an analysis of task-based functional magnetic resonanceimaging (fMRI) data. We observed lower 'stimulus stickiness' in gambling disorder, and report differences in tracking expected values in individuals with gambling disorder compared to controls, with greater activity during reward expected value tracking in the cingulate gyrus and amygdala. In cocaine use disorder, we observed lower responses to positive punishment prediction errors and greater activity following negative punishment prediction errors in the superior frontal gyrus compared to controls. Using a computational approach, we show that individuals with gambling disorder and cocaine use disorder differed in their perseverative tendencies and in how they tracked value neurally, which has implications for psychiatric classification.

Ketamine produces no detectable long-term positive or negative effects on cognitive flexibility or reinforcement learning of male rats.

Patients with major depressive disorder (MDD) often experience abnormalities in behavioral adaptation following environmental changes (i.e., cognitive flexibility) and tend to undervalue positive outcomes but overvalue negative outcomes. The probabilistic reversal learning task (PRL) is used to study these deficits across species and to explore drugs that may have therapeutic value. Selective serotonin-reuptake inhibitors (SSRIs) have limited effectiveness in treating MDD and produce inconsistent effects in non-human versions of the PRL. As such, ketamine, a novel and potentially rapid-acting therapeutic, has begun to be examined using the PRL. Two previous studies examining the effects of ketamine in the PRL have shown conflicting results and only examined short-term effects of ketamine. This experiment examined PRL performance across a 2-week period following a single exposure to a ketamine dose that varied across groups. After five sessions of PRL training, groups of rats received an injection of either 0, 10, 20 or 30 mg/kg ketamine. One-hour post-injection, rats engaged in the PRL, and subsequently sessions continued daily for 2 weeks. Traditional behavioral and computational reinforcement learning-derived measures were examined. Results showed that ketamine had acute effects 1-h post-injection, including a significant decrease in the value of the punishment learning rate. Beyond 1 h, ketamine produced no detectable improvements nor decrements in performance across 2 weeks. Overall, the present results suggest that the range of ketamine doses examined do not have long-term positive or negative effects on cognitive flexibility or reward processing in healthy rats as measured by the PRL.

Compulsivity and Inhibitory Control Deficits in Abstinent Individuals With Heroin Addiction and Their Biological Siblings Compared With Unrelated Healthy Control Participants

BackgroundCompulsivity represents the performance of persistent and repetitive acts despite negative consequences and is considered one of the critical mechanisms for drug addiction. Although compulsivity-related neurocognitive impairments have been linked to addiction, it remains unclear whether these deficits might have predated drug abuse as potential familial susceptibilities. MethodsA large sample of 213 adult participants were recruited, including 70 abstinent individuals addicted to heroin (HAs), 69 unaffected biological siblings of the HAs (siblings), and 74 unrelated healthy control participants. Compulsivity-related neurocognitive functions were evaluated using the intradimensional/extradimensional set-shift task and a probabilistic reversal learning task. Compulsive traits were measured by the Obsessive-Compulsive Inventory-Revised. Inhibitory control was assessed using the stop signal task and Stroop Color and Word Test. Network models for group recognition were conducted using multilayer perceptron neural networks. ResultsData indicated that both HAs and siblings performed worse than healthy control participants on compulsivity-related aspects (i.e., shifting and reversal learning functions) and inhibitory control and had higher levels of self-reported compulsive traits. Furthermore, neural models revealed that a possible 3-facet clustering of neurocognitive deficits was linked to both HAs and siblings. ConclusionsOur findings suggest that deficits in shift reversal and inhibitory control aspects and elevated compulsive traits, shared by HAs and their unaffected siblings, may putatively represent conceivable markers associated with familial vulnerabilities implicated in the development of heroin dependence.

Meta-reinforcement learning via orbitofrontal cortex

The meta-reinforcement learning (meta-RL) framework, which involves RL over multiple timescales, has been successful in training deep RL models that generalize to new environments. It has been hypothesized that the prefrontal cortex may mediate meta-RL in the brain, but the evidence is scarce. Here we show that the orbitofrontal cortex (OFC) mediates meta-RL. We trained mice and deep RL models on a probabilistic reversal learning task across sessions during which they improved their trial-by-trial RL policy through meta-learning. Ca2+/calmodulin-dependent protein kinase II-dependent synaptic plasticity in OFC was necessary for this meta-learning but not for the within-session trial-by-trial RL in experts. After meta-learning, OFC activity robustly encoded value signals, and OFC inactivation impaired the RL behaviors. Longitudinal tracking of OFC activity revealed that meta-learning gradually shapes population value coding to guide the ongoing behavioral policy. Our results indicate that two distinct RL algorithms with distinct neural mechanisms and timescales coexist in OFC to support adaptive decision-making.

Comparable roles for serotonin in rats and humans for computations underlying flexible decision-making

Serotonin is critical for adapting behavior flexibly to meet changing environmental demands. Cognitive flexibility is important for successful attainment of goals, as well as for social interactions, and is frequently impaired in neuropsychiatric disorders, including obsessive–compulsive disorder. However, a unifying mechanistic framework accounting for the role of serotonin in behavioral flexibility has remained elusive. Here, we demonstrate common effects of manipulating serotonin function across two species (rats and humans) on latent processes supporting choice behavior during probabilistic reversal learning, using computational modelling. The findings support a role of serotonin in behavioral flexibility and plasticity, indicated, respectively, by increases or decreases in choice repetition (‘stickiness’) or reinforcement learning rates following manipulations intended to increase or decrease serotonin function. More specifically, the rate at which expected value increased following reward and decreased following punishment (reward and punishment ‘learning rates’) was greatest after sub-chronic administration of the selective serotonin reuptake inhibitor (SSRI) citalopram (5 mg/kg for 7 days followed by 10 mg/kg twice a day for 5 days) in rats. Conversely, humans given a single dose of an SSRI (20 mg escitalopram), which can decrease post-synaptic serotonin signalling, and rats that received the neurotoxin 5,7-dihydroxytryptamine (5,7-DHT), which destroys forebrain serotonergic neurons, exhibited decreased reward learning rates. A basic perseverative tendency (‘stickiness’), or choice repetition irrespective of the outcome produced, was likewise increased in rats after the 12-day SSRI regimen and decreased after single dose SSRI in humans and 5,7-DHT in rats. These common effects of serotonergic manipulations on rats and humans—identified via computational modelling—suggest an evolutionarily conserved role for serotonin in plasticity and behavioral flexibility and have clinical relevance transdiagnostically for neuropsychiatric disorders.

Metabolic and functional substrates of impulsive decision-making in individuals with heroin addiction after prolonged methadone maintenance treatment

Elevated impulsivity has been frequently reported in individuals with opioid addiction receiving methadone maintenance therapy (MMT), but the underlying neural mechanisms and cognitive subprocesses are not fully understood. We acquired functional magnetic resonance imaging (fMRI) data from 37 subjects with heroin addiction receiving long-term MMT and 33 healthy controls who performed a probabilistic reversal learning task, and measured their resting-state brain glucose using fluorine-18-fluorodeoxyglucose positron emission tomography (18F-FDG PET). Subjects receiving MMT exhibited significantly elevated self-reported impulsivity, and computational modeling revealed a marked impulsive decision bias manifested as switching more frequently without available evidence. Moreover, this impulsive decision bias was associated with the dose and duration of methadone use, irrelevant to the duration of heroin use. During the task, the switch-related hypoactivation in the left rostral middle frontal gyrus was correlated with the impulsive decision bias while the function of reward sensitivity was intact in subjects receiving MMT. Using prior brain-wide receptor density data, we found that the highest variance of regional metabolic abnormalities was explained by the spatial distribution of μ-opioid receptors among 10 types of neurotransmitter receptors. Heightened impulsivity in individuals receiving prolonged MMT is manifested as atypical choice bias and noise in decision-making processes, which is further driven by deficits in top-down cognitive control, other than reward sensitivity. Our findings uncover multifaceted mechanisms underlying elevated impulsivity in subjects receiving MMT, which might provide insights for developing complementary therapies to improve retention during MMT.

Effects of early-life stress on probabilistic reversal learning and response perseverance in young adults

Early life stress (ELS), including experiences with abuse and neglect, are related to several negative health outcomes in adulthood. One area that has received attention is the increased rate of substance abuse disorder in individuals who had experienced ELS. Given the critical role habitual behavior in the development of substance abuse, ELS may affect the trajectory of neural development such that habitual responding is more dominant than in individuals who did not experience ELS. Here, we examine learning of a probabilistic classification task (the Weather Prediction Task) in healthy young adults who reported significant ELS and those that did not. This task can be learned in a declarative, model-based manner, or in a more habitual, stimulus-response manner. Participants learned to choose the outcome (sun or rain) that was probabilistically associated with each cue combination through reinforcement on each trial. After 100 trials, the probabilities were reversed, and we conceptualized habitual behavior as perseverating responses based on the old probabilities. We also collected information about subjective socio-economic status (sSES), anxiety, depression, and substance use from participants. Using multiple regression, we found that our measure of habitual responding was correlated with reported alcohol use, suggesting that our measure of habit has validity for health behaviors. Furthermore, we found that some forms of early life stress led to greater response perseverance after contingencies were reversed. Overall, the results suggest that childhood adversity may contribute to the development of habit.

Trait sensitivity to positive feedback is a predisposing factor for several aspects of compulsive alcohol drinking in male rats: behavioural, physiological, and molecular correlates

IntroductionAlcohol use disorder (AUD) is one of the most common psychiatric disorders and a leading cause of mortality worldwide. While the pathophysiology underlying AUD is relatively well known, the cognitive mechanisms of an individual’s susceptibility to the development of alcohol dependence remain poorly understood. In this study, we investigated the theoretical claim that sensitivity to positive feedback (PF), as a stable and enduring behavioural trait, can predict individual susceptibility to the acquisition and maintenance of alcohol-seeking behaviour in rats.MethodsTrait sensitivity to PF was assessed using a series of probabilistic reversal learning tests. The escalation of alcohol intake in rats was achieved by applying a mix of intermittent free access and instrumental paradigms of alcohol drinking. The next steps included testing the influence of sensitivity to PF on the acquisition of compulsive alcohol-seeking behaviour in the seeking-taking punishment task, measuring motivation to seek alcohol, and comparing the speed of extinction and reinstatement of alcohol-seeking after a period of abstinence between rats expressing trait insensitivity and sensitivity to PF. Finally, trait differences in the level of stress hormones and in the expression of genes and proteins in several brain regions of interest were measured to identify potential physiological and neuromolecular mechanisms of the observed interactions.ResultsWe showed that trait sensitivity to PF in rats determines the level of motivation to seek alcohol following the experience of its negative consequences. They also revealed significant differences between animals classified as insensitive and sensitive to PF in their propensity to reinstate alcohol-seeking behaviours after a period of forced abstinence. The abovementioned effects were accompanied by differences in blood levels of stress hormones and differences in the cortical and subcortical expression of genes and proteins related to dopaminergic, serotonergic, and GABAergic neurotransmission.ConclusionTrait sensitivity to PF can determine the trajectory of alcohol addiction in rats. This effect is, at least partially, mediated via distributed physiological and molecular changes within cortical and subcortical regions of the brain.

Effects of fentanyl self-administration on risk-taking behavior in male rats.

Individuals with opioid use disorder (OUD) exhibit impaired decisionmaking and elevated risk-taking behavior. In contrast to the effects of natural and semi-synthetic opioids, however, the impact of synthetic opioids on decisionmaking is still unknown. The objective of the current study was to determine how chronic exposure to the synthetic opioid fentanyl alters risk-based decisionmaking in adult male rats. Male rats underwent 14 days of intravenous fentanyl or oral sucrose self-administration. After 3 weeks of abstinence, rats were tested in a decision-making task in which they chose between a small, safe food reward and a large food reward accompanied by variable risk of footshock punishment. Following testing in the decision-making task, rats were tested in control assays that assessed willingness to work for food and shock reactivity. Lastly, rats were tested on a probabilistic reversal learning task to evaluate enduring effects of fentanyl on behavioral flexibility. Relative to rats in the sucrose group, rats in the fentanyl group displayed greater choice of the large, risky reward (risk taking), an effect that was present as long as 7 weeks into abstinence. This increased risktaking was driven by enhanced sensitivity to the large rewards and diminished sensitivity to punishment. The fentanyl-induced elevation in risktaking was not accompanied by alterations in food motivation or shock reactivity or impairments in behavioral flexibility. Results from the current study reveal that the synthetic opioid fentanyl leads to long-lasting increases in risktaking in male rats. Future experiments will extend this work to females and identify neural mechanisms that underlie these drug-induced changes in risktaking.

Importance of additional behavioral observation in psychopharmacology: a case study on agomelatine's effects on feedback sensitivity in probabilistic reversal learning in rats.

Since the second half of the twentieth century, many important discoveries in the field of behavioral psychopharmacology have been made using operant conditioning cages. These cages provide objective data collection and have revolutionized behavioral research. Unfortunately, in the rush towards automation, many mistakes may have been made that could have been avoided by observing experimental animals. The study described in this paper is an excellent example of how important additional behavioral observation can be for interpreting instrumental data. In this study, we evaluated the effects of single injections of 3 different doses of agomelatine (5, 10, and 40 mg/kg) on feedback sensitivity in rats. To this end, we tested 40 animals in the instrumental probabilistic reversal learning task in a Latin square design. The highest applied dose of agomelatine, prima facie, reduced the sensitivity of rats to negative feedback - an effect that can be considered antidepressant. However, additional behavioral observation dramatically changed the interpretation of the results and revealed that the perceived effect of agomelatine on sensitivity to negative feedback can actually be attributed to drug-induced drowsiness.