Aversive Learning Research Articles

Octopamine in the mushroom body circuitry for learning and memory.

Octopamine, the functional analog of noradrenaline, modulates many different behaviors and physiological processes in invertebrates. In the central nervous system, a few octopaminergic neurons project throughout the brain and innervate almost all neuropils. The center of memory formation in insects, the mushroom bodies, receive octopaminergic innervations in all insects investigated so far. Different octopamine receptors, either increasing or decreasing cAMP or calcium levels in the cell, are localized in Kenyon cells, further supporting the release of octopamine in the mushroom bodies. In addition, different mushroom body (MB) output neurons, projection neurons, and dopaminergic PAM cells are targets of octopaminergic neurons, enabling the modulation of learning circuits at different neural sites. For some years, the theory persisted that octopamine mediates rewarding stimuli, whereas dopamine (DA) represents aversive stimuli. This simple picture has been challenged by the finding that DA is required for both appetitive and aversive learning. Furthermore, octopamine is also involved in aversive learning and a rather complex interaction between these biogenic amines seems to modulate learning and memory. This review summarizes the role of octopamine in MB function, focusing on the anatomical principles and the role of the biogenic amine in learning and memory.

Breeding density affects the movements of gull chicks, the size of their home ranges and their association with neighbours.

Colonies of ground-nesting species often have heterogeneous nest densities and their offspring experience different social conditions depending on the size and location of the breeding territory. For example, unintentional territory crossing by mobile chicks can trigger strong aggression from neighbouring adults, as observed in semi-precocial gulls. This would be expected to shape chicks' movement tendencies, exploratory behaviour and propensity for social contact through aversive feedback learning or pre-natal maternal effects, as mothers may pre-adapt their offspring's behaviour to the expected early life conditions. Therefore, we hypothesize that lesser black-backed gull chicks reared in denser areas of the breeding colony will move less, have smaller home ranges and have fewer social contacts with chicks from neighbouring nests. To test this, we first cross-fostered full clutches between and within high- and low-density parts of the colony, and then used ultra-wideband tags to track free-ranging chicks. In line with our predictions, we found that chicks reared in denser areas had a lower movement activity and smaller home ranges. However, these chicks still had more social contacts, although not necessarily with a higher number of unique individuals. Pre-natal breeding density had no significant effect on any of the parameters. We conclude that parental nest choice strongly affects the early social environment of their chicks, which can shape the development of their (social) phenotype, with potentially long-lasting consequences.

The caloric value of food intake structurally adjusts a neuronal mushroom body circuit mediating olfactory learning in Drosophila.

Associative learning enables the adaptive adjustment of behavioral decisions based on acquired, predicted outcomes. The valence of what is learned is influenced not only by the learned stimuli and their temporal relations, but also by prior experiences and internal states. In this study, we used the fruit fly Drosophila melanogaster to demonstrate that neuronal circuits involved in associative olfactory learning undergo restructuring during extended periods of low-caloric food intake. Specifically, we observed a decrease in the connections between specific dopaminergic neurons (DANs) and Kenyon cells at distinct compartments of the mushroom body. This structural synaptic plasticity was contingent upon the presence of allatostatin A receptors in specific DANs and could be mimicked optogenetically by expressing a light-activated adenylate cyclase in exactly these DANs. Importantly, we found that this rearrangement in synaptic connections influenced aversive, punishment-induced olfactory learning but did not impact appetitive, reward-based learning. Whether induced by prolonged low-caloric conditions or optogenetic manipulation of cAMP levels, this synaptic rearrangement resulted in a reduction of aversive associative learning. Consequently, the balance between positive and negative reinforcing signals shifted, diminishing the ability to learn to avoid odor cues signaling negative outcomes. These results exemplify how a neuronal circuit required for learning and memory undergoes structural plasticity dependent on prior experiences of the nutritional value of food.

Formalizing Pavlovian Biases During Probabilistic Aversive Learning for Suicide Prevention

Formalizing Pavlovian Biases During Probabilistic Aversive Learning for Suicide Prevention

The selective serotonin reuptake inhibitor sertraline alters learning from aversive reinforcements in patients with depression: evidence from a randomized controlled trial.

Selective serotonin reuptake inhibitors (SSRIs) are first-line pharmacological treatments for depression and anxiety. However, little is known about how pharmacological action is related to cognitive and affective processes. Here, we examine whether specific reinforcement learning processes mediate the treatment effects of SSRIs. The PANDA trial was a multicentre, double-blind, randomized clinical trial in UK primary care comparing the SSRI sertraline with placebo for depression and anxiety. Participants (N = 655) performed an affective Go/NoGo task three times during the trial and computational models were used to infer reinforcement learning processes. There was poor task performance: only 54% of the task runs were informative, with more informative task runs in the placebo than in the active group. There was no evidence for the preregistered hypothesis that Pavlovian inhibition was affected by sertraline. Exploratory analyses revealed that in the sertraline group, early increases in Pavlovian inhibition were associated with improvements in depression after 12 weeks. Furthermore, sertraline increased how fast participants learned from losses and faster learning from losses was associated with more severe generalized anxiety symptoms. The study findings indicate a relationship between aversive reinforcement learning mechanisms and aspects of depression, anxiety, and SSRI treatment, but these relationships did not align with the initial hypotheses. Poor task performance limits the interpretability and likely generalizability of the findings, and highlights the critical importance of developing acceptable and reliable tasks for use in clinical studies. This article presents research supported by NIHR Program Grants for Applied Research (RP-PG-0610-10048), the NIHR BRC, and UCL, with additional support from IMPRS COMP2PSYCH (JM, QH) and a Wellcome Trust grant (QH).

An analysis of reinstatement after extinction of a conditioned taste aversion.

Taste aversion learning has sometimes been considered a specialized form of learning. In several other conditioning preparations, after a conditioned stimulus (CS) is conditioned and extinguished, reexposure to the unconditioned stimulus (US) by itself can reinstate the extinguished conditioned response. Reinstatement has been widely studied in fear and appetitive Pavlovian conditioning, as well as operant conditioning, but its status in taste aversion learning is more controversial. Six taste-aversion experiments with rats therefore sought to discover conditions that might encourage it there. The results often yielded little to no evidence of reinstatement, and we also found no evidence of concurrent recovery, a related phenomenon in which responding to a CS that has been conditioned and extinguished is restored if a second CS is separately conditioned. However, a key result was that reinstatement occurred when the conditioning procedure involved multiple closely spaced conditioning trials that could have allowed the animal to learn that a US presentation signaled or set the occasion for another trial with a US. Such a mechanism is precluded in many taste aversion experiments because they often use very few conditioning trials. Overall, the results suggest that taste aversion learning is experimentally unique, though not necessarily biologically or evolutionarily unique. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

An octopamine-specific GRAB sensor reveals a monoamine relay circuitry that boosts aversive learning.

Octopamine (OA), analogous to norepinephrine in vertebrates, is an essential monoamine neurotransmitter in invertebrates that plays a significant role in various biological functions, including olfactory associative learning. However, the spatial and temporal dynamics of OA in vivo remain poorly understood due to limitations associated with the currently available methods used to detect it. To overcome these limitations, we developed a genetically encoded GPCR activation-based (GRAB) OA sensor called GRABOA1.0. This sensor is highly selective for OA and exhibits a robust and rapid increase in fluorescence in response to extracellular OA. Using GRABOA1.0, we monitored OA release in the Drosophila mushroom body (MB), the fly's learning center, and found that OA is released in response to both odor and shock stimuli in an aversive learning model. This OA release requires acetylcholine (ACh) released from Kenyon cells, signaling via nicotinic ACh receptors. Finally, we discovered that OA amplifies aversive learning behavior by augmenting dopamine-mediated punishment signals via Octβ1R in dopaminergic neurons, leading to alterations in synaptic plasticity within the MB. Thus, our new GRABOA1.0 sensor can be used to monitor OA release in real time under physiological conditions, providing valuable insights into the cellular and circuit mechanisms that underlie OA signaling.

Auditory aversive generalization learning prompts threat-specific changes in alpha-band activity.

Pairing a neutral stimulus with aversive outcomes prompts neurophysiological and autonomic changes in response to the conditioned stimulus (CS+), compared to cues that signal safety (CS-). One of these changes-selective amplitude reduction of parietal alpha-band oscillations-has been reliably linked to processing of visual CS+. It is, however, unclear to what extent auditory conditioned cues prompt similar changes, how these changes evolve as learning progresses, and how alpha reduction in the auditory domain generalizes to similar stimuli. To address these questions, 55 participants listened to three sine wave tones, with either the highest or lowest pitch (CS+) being associated with a noxious white noise burst. A threat-specific (CS+) reduction in occipital-parietal alpha-band power was observed similar to changes expected for visual stimuli. No evidence for aversive generalization to the tone most similar to the CS+ was observed in terms of alpha-band power changes, aversiveness ratings, or pupil dilation. By-trial analyses found that selective alpha-band changes continued to increase as aversive conditioning continued, beyond when participants reported awareness of the contingencies. The results support a theoretical model in which selective alpha power represents a cross-modal index of continuous aversive learning, accompanied by sustained sensory discrimination of conditioned threat from safety cues.

Mouse Model of Chronic Social Stress-Induced Excessive Pavlovian Aversion Learning-Memory.

Increased experience of aversive stimuli/events is a psychological-neurobiological state of major importance in psychiatry. It occurs commonly in generalized anxiety disorder, post-traumatic stress disorder, and major depression. A sustained period of exposure to threat (chronic stressor) is a common risk factor, and a major symptom is generalized excessive perception of, and reactivity to, aversive stimuli. In rodents, Pavlovian aversion learning and memory (PAL, PAM), quantified in terms of the conditioned defensive behavior freezing, is an extensively studied behavioral paradigm, and well understood in terms of underlying neural circuitry. In mice, chronic social stress (CSS) is a 15-day resident-intruder paradigm in which C57BL/6 adult males are exposed continuously and distally to dominant-aggressive CD-1 male mice (sustained threat) interspersed with a brief daily period of proximal attack (acute threat). To ensure that physical wounding is minimized, proximal attacks are limited to 30 to 60 s/day and lower incisor teeth of CD-1 mice are blunted. Control (comparison) mice are maintained in littermate pairs. The CSS and CD-1 mice are maintained in distal contact during subsequent behavioral testing. For PAL, CSS and control (CON) mice are placed in a conditioning chamber (context) and exposed to a tone [conditioned stimulus (CS)] and mild, brief foot shock [unconditioned stimulus (US)]. For PAM, mice are placed in the same context and presented with CS repetitions. The CSS mice acquire (learn) and express (memory) a higher level of freezing than CON mice, indicating that CSS leads to generalized hypersensitivity to aversion, i.e., chronic social aversion leads to increased aversion salience of foot shock. Distinctive features of the model include the following: high reproducibility; rare, mild wounding only; male specificity; absence of "susceptible" vs "resilient" subgroups; behavioral effects dependent on continued presence of CD-1 mice; and preclinical validation of novel compounds for normalizing aversion hypersensitivity with accurate feedforward prediction of efficacy in human patients. © 2024 The Authors. Current Protocols published by Wiley Periodicals LLC. Basic Protocol 1: Chronic social stress (CSS) Basic Protocol 2: Pavlovian aversion learning and memory (PALM).

Characterizing the dynamic learning process: Implications of a quantitative analysis

Understanding the neural mechanisms involved in learning processes is crucial for unraveling the complexities of behavior and cognition. Sudden change from the untrained level to the fully-learned level is a pivotal feature of instrumental learning. However, the concept of change point and suitable methods to conveniently analyze the characteristics of sudden change in groups remain elusive, which might hinder a fuller understanding of the neural mechanism underlying dynamic leaning process. In the current study, we investigated the learning processes of mice that were trained in an aversive instrumental learning task, and introduced a novel strategy to analyze behavioral variations in instrumental learning, leading to improved clarity on the concept of sudden change and enabling comprehensive group analysis. By applying this novel strategy, we examined the effects of cocaine and a cannabinoid receptor agonist on instrumental learning. Intriguingly, our analysis revealed significant differences in timing and occurrence of sudden changes that were previously overlooked using traditional analysis. Overall, our research advances understanding of behavioral variation during instrumental learning and the interplay between learning behaviors and neurotransmitter systems, contributing to a deeper comprehension of learning processes and informing future investigations and therapeutic interventions.

Taste aversion learning during successive negative contrast

Previous experiments found that acceptance of saccharin by rats was reduced if they had prior experience of sucrose or some other highly palatable solution. This study tested whether such successive negative contrast (SNC) effects involve acquisition of an aversion to the new taste. In three experiments, rats were switched from sucrose exposure in Stage 1 to a less palatable solution containing a new taste in Stage 2. In Experiments 1 and 2, a novel flavor was added to a saccharin solution at the start of Stage 2. In Experiment 1, preference tests revealed a weak aversion to the added vanilla flavor in the Suc-Sacch group, while in Experiment 2 an aversion was found in the Suc-Sacch group to the salty flavor that was used, compared with controls given access either saccharin or water in Stage 1. In Experiment 3, the Suc-Quin group, given quinine solution in Stage 2, displayed a greater aversion to quinine than a Water-Quin control group. These results support the suggestion that taste aversion learning plays a role in the initial suppression of intakes in a qualitative consummatory SNC effect. However, in the light of other evidence, it seems that the unusual persistence of successive negative contrast when rats are switched from sucrose to saccharin is not due to a long-lasting reduction in the value of saccharin.

Nucleus accumbens neurons dynamically respond to appetitive and aversive associative learning.

To survive, individuals must learn to associate cues in the environment with emotionally relevant outcomes. This association is partially mediated by the nucleus accumbens (NAc), a key brain region of the reward circuit that is mainly composed by GABAergic medium spiny neurons (MSNs), that express either dopamine receptor D1 or D2. Recent studies showed that both populations can drive reward and aversion, however, the activity of these neurons during appetitive and aversive Pavlovian conditioning remains to be determined. Here, we investigated the relevance of D1- and D2-neurons in associative learning, by measuring calcium transients with fiber photometry during appetitive and aversive Pavlovian tasks in mice. Sucrose was used as a positive valence unconditioned stimulus (US) and foot shock was used as a negative valence US. We show that during appetitive Pavlovian conditioning, D1- and D2-neurons exhibit a general increase in activity in response to the conditioned stimuli (CS). Interestingly, D1- and D2-neurons present distinct changes in activity after sucrose consumption that dynamically evolve throughout learning. During the aversive Pavlovian conditioning, D1- and D2-neurons present an increase in the activity in response to the CS and to the US (shock). Our data support a model in which D1- and D2-neurons are concurrently activated during appetitive and aversive conditioning.

Frontal Norepinephrine Represents a Threat Prediction Error Under Uncertainty

BackgroundTo adapt to threats in the environment, animals must predict them and engage in defensive behavior. While the representation of a prediction error signal for reward has been linked to dopamine, a neuromodulatory prediction error for aversive learning has not been identified. MethodsWe measured and manipulated norepinephrine release during threat learning using optogenetics and a novel fluorescent norepinephrine sensor. ResultsWe found that norepinephrine response to conditioned stimuli reflects aversive memory strength. When delays between auditory stimuli and footshock are introduced, norepinephrine acts as a prediction error signal. However, temporal difference prediction errors do not fully explain norepinephrine dynamics. To explain noradrenergic signaling, we used an updated reinforcement learning model with uncertainty about time and found that it explained norepinephrine dynamics across learning and variations in temporal and auditory task structure. ConclusionsNorepinephrine thus combines cognitive and affective information into a predictive signal and links time with the anticipation of danger.

Reduction of Aversive Learning Rates in Pavlovian Conditioning by Angiotensin II Antagonist Losartan: A Randomized Controlled Trial

BackgroundAngiotensin receptor blockade has been linked to aspects of aversive learning and memory formation and to the prevention of posttraumatic stress disorder symptom development. MethodsWe investigated the influence of the angiotensin receptor blocker losartan on aversive Pavlovian conditioning using a probabilistic learning paradigm. In a double-blind, randomized, placebo-controlled design, we tested 45 (18 female) healthy volunteers during a baseline session, after application of losartan or placebo (drug session), and during a follow-up session. During each session, participants engaged in a task in which they had to predict the probability of an electrical stimulation on every trial while the true shock contingencies switched repeatedly between phases of high and low shock threat. Computational reinforcement learning models were used to investigate learning dynamics. ResultsAcute administration of losartan significantly reduced participants’ adjustment during both low-to-high and high-to-low threat changes. This was driven by reduced aversive learning rates in the losartan group during the drug session compared with baseline. The 50-mg drug dose did not induce reduction of blood pressure or change in reaction times, ruling out a general reduction in attention and engagement. Decreased adjustment of aversive expectations was maintained at a follow-up session 24 hours later. ConclusionsThis study shows that losartan acutely reduces Pavlovian learning in aversive environments, thereby highlighting a potential role of the renin-angiotensin system in anxiety development.

A dedicated hypothalamic oxytocin circuit controls aversive social learning.

To survive in a complex social group, one needs to know who to approach and, more importantly, who to avoid. In mice, a single defeat causes the losing mouse to stay away from the winner for weeks1. Here through a series of functional manipulation and recording experiments, we identify oxytocin neurons in the retrochiasmatic supraoptic nucleus (SOROXT) and oxytocin-receptor-expressing cells in the anterior subdivision of the ventromedial hypothalamus, ventrolateral part (aVMHvlOXTR) as a key circuit motif for defeat-induced social avoidance. Before defeat, aVMHvlOXTR cells minimally respond to aggressor cues. During defeat, aVMHvlOXTR cells are highly activated and, with the help of an exclusive oxytocin supply from the SOR, potentiate their responses to aggressor cues. After defeat, strong aggressor-induced aVMHvlOXTR cell activation drives the animal to avoid the aggressor and minimizes future defeat. Our study uncovers a neural process that supports rapid social learning caused by defeat and highlights the importance of the brain oxytocin system in social plasticity.

Consensus and Risk Aversion Learning in Ensemble of Multiple Experts for Long-Tailed Classification

Consensus and Risk Aversion Learning in Ensemble of Multiple Experts for Long-Tailed Classification

Patterns of ingestion of rats during chronic oral administration of lithium chloride

Chronic lithium administration to rodents is used to explore the potential neural mechanisms of mood stabilization, as well as to model the side effects of chronic lithium on multiple organ systems. Oral administration of lithium in the maintenance diet or drinking water is convenient, but lithium can acutely affect intake and it can mediate acquisition of conditioned taste aversions (CTA). We compared ad libitum food and fluid intake by male rats with LiCl or NaCl solutions as their sole source of fluid across 20 days, with a commonly used dosage of LiCl (24 mM: 1 g / L LiCl). To quantify the pattern of intake, rats were housed in cages equipped with lickometers to detect licks and infrared photobeams to detect food access with 6-s resolution. To determine if rats formed a CTA to LiCl, they were subsequently tested with access to NaCl. Rats showed an immediate avoidance of the LiCl solution, as seen on the first day of access by an increased latency to initiate drinking and a decreased size of drinking bouts. Rats showed a differential response to LiCl vs. NaCl after as few as 5 licks. Chronic consumption of LiCl solution led to significantly decreased food and fluid intake compared to baseline, with concomitant weight loss. The decreased intake was realized by marked changes in the pattern of drinking and feeding bouts: a decrease in per-lick volume and a decrease in licks per drinking bout, and an increase in feeding bout duration resulting in an overall decrease in eating rate. Conversely, chronic NaCl access led to an increase in drinking bout number and licks/bout. The avoidance of LiCl was likely a combination of toxic effects of ingested LiCl and rapid acquisition of a learned aversion to the taste of LiCl, as shown by an extinguishable generalized aversion to NaCl solution during subsequent NaCl test days. The marked effect of chronic oral LiCl on ingestion may impact the oral dosing of lithium as well as the rat's metabolic status.

Glia transmit negative valence information during aversive learning in Drosophila.

During Drosophila aversive olfactory conditioning, aversive shock information needs to be transmitted to the mushroom bodies (MBs) to associate with odor information. We report that aversive information is transmitted by ensheathing glia (EG) that surround the MBs. Shock induces vesicular exocytosis of glutamate from EG. Blocking exocytosis impairs aversive learning, whereas activation of EG can replace aversive stimuli during conditioning. Glutamate released from EG binds to N-methyl-d-aspartate receptors in the MBs, but because of Mg2+ block, Ca2+ influx occurs only when flies are simultaneously exposed to an odor. Vesicular exocytosis from EG also induces shock-associated dopamine release, which plays a role in preventing formation of inappropriate associations. These results demonstrate that vesicular glutamate released from EG transmits negative valence information required for associative learning.

Aversive conditioning is impaired in impulsive individuals: A study on learning asymmetries

Background and objectivesAppetitive and aversive conditioning are thought to be involved in the development and maintenance of mental disorders including anxiety, mood, eating, and substance use disorders. However, few studies measure the relative strength of appetitive and aversive associations, and their relevance to the risk of mental disorders. This study aims to address this gap. MethodsWe tested how readily healthy volunteers acquire appetitive vs. aversive associations. 150 participants associated complex 3D objects with either gain or loss and made decisions to gain or avoid losing points. We investigated the relationship of a learning asymmetry with neuroticism, impulsivity, and anhedonia, to test the hypothesis that a stronger learning asymmetry corresponds to more extreme scores on these traits. ResultsImpulsivity was positively associated with the learning asymmetry (R2 = .10). This resulted from an inverse relation with the strength of aversive associations, indicating that impulsive individuals are worse at aversive learning. However, appetitive associations did not differ significantly. No correlations with neuroticism or anhedonia were found. LimitationsConditioning studies typically use primary reinforcers and a CS-. Lacking these may make these results less comparable to other studies. ConclusionsWe demonstrate that the learning asymmetry can measure individual differences linked to personality traits, and that impulsivity, normally linked with appetitive learning, also influences aversive learning. These results enable additional studies of learning asymmetry in relation to mental disorders, which could include measurements of mental health symptoms to provide further insight into how appetitive and aversive learning interacts with mental disorders.

Social Threat-Safety Test Uncovers Psychosocial Stress-Related Phenotypes.

Social stress is a major cause of the development of mental disorders. To enhance the translational value of preclinical studies, social stress experience and its behavioral impact on mice should be comparable to humans. Chronic social defeat (CSD) utilizes a type of social stress involving physical attacks and sensory threats to induce mental dysfunctions resembling human affective disorders. To strengthen the psychosocial component of CSD, a 10-day CSD protocol was applied in which daily physical attacks are standardized to three 10 s episodes followed by a 24 h sensory phase. After the 10th sensory phase, the CSD protocol is followed by a refined behavioral assay called the social threat-safety test (STST). Post-stress behavioral assays need to determine how and to what extent the social stressor has influenced behavior. The STST allows chronically socially defeated male mice to interact with 2 novel male individuals (social targets): one social target from the attacking strain encountered during the CSD days and the other from a novel strain. Both are presented simultaneously in different compartments of a three-chambered test arena. The test enables a simultaneous assessment of social avoidance development to measure successful aversive conditioned learning and social threat-safety discrimination ability. The development of social avoidance towards both strains reflects a generalized aversive response and thus, a measurement of stress susceptibility. Meanwhile, the development of social avoidance towards only the attacking strain reflects threat-safety discrimination and thus, a measurement of stress resilience. Finally, the absence of social avoidance towards the attacking strain reflects impaired aversive conditioned learning. The protocol aims to refine the currently used mouse models of stress susceptibility/resilience by including translational criteria, specifically threat-safety discrimination and aversive response generalization, to categorize a single group of chronically socially defeated animals into resilient and susceptible subgroups, eventually advancing future translational approaches.