Reward Learning Research Articles

Early-Life Adversities Are Associated With Lower Expected Value Signaling in the Adult Brain

BackgroundEarly adverse experiences are assumed to affect fundamental processes of reward learning and decision making. However, computational neuroimaging studies investigating these circuits in the context of adversity are sparse and limited to studies conducted in adolescent samples, leaving the long-term effects unexplored. MethodsUsing data from a longitudinal birth cohort study (n = 156; 87 female), we investigated associations between adversities and computational markers of reward learning (i.e., expected value, prediction errors). At age 33 years, all participants completed a functional magnetic resonance imaging–based passive avoidance task. Psychopathology measures were collected at the time of functional magnetic resonance imaging investigation and during the COVID-19 pandemic. We applied a principal component analysis to capture common variations across 7 adversity measures. The resulting adversity factors (factor 1: postnatal psychosocial adversities and prenatal maternal smoking; factor 2: prenatal maternal stress and obstetric adversity; factor 3: lower maternal stimulation) were linked with psychopathology and neural responses in the core reward network using multiple regression analysis. ResultsWe found that the adversity dimension primarily informed by lower maternal stimulation was linked to lower expected value representation in the right putamen, right nucleus accumbens, and anterior cingulate cortex. Expected value encoding in the right nucleus accumbens further mediated the relationship between this adversity dimension and psychopathology and predicted higher withdrawn symptoms during the COVID-19 pandemic. ConclusionsOur results suggested that early adverse experiences in caregiver context might have a long-term disruptive effect on reward learning in reward-related brain regions, which can be associated with suboptimal decision making and thereby may increase the vulnerability of developing psychopathology.

Characterisation of behaviours relevant to apathy syndrome in the aged male rat

Apathy is a complex psychiatric syndrome characterised by motivational deficit, emotional blunting and cognitive changes. It occurs alongside a broad range of neurological disorders, but also occurs in otherwise healthy ageing. Despite its clinical prevalence, apathy does not yet have a designated treatment strategy. Generation of a translational animal model of apathy syndrome would facilitate the development of novel treatments. Given the multidimensional nature of apathy, a model cannot be achieved with a single behavioural test. Using a battery of behavioural tests we investigated whether aged rats exhibit behavioural deficits across different domains relevant to apathy. Using the effort for reward and progressive ratio tasks we found that aged male rats (21–27 months) show intact reward motivation. Using the novelty supressed feeding test and position-based object exploration we found aged rats showed increased anxiety-like behaviour inconsistent with emotional blunting. The sucrose preference test and reward learning assay showed intact reward sensitivity and reward-related cognition in aged rats. However, using a bowl-digging version of the probabilistic reversal learning task, we found a deficit in cognitive flexibility in aged rats that did not translate across to a touchscreen version of the task. While these data reveal important changes in cognitive flexibility and anxiety associated with ageing, aged rats do not show deficits across other behavioural domains relevant to apathy. This suggests that aged rats are not a suitable model for age-related apathy syndrome. These findings contrast with previous work in mice, revealing important species differences in behaviours relevant to apathy syndrome in ageing.

Mild exogenous inflammation blunts neural signatures of bounded evidence accumulation and reward prediction error processing in healthy male participants

BackgroundAltered neural haemodynamic activity during decision making and learning has been linked to the effects of inflammation on mood and motivated behaviours. So far, it has been reported that blunted mesolimbic dopamine reward signals are associated with inflammation-induced anhedonia and apathy. Nonetheless, it is still unclear whether inflammation impacts neural activity underpinning decision dynamics. The process of decision making involves integration of noisy evidence from the environment until a critical threshold of evidence is reached. There is growing empirical evidence that such process, which is usually referred to as bounded accumulation of decision evidence, is affected in the context of mental illness. MethodsIn a randomised, placebo-controlled, crossover study, 19 healthy male participants were allocated to placebo and typhoid vaccination. Three to four hours post-injection, participants performed a probabilistic reversal-learning task during functional magnetic resonance imaging. To capture the hidden neurocognitive operations underpinning decision-making, we devised a hybrid sequential sampling and reinforcement learning computational model. We conducted whole brain analyses informed by the modelling results to investigate the effects of inflammation on the efficiency of decision dynamics and reward learning. ResultsWe found that during the decision phase of the task, typhoid vaccination attenuated neural signatures of bounded evidence accumulation in the dorsomedial prefrontal cortex, only for decisions requiring short integration time. Consistent with prior work, we showed that, in the outcome phase, mild acute inflammation blunted the reward prediction error in the bilateral ventral striatum and amygdala. ConclusionsOur study extends current insights into the effects of inflammation on the neural mechanisms of decision making and shows that exogenous inflammation alters neural activity indexing efficiency of evidence integration, as a function of choice discriminability. Moreover, we replicate previous findings that inflammation blunts striatal reward prediction error signals.

Recent Advancements in Inverse Reinforcement Learning

Inverse reinforcement learning (IRL) has seen significant advancements in recent years. This class of approaches aims to efficiently learn the underlying reward function that rationalizes the behavior exhibited by expert agents, often represented by humans. In contrast to mere behavioral cloning, the reconstruction of a reward function yields appealing implications, as it allows for more effective interpretability of the expert’s decisions and provides a transferable specification of the expert’s objectives for application in even different environments. Unlike the well-understood field of reinforcement learning (RL) from a theoretical perspective, IRL still grapples with limited understanding, significantly constraining its applicability. A fundamental challenge in IRL is the inherent ambiguity in selecting a reward function, given the existence of multiple candidate functions, all explaining the expert’s behavior. In this talk, I will survey three of my papers that have made notable contributions to the IRL field: “Provably Efficient Learning of Transferable Rewards”, “Towards Theoretical Understanding of Inverse Reinforcement Learning”, and “Inverse Reinforcement Learning with Sub-optimal Experts". The central innovation introduced by the first paper is a novel formulation of the IRL problem that overcomes the issue of ambiguity. IRL is reframed as the problem of learning the feasible reward set, which is the set of all rewards that can explain the expert’s behavior. This approach postpones the selection of the reward function, thereby circumventing the ambiguity issues. Furthermore, the feasible reward set exhibits convenient geometric properties that enable the development of efficient algorithms for its computation. Building on this novel formulation of IRL, the second paper addresses the problem of efficiently learning the feasible reward set when the environment and the expert’s policy are not known in advance. It introduces a novel way to assess the dissimilarity between feasible reward sets based on the Hausdorff distance and presents a new PAC (probabilistic approximately correct) framework. The most significant contribution of this paper is the introduction of the first sample complexity lower bound, which highlights the challenges inherent in the IRL problem. Deriving this lower bound necessitated the development of novel technical tools. The paper also demonstrates that when a generative model of the environment is available, a uniform sampling strategy achieves a sample complexity that matches the lower bound, up to logarithmic factors. Finally, in the third paper, the IRL problem in the presence of sub-optimal experts is investigated. Specifically, the paper assumes the availability of multiple sub-optimal experts, in addition to the expert agent, which provides additional demonstrations, associated with a known quantification of the maximum amount of sub-optimality. The paper shows that this richer information mitigates the ambiguity problem, significantly reducing the size of the feasible reward set while retaining its favorable geometric properties. Furthermore, the paper explores the associated statistical problem and derives novel lower bounds for sample complexity, along with almost matching algorithms. These selected papers represent notable advancements in IRL, contributing to the establishment of a solid theoretical foundation for IRL and extending the framework to accommodate scenarios with sub-optimal experts.

Lunar Rover Collaborated Path Planning with Artificial Potential Field-Based Heuristic on Deep Reinforcement Learning

The International Lunar Research Station, to be established around 2030, will equip lunar rovers with robotic arms as constructors. Construction requires lunar soil and lunar rovers, for which rovers must go toward different waypoints without encountering obstacles in a limited time due to the short day, especially near the south pole. Traditional planning methods, such as uploading instructions from the ground, can hardly handle many rovers moving on the moon simultaneously with high efficiency. Therefore, we propose a new collaborative path-planning method based on deep reinforcement learning, where the heuristics are demonstrated by both the target and the obstacles in the artificial potential field. Environments have been randomly generated where small and large obstacles and different waypoints are created to collect resources, train the deep reinforcement learning agent to propose actions, and lead the rovers to move without obstacles, finish rovers’ tasks, and reach different targets. The artificial potential field created by obstacles and other rovers in every step affects the action choice of the rover. Information from the artificial potential field would be transformed into rewards in deep reinforcement learning that helps keep distance and safety. Experiments demonstrate that our method can guide rovers moving more safely without turning into nearby large obstacles or collision with other rovers as well as consuming less energy compared with the multi-agent A-Star path-planning algorithm with improved obstacle avoidance method.

DiffAIL: Diffusion Adversarial Imitation Learning

Imitation learning aims to solve the problem of defining reward functions in real-world decision-making tasks. The current popular approach is the Adversarial Imitation Learning (AIL) framework, which matches expert state-action occupancy measures to obtain a surrogate reward for forward reinforcement learning. However, the traditional discriminator is a simple binary classifier and doesn't learn an accurate distribution, which may result in failing to identify expert-level state-action pairs induced by the policy interacting with the environment. To address this issue, we propose a method named diffusion adversarial imitation learning (DiffAIL), which introduces the diffusion model into the AIL framework. Specifically, DiffAIL models the state-action pairs as unconditional diffusion models and uses diffusion loss as part of the discriminator's learning objective, which enables the discriminator to capture better expert demonstrations and improve generalization. Experimentally, the results show that our method achieves state-of-the-art performance and significantly surpasses expert demonstration on two benchmark tasks, including the standard state-action setting and state-only settings.

Balance Reward and Safety Optimization for Safe Reinforcement Learning: A Perspective of Gradient Manipulation

Ensuring the safety of Reinforcement Learning (RL) is crucial for its deployment in real-world applications. Nevertheless, managing the trade-off between reward and safety during exploration presents a significant challenge. Improving reward performance through policy adjustments may adversely affect safety performance. In this study, we aim to address this conflicting relation by leveraging the theory of gradient manipulation. Initially, we analyze the conflict between reward and safety gradients. Subsequently, we tackle the balance between reward and safety optimization by proposing a soft switching policy optimization method, for which we provide convergence analysis. Based on our theoretical examination, we provide a safe RL framework to overcome the aforementioned challenge, and we develop a Safety-MuJoCo Benchmark to assess the performance of safe RL algorithms. Finally, we evaluate the effectiveness of our method on the Safety-MuJoCo Benchmark and a popular safe benchmark, Omnisafe. Experimental results demonstrate that our algorithms outperform several state-of-the-art baselines in terms of balancing reward and safety optimization.

Learning Optimal Advantage from Preferences and Mistaking It for Reward

We consider algorithms for learning reward functions from human preferences over pairs of trajectory segments, as used in reinforcement learning from human feedback (RLHF). Most recent work assumes that human preferences are generated based only upon the reward accrued within those segments, or their partial return. Recent work casts doubt on the validity of this assumption, proposing an alternative preference model based upon regret. We investigate the consequences of assuming preferences are based upon partial return when they actually arise from regret. We argue that the learned function is an approximation of the optimal advantage function, not a reward function. We find that if a specific pitfall is addressed, this incorrect assumption is not particularly harmful, resulting in a highly shaped reward function. Nonetheless, this incorrect usage of the approximation of the optimal advantage function is less desirable than the appropriate and simpler approach of greedy maximization of it. From the perspective of the regret preference model, we also provide a clearer interpretation of fine tuning contemporary large language models with RLHF. This paper overall provides insight regarding why learning under the partial return preference model tends to work so well in practice, despite it conforming poorly to how humans give preferences.

Basolateral amygdala parvalbumin interneurons coordinate oscillations to drive reward behaviors

The basolateral amygdala (BLA) mediates both fear and reward learning.1,2 Previous work has shown that parvalbumin (PV) interneurons in the BLA contribute to BLA oscillatory states integral to fear expression.3,4,5,6,7 However, despite it being critical to our understanding of reward behaviors, it is unknown whether BLA oscillatory states and PV interneurons similarly contribute to reward processing. Local field potentials in the BLA were collected as male and female mice consumed sucrose reward, where prominent changes in the beta band (15-30Hz) emerged with reward experience. During consumption of one water bottle during a two-water-bottle choice test, rhythmic optogenetic stimulation of BLA PVs produced a robust bottle preference, showing that PVs can sufficiently drive reward seeking. Finally, to demonstrate that PV activity is necessary for reward value use, PVs were chemogenetically inhibited following outcome devaluation, rendering mice incapable of using updated reward representations to guide their behavior. Taken together, these experiments provide novel information about the physiological signatures of reward while highlighting BLA PV interneuron contributions to behaviors that are BLA dependent. This work builds upon established knowledge of PV involvement in fear expression and provides evidence that PV orchestration of unique BLA network states is involved in both learning types.

Task demand modulates the effects of reward learning on emotional stimuli

Task demand modulates the effects of reward learning on emotional stimuli

Neural correlates of the addictions neuroclinical assessment (ANA) incentive salience factor among individuals with alcohol use disorder

The Addictions Neuroclinical Assessment (ANA) is a recently-developed framework offering a more holistic understanding of three neurofunctional and behavioral domains that reflect the neurobiological dysfunction seen in alcohol use disorder (AUD). While the ANA domains have been well-validated across independent laboratories, there is a critical need to identify neural markers that subserve the proposed neurofunctional domains. The current study involves secondary data analysis of a two-week experimental medication trial of ibudilast (50 mg BID). Forty-five non-treatment-seeking participants with AUD (17F / 28 M) completed a battery of validated behavioral assessments forming the basis of their incentive salience factor score, computed via factor analysis, as well as a functional neuroimaging (fMRI) task assessing their neural reactivity to visual alcohol cues after being on placebo or ibudilast for 7 days. General linear models were conducted to examine the relationship between incentive salience and neural alcohol cue-reactivity in the ventral and dorsal stratum. Whole-brain generalized linear model analyses were conducted to examine associations between neural alcohol cue-reactivity and incentive salience. Age, sex, medication, and smoking status were included as covariates. Incentive salience was not associated with cue-elicited activation in the dorsal or ventral striatum. Incentive salience was significantly positively correlated (p < 0.05) with alcohol cue-elicited brain activation in reward-learning and affective regions including the insula and posterior cingulate cortices, bilateral precuneus, and bilateral precentral gyri. The ANA incentive salience factor is reflected in brain circuitry important for reward learning and emotion processing. Identifying a sub-phenotype of AUD characterized by increased incentive salience to alcohol cues allows for precision medicine approaches, i.e. treatments specifically targeting craving and reward from alcohol use. This study serves as a preliminary bio-behavioral validation for the incentive salience factor of the ANA. Further studies validating the neural correlates of other ANA factors, as well as replication in larger samples, appear warranted.

ReptiLearn: An automated home cage system for behavioral experiments in reptiles without human intervention.

Understanding behavior and its evolutionary underpinnings is crucial for unraveling the complexities of brain function. Traditional approaches strive to reduce behavioral complexity by designing short-term, highly constrained behavioral tasks with dichotomous choices in which animals respond to defined external perturbation. In contrast, natural behaviors evolve over multiple time scales during which actions are selected through bidirectional interactions with the environment and without human intervention. Recent technological advancements have opened up new possibilities for experimental designs that more closely mirror natural behaviors by replacing stringent experimental control with accurate multidimensional behavioral analysis. However, these approaches have been tailored to fit only a small number of species. This specificity limits the experimental opportunities offered by species diversity. Further, it hampers comparative analyses that are essential for extracting overarching behavioral principles and for examining behavior from an evolutionary perspective. To address this limitation, we developed ReptiLearn-a versatile, low-cost, Python-based solution, optimized for conducting automated long-term experiments in the home cage of reptiles, without human intervention. In addition, this system offers unique features such as precise temperature measurement and control, live prey reward dispensers, engagement with touch screens, and remote control through a user-friendly web interface. Finally, ReptiLearn incorporates low-latency closed-loop feedback allowing bidirectional interactions between animals and their environments. Thus, ReptiLearn provides a comprehensive solution for researchers studying behavior in ectotherms and beyond, bridging the gap between constrained laboratory settings and natural behavior in nonconventional model systems. We demonstrate the capabilities of ReptiLearn by automatically training the lizard Pogona vitticeps on a complex spatial learning task requiring association learning, displaced reward learning, and reversal learning.

생성형 인공지능 챗GPT의 활용과 규제에 대한 법제검토

With the advent of ChatGPT, the boundary between humans and machines has disappeared, making it increasingly difficult for people to discern whether the person they are dealing with is a machine or a human. Nevertheless, ChatGPT is bringing about changes in the way we work, communicate, and play in the digital world by bringing about a shift in information search and access methods and creating products such as books and art.&#x0D; ChatGPT optimizes conversations using Reinforcement Learning from Human Feedback (RLHF). Reinforcement Learning is a type of machine learning technique that learns how to perform a task through repeated trial and error interactions. A person scores ChatGPT's responses and reflects them in the reinforcement learning rewards algorithm to learn ChatGPT's reward model. It is learned that the more a human user judges a given query to be a good response, the greater the reward, and the more harmful text is generated, the less reward it receives. It is thanks to this RLHF technology that when talking with ChatGPT, you feel like you are talking with a logical and fluent person rather than a machine.&#x0D; ChatGPT will enable the commercialization of a generative AI service that enables interactive conversation rather than focusing on one-sided commands, and since it is a very large-scale model, versatility is its core competitiveness. In other words, ChatGPT allows you to exchange conversations and finds or converts results such as writing, composing, coding, or drawing with a single text input. As ChatGPT significantly lowers the threshold of time and limitations associated with content production, it could gain attention in a variety of fields including advertising, publishing, and gaming. Additionally, ChatGPT has great potential for growth in the field of education as it can unravel and explain complex concepts and correct errors through conversation.&#x0D; This study reviewed the technological development process and general-purpose use of generative artificial intelligence (ChatGPT) and the legal regulations of ChatGPT, focusing on the legal issues surrounding the emergence of generative artificial intelligence (ChatGPT) in the AI era.

Maternal choline supplementation lessens the behavioral dysfunction produced by developmental manganese exposure in a rodent model of ADHD

Studies in children have reported associations between elevated manganese (Mn) exposure and ADHD-related symptoms of inattention, impulsivity/hyperactivity, and psychomotor impairment. Maternal choline supplementation (MCS) during pregnancy/lactation may hold promise as a protective strategy because it has been shown to lessen cognitive dysfunction caused by numerous early insults. Our objectives were to determine whether (1) developmental Mn exposure alters behavioral reactivity/emotion regulation, in addition to impairing learning, attention, impulse control, and sensorimotor function, and (2) MCS protects against these Mn-induced impairments. Pregnant Long-Evans rats were given standard diet, or a diet supplemented with additional choline throughout gestation and lactation (GD 3 - PND 21). Male offspring were exposed orally to 0 or 50 mg Mn/kg/day over PND 1–21. In adulthood, animals were tested in a series of learning, attention, impulse control, and sensorimotor tasks. Mn exposure caused lasting dysfunction in attention, reactivity to errors and reward omission, learning, and sensorimotor function, recapitulating the constellation of symptoms seen in ADHD children. MCS lessened Mn-induced attentional dysfunction and partially normalized reactivity to committing an error or not receiving an expected reward but provided no protection against Mn-induced learning or sensorimotor dysfunction. In the absence of Mn exposure, MCS produces lasting offspring benefits in learning, attention, and reactivity to errors. To conclude, developmental Mn exposure produces a constellation of deficits consistent with ADHD symptomology, and MCS offered some protection against the adverse Mn effects, adding to the evidence that maternal choline supplementation is neuroprotective for offspring and improves offspring cognitive functioning.

Ghrelin decreases sensitivity to negative feedback and increases prediction-error related caudate activity in humans, a randomized controlled trial

The stomach-derived hormone ghrelin plays not only a role in feeding, starvation, and survival, but it has been suggested to also be involved in the stress response, in neuropsychiatric conditions, and in alcohol and drug use disorders. Mechanisms related to reward processing might mediate ghrelin’s broader effects on complex behaviors, as indicated by animal studies and mostly correlative human studies. Here, using a within-subject double-blind placebo-controlled design with intravenous ghrelin infusion in healthy volunteers (n = 30), we tested whether ghrelin alters sensitivity to reward and punishment in a reward learning task. Parameters were derived from a computational model of participants’ task behavior. The reversal learning task with monetary rewards was performed during functional brain imaging to investigate ghrelin effects on brain signals related to reward prediction errors. Compared to placebo, ghrelin decreased punishment sensitivity (t = −2.448, p = 0.021), while reward sensitivity was unaltered (t = 0.8, p = 0.43). We furthermore found increased prediction-error related activity in the dorsal striatum during ghrelin administration (region of interest analysis: t-values ≥ 4.21, p-values ≤ 0.044). Our results support a role for ghrelin in reward processing that extends beyond food-related rewards. Reduced sensitivity to negative outcomes and increased processing of prediction errors may be beneficial for food foraging when hungry but could also relate to increased risk taking and impulsivity in the broader context of addictive behaviors.

The effect of noninstrumental information on reward learning

Investigations of information-seeking often highlight people’s tendency to forgo financial reward in return for advance information about future outcomes. Most of these experiments use tasks in which reward contingencies are described to participants. The use of such descriptions leaves open the question of whether the opportunity to obtain such noninstrumental information influences people’s ability to learn and represent the underlying reward structure of an experimental environment. In two experiments, participants completed a two-armed bandit task with monetary incentives where reward contingencies were learned via trial-by-trial experience. We find, akin to description-based tasks, that participants are willing to forgo financial reward to receive information about a delayed, unchangeable outcome. Crucially, however, there is little evidence this willingness to pay for information is driven by an inaccurate representation of the reward structure: participants’ representations approximated the underlying reward structure regardless of the presence of advance noninstrumental information. The results extend previous conclusions regarding the intrinsic value of information to an experience-based domain and highlight challenges of probing participants’ memories for experienced rewards.

Foxp2 Is Required for Nucleus Accumbens-mediated Multifaceted Limbic Function

The forkhead box protein P2 (Foxp2), initially identified for its role in speech and language development, plays an important role in neural development. Previous studies investigated the function of the Foxp2 gene by deleting or mutating Foxp2 from developmental stages. Little is known about its physiological function in adult brains. Although Foxp2 has been well studied in the dorsal striatum, its function in the nucleus accumbens (NAc) of the ventral striatum remains elusive. Here, we examine the physiological function of Foxp2 in NAc of mouse brains. We conditionally knocked out Foxp2 by microinjections of AAV-EGFP-Cre viruses into the medial shell of NAc of Foxp2 floxed (cKO) mice. Immunostaining showed increased c-Fos positive cells in cKO NAc at basal levels, suggesting an abnormality in Foxp2-deficient NAc cells. Unbiased behavioral profiling of Foxp2 cKO mice showed abnormalities in limbic-associated function. Foxp2 cKO mice exhibited abnormal social novelty without preference for interaction with strangers and familiar mice. In appetitive reward learning, Foxp2 cKO mice failed to learn the time expectancy of food delivery. In fear learning, Foxp2 cKO mice exhibited abnormal increases in freezing levels in response to tone paired with foot shock during fear conditioning. The extinction of the fear response was also altered in Foxp2 cKO mice. In contrast, conditional knockout of Foxp2 in NAc did not affect locomotion, motor coordination, thermal pain sensation, anxiety- and depression-like behaviors. Collectively, our study suggests that Foxp2 has a multifaceted physiological role in NAc in the regulation of limbic function in the adult brain.

The effect of students' effort-reward imbalance on learning engagement: the mediating role of learned helplessness and the moderating role of social support.

To explore the predictive effect of effort-reward imbalance on students' learning engagement and to elucidate the underlying mechanism, 796 students were selected for a survey. The participants were required to complete four scales: the Effort-reward Imbalance Scale, the Learning Engagement Scale, the Learned Helplessness Questionnaire, and the Perceived Social Support Scale. (1) Students' effort-reward imbalance significantly and negatively predicts their learning engagement; (2) Learned helplessness serves as a mediator in the relationship between students' effort-reward imbalance and learning engagement; (3) Social support plays a moderating role in the association between effort-reward imbalance and learned helplessness. High levels of social support can buffer the impact of an effort-reward imbalance on learned helplessness, and the protective effect of social support is more obvious when the effort-reward imbalance is low. The present study revealed how an effort-reward imbalance affects learning engagement among students through the dimensions of learned helplessness and perceived social support. The constructed model not only further clarifies the mechanism underlying the relationship between effort-reward imbalance and learning engagement but also holds significant implications for guiding students' education.

Stressful Life Events and Reward Processing in Adults: Moderation by Depression and Anhedonia

Background. Exposure to acute stress is associated with reduced reward processing in laboratory studies in animals and humans. However, less clear is the association between reward processing and exposure to naturalistic stressful life events. The goal of the current study was to provide a novel investigation of the relation between past 6-month stressful life events and reward processing, and the extent to which this relation was moderated by depression diagnostic status and state symptoms of anhedonia. Methods. The current study included a secondary analysis of data from 107 adults (37 current-depressed, 25 past-depressed, 45 never-depressed; 75% women) drawn from two previous community studies. Past 6-month stressful life events were assessed with a rigorous contextual interview with independent ratings. Response to monetary reward was assessed with a probabilistic reward task. Results. Among current-depressed participants, and among both current- and past-depressed participants with high levels of anhedonia, greater exposure to independent life events outside of individuals’ control was significantly associated with poorer reward learning. In direct contrast, among those with low levels of anhedonia, greater exposure to independent life events was significantly associated with a greater overall bias toward the more frequently rewarded stimulus. Conclusions. Results suggest that depression and anhedonia are uniquely associated with vulnerability to blunted reward learning in the face of uncontrollable stressors. In contrast, in the absence of anhedonia symptoms, heightened reward processing during or following independent stressful life event exposure may represent an adaptive response.

IR76b-expressing neurons in Drosophila melanogaster are necessary for associative reward learning of an amino acid mixture.

Learning where to find nutrients while at the same time avoiding toxic food is essential for survival of any animal. Using Drosophila melanogaster larvae as a study case, we investigate the role of gustatory sensory neurons expressing IR76b for associative learning of amino acids, the building blocks of proteins. We found surprising complexity in the neuronal underpinnings of sensing amino acids, and a functional division of sensory neurons. We found that the IR76b receptor is dispensable for amino acid learning, whereas the neurons expressing IR76b are specifically required for the rewarding but not the punishing effect of amino acids. This unexpected dissociation in neuronal processing of amino acids for different behavioural functions provides a study case for functional divisions of labour in gustatory systems.