Reward Prediction Error Research Articles

Plastic and stimulus-specific coding of salient events in the central amygdala.

The central amygdala (CeA) is implicated in a range of mental processes including attention, motivation, memory formation and extinction and in behaviours driven by either aversive or appetitive stimuli1-7. How it participates in these divergent functions remains elusive. Here we show that somatostatin-expressing (Sst+) CeA neurons, which mediate much of CeA functions3,6,8-10, generate experience-dependent and stimulus-specific evaluative signals essential for learning. The population responses of these neurons in mice encode the identities of a wide range of salient stimuli, with the responses of separate subpopulations selectively representing the stimuli that have contrasting valences, sensory modalities or physical properties (for example, shock and water reward). These signals scale with stimulus intensity, undergo pronounced amplification and transformation during learning, and are required for both reward and aversive learning. Notably, these signals contribute to the responses of dopamine neurons to reward and reward prediction error, but not to their responses to aversive stimuli. In line with this, Sst+ CeA neuron outputs to dopamine areas are required for reward learning, but are dispensable for aversive learning. Our results suggest that Sst+ CeA neurons selectively process information about differing salient events for evaluation during learning, supporting the diverse roles of the CeA. In particular, the information for dopamine neurons facilitates reward evaluation.

Beta oscillations in monkey striatum encode reward prediction error signals.

Reward prediction error (RPE) signals are crucial for reinforcement learning and decision making as they quantify the mismatch between predicted and obtained rewards. RPE signals are encoded in the neural activity of multiple brain areas, such as midbrain dopaminergic neurons, prefrontal cortex and striatum. However, it remains unclear how these signals are expressed through anatomically and functionally distinct subregions of the striatum. In the current study, we examined to which extent RPE signals are represented across different striatal regions. To do so, we recorded local field potentials (LFPs) in sensorimotor, associative, and limbic striatal territories of two male rhesus monkeys performing a free-choice probabilistic learning task. The trial-by-trial evolution of RPE during task performance was estimated using a reinforcement learning model fitted on monkeys' choice behavior. Overall, we found that changes in beta-band oscillations (15-35 Hz), after the outcome of the animal's choice, are consistent with RPE encoding. Moreover, we provide evidence that the signals related to RPE are more strongly represented in the ventral (limbic) than dorsal (sensorimotor and associative) part of the striatum. To conclude, our results suggest a relationship between striatal beta oscillations and the evaluation of outcomes based on RPE signals and highlight a major contribution of the ventral striatum to the updating of learning processes.Significance StatementReward prediction error (RPE) signals are crucial for reinforcement learning and decision making as they quantify the mismatch between predicted and obtained rewards. Current models suggest that RPE signals are encoded in the neural activity of multiple brain areas, including the midbrain dopaminergic neurons, prefrontal cortex and striatum. However, it remains elusive whether RPEs recruit anatomically and functionally distinct subregions of the striatum. Our study provides evidence that RPE-related modulations in LFP power are dominant in the striatum. In particular, they are stronger in the rostro-ventral rather than the caudo-dorsal striatum. Our findings contribute to a better understanding of the role of striatal territories in reward-based learning and may be relevant for neuropsychiatric and neurological diseases that affect striatal circuits.

Rethinking model-based and model-free influences on mental effort and striatal prediction errors.

A standard assumption in neuroscience is that low-effort model-free learning is automatic and continuously used, whereas more complex model-based strategies are only used when the rewards they generate are worth the additional effort. We present evidence refuting this assumption. First, we demonstrate flaws in previous reports of combined model-free and model-based reward prediction errors in the ventral striatum that probably led to spurious results. More appropriate analyses yield no evidence of model-free prediction errors in this region. Second, we find that task instructions generating more correct model-based behaviour reduce rather than increase mental effort. This is inconsistent with cost-benefit arbitration between model-based and model-free strategies. Together, our data indicate that model-free learning may not be automatic. Instead, humans can reduce mental effort by using a model-based strategy alone rather than arbitrating between multiple strategies. Our results call for re-evaluation of the assumptions in influential theories of learning and decision-making.

Adaptive coding of pain prediction error in the anterior insula.

Understanding the mechanisms behind the influence of expectation and context on pain perception is crucial for improving analgesic treatments. Prediction error (PE) signals how much a noxious stimulus deviates from expectation and is therefore crucial for our understanding of pain perception. It is thought that the brain engages in 'adaptive coding' of pain PE, such that sensitivity to unexpected outcomes is modulated by contextual information. While there is behavioural evidence that pain is coded adaptively, and evidence that reward PE signals are coded adaptively, controversy remains regarding the underlying neural mechanism of adaptively-coded pain PEs. A cued-pain task was performed by 19 healthy adults while undergoing FMRI scanning. BOLD responses to the task were tested using an axiomatic approach to identify areas that may code pain PE adaptively. The left dorsal anterior insula demonstrated a pattern of response consistent with adaptively-coded pain PE. Signals from this area were sensitive to both predicted pain magnitudes on the instigation of expectation, and the unexpectedness of pain delivery. Crucially however, the response at pain delivery was consistent with the local context of the pain stimulation, rather than the absolute magnitude of delivered pain, a pattern suggestive of an adaptively-coded PE signal. The study advances our understanding of the neural basis of pain prediction. Alongside existing evidence that the periaqueductal grey codes pain PE and the posterior insula codes pain magnitude, the results highlight a distinct contribution of the left dorsal anterior insula in the processing of pain. Although there is behavioural evidence that pain is coded adaptively, the neural mechanisms serving this process are not well understood. This study used functional MRI to provide the first evidence that the left dorsal anterior insula, an area associated with aversive learning, responds to pain in a manner consistent with the adaptive coding of pain prediction error. This study aids our understanding of the neural basis of subjective pain representation, and thus can contribute to the advancement of analgesic treatments.

Do the negative symptoms of schizophrenia reflect reduced responsiveness to reward? Examination using a reward prediction error (RPE) task.

A leading theory of the negative symptoms of schizophrenia is that they reflect reduced responsiveness to rewarding stimuli. This proposal has been linked to abnormal (reduced) dopamine function in the disorder, because phasic release of dopamine is known to code for reward prediction error (RPE). Nevertheless, few functional imaging studies have examined if patients with negative symptoms show reduced RPE-associated activations. Matched groups of DSM-5 schizophrenia patients with high negative symptom scores (HNS, N = 27) or absent negative symptoms (ANS, N = 27) and healthy controls (HC, N = 30) underwent fMRI scanning while they performed a probabilistic monetary reward task designed to generate a measure of RPE. In the HC, whole-brain analysis revealed that RPE was positively associated with activation in the ventral striatum, the putamen, and areas of the lateral prefrontal cortex and orbitofrontal cortex, among other regions. Group comparison revealed no activation differences between the healthy controls and the ANS patients. However, compared to the ANS patients, the HNS patients showed regions of significantly reduced activation in the left ventrolateral and dorsolateral prefrontal cortex, and in the right lingual and fusiform gyrus. HNS and ANS patients showed no activation differences in ventral striatal or midbrain regions-of-interest (ROIs), but the HNS patients showed reduced activation in a left orbitofrontal cortex ROI. The findings do not suggest that a generalized reduction of RPE signalling underlies negative symptoms. Instead, they point to a more circumscribed dysfunction in the lateral frontal and possibly the orbitofrontal cortex.

The NK1 antagonist L-733,060 facilitates sequence learning.

Although several brain regions and electrophysiological patterns have been related to sequence learning, less attention has been paid to the role that different neuromodulators play. Here we sought to investigate the role of substance P (SP) in sequence learning in an operant conditioning preparation, supported by a reinforcement learning model. Two experiments were performed to test the effects of an NK1 receptor (at which SP primarily acts) antagonist on learning and performing action sequences. In experiment 1, rats were trained to perform an action sequence until stable performance was achieved, and then, in phase 2, they were switched to perform the reverse sequence. In experiment 2, rats were trained to perform an action sequence, and in phase 2, they continued to do the same sequence. In both experiments in the first 3 days of phase 2, rats were injected with an NK1 receptor antagonist (L-733,060, i.p.) or with vehicle. Additionally, we developed a reinforcement learning model which allowed the in silico replication of our experimental tasks. We found that administering an NK1 receptor antagonist weakened the stable retention of a well-learned sequence, allowing the faster acquisition of a new sequence, without impairing the continued performance of a crystallized sequence. Using our reinforcement learning model, we suggest that SP could be acting through the state value learning rate, modulating the effects of the reward prediction error. Our results suggest that SP could be involved in the consolidation of a sequence representation through a modulatory effect on the reward prediction error.

Distinct cortico-striatal compartments drive competition between adaptive and automatized behavior.

Cortical and basal ganglia circuits play a crucial role in the formation of goal-directed and habitual behaviors. In this study, we investigate the cortico-striatal circuitry involved in learning and the role of this circuitry in the emergence of inflexible behaviors such as those observed in addiction. Specifically, we develop a computational model of cortico-striatal interactions that performs concurrent goal-directed and habit learning. The model accomplishes this by distinguishing learning processes in the dorsomedial striatum (DMS) that rely on reward prediction error signals as distinct from the dorsolateral striatum (DLS) where learning is supported by salience signals. These striatal subregions each operate on unique cortical input: the DMS receives input from the prefrontal cortex (PFC) which represents outcomes, and the DLS receives input from the premotor cortex which determines action selection. Following an initial learning of a two-alternative forced choice task, we subjected the model to reversal learning, reward devaluation, and learning a punished outcome. Behavior driven by stimulus-response associations in the DLS resisted goal-directed learning of new reward feedback rules despite devaluation or punishment, indicating the expression of habit. We repeated these simulations after the impairment of executive control, which was implemented as poor outcome representation in the PFC. The degraded executive control reduced the efficacy of goal-directed learning, and stimulus-response associations in the DLS were even more resistant to the learning of new reward feedback rules. In summary, this model describes how circuits of the dorsal striatum are dynamically engaged to control behavior and how the impairment of executive control by the PFC enhances inflexible behavior.

Individuals with problem gambling and obsessive-compulsive disorder learn through distinct reinforcement mechanisms.

Obsessive-compulsive disorder (OCD) and pathological gambling (PG) are accompanied by deficits in behavioural flexibility. In reinforcement learning, this inflexibility can reflect asymmetric learning from outcomes above and below expectations. In alternative frameworks, it reflects perseveration independent of learning. Here, we examine evidence for asymmetric reward-learning in OCD and PG by leveraging model-based functional magnetic resonance imaging (fMRI). Compared with healthy controls (HC), OCD patients exhibited a lower learning rate for worse-than-expected outcomes, which was associated with the attenuated encoding of negative reward prediction errors in the dorsomedial prefrontal cortex and the dorsal striatum. PG patients showed higher and lower learning rates for better- and worse-than-expected outcomes, respectively, accompanied by higher encoding of positive reward prediction errors in the anterior insula than HC. Perseveration did not differ considerably between the patient groups and HC. These findings elucidate the neural computations of reward-learning that are altered in OCD and PG, providing a potential account of behavioural inflexibility in those mental disorders.

Dopamine error signal to actively cope with lack of expected reward

To obtain more of a particular uncertain reward, animals must learn to actively overcome the lack of reward and adjust behavior to obtain it again. The neural mechanisms underlying such coping with reward omission remain unclear. Here, we developed a task in rats to monitor active behavioral switch toward the next reward after no reward. We found that some dopamine neurons in the ventral tegmental area exhibited increased responses to unexpected reward omission and decreased responses to unexpected reward, following the opposite responses of the well-known dopamine neurons that signal reward prediction error (RPE). The dopamine increase reflected in the nucleus accumbens correlated with behavioral adjustment to actively overcome unexpected no reward. We propose that these responses signal error to actively cope with lack of expected reward. The dopamine error signal thus cooperates with the RPE signal, enabling adaptive and robust pursuit of uncertain reward to ultimately obtain more reward.

Feedback-related negativity in perfectionists: An index of performance outcome evaluation

It has been suggested that maladaptive perfectionists are more prone to concern over their performance outcomes than adaptive perfectionists. Performance outcome evaluation is reflected in the amplitude of feedback-related negativity (FRN) in brain electroencephalography (EEG). Hence, the amplitude of the FRN after receiving unfavorable feedback indicating a negative performance outcome may reflect personality characteristics. In other words, EEG could be a better marker of personality characteristics than self-report measures. However, the FRN component has not yet been investigated between different types of perfectionists. In the present study, group differences in the FRN were examined between two groups of adaptive and maladaptive perfectionists and a group of non-perfectionists during a monetary gambling task. We observed a larger FRN amplitude for adaptive perfectionists than for maladaptive perfectionists. This finding is consistent with previous reports that reward prediction error is reflected in the amplitude of the FRN. This difference in FRN could be interpreted as the pessimistic outcome expectation biases in maladaptive perfectionists.

Acute stress alters probabilistic reversal learning in healthy male adults.

Behavioural adaptation is a fundamental cognitive ability, ensuring survival by allowing for flexible adjustment to changing environments. In laboratory settings, behavioural adaptation can be measured with reversal learning paradigms requiring agents to adjust reward learning to stimulus-action-outcome contingency changes. Stress is found to alter flexibility of reward learning, but effect directionality is mixed across studies. Here, we used model-based functional MRI (fMRI) in a within-subjects design to investigate the effect of acute psychosocial stress on flexible behavioural adaptation. Healthy male volunteers (n= 28) did a reversal learning task during fMRI in two sessions, once after the Trier Social Stress Test (TSST), a validated psychosocial stress induction method, and once after a control condition. Stress effects on choice behaviour were investigated using multilevel generalized linear models and computational models describing different learning processes that potentially generated the data. Computational models were fitted using a hierarchical Bayesian approach, and model-derived reward prediction errors (RPE) were used as fMRI regressors. We found that acute psychosocial stress slightly increased correct response rates. Model comparison revealed that double-update learning with altered choice temperature under stress best explained the observed behaviour. In the brain, model-derived RPEs were correlated with BOLD signals in striatum and ventromedial prefrontal cortex (vmPFC). Striatal RPE signals for win trials were stronger during stress compared with the control condition. Our study suggests that acute psychosocial stress could enhance reversal learning and RPE brain responses in healthy male participants and provides a starting point to explore these effects further in a more diverse population.

Stronger coupling of emotional instability with reward processing in borderline personality disorder is predicted by schema modes.

Mood instability and risk-taking are hallmarks of borderline personality disorder (BPD). Schema modes are combinations of self-reflective evaluations, negative emotional states, and destructive coping strategies common in BPD. When activated, they can push patients with BPD into emotional turmoil and a dissociative state of mind. Our knowledge of the underlying neurocognitive mechanisms driving these changes is incomplete. We hypothesized that in patients with BPD, affective instability is more influenced by reward expectation, outcomes, and reward prediction errors (RPEs) during risky decision-making than in healthy controls. Additionally, we expected that these alterations would be related to schema modes. Thirty-two patients with BPD and thirty-one healthy controls were recruited. We used an established behavioral paradigm to measure mood fluctuations during risky decision-making. The impact of expectations and RPEs on momentary mood was quantified by a computational model, and its parameters were estimated with hierarchical Bayesian analysis. Model parameters were compared using High-Density Intervals. We found that model parameters capturing the influence of RPE and Certain Rewards on mood were significantly higher in patients with BPD than in controls. These model parameters correlated significantly with schema modes, but not with depression severity. BPD is coupled with altered associations between mood fluctuation and reward processing under uncertainty. Our findings seem to be BPD-specific, as they stand in contrast with the correlates of depressive symptoms. Future studies should establish the clinical utility of these alterations, such as predicting or assessing therapeutic response in BPD.

A computational neuroimaging study of reinforcement learning and goal-directed exploration in schizophrenia spectrum disorders.

Prior evidence indicates that negative symptom severity and cognitive deficits, in people with schizophrenia (PSZ), relate to measures of reward-seeking and loss-avoidance behavior (implicating the ventral striatum/VS), as well as uncertainty-driven exploration (reliant on rostrolateral prefrontal cortex/rlPFC). While neural correlates of reward-seeking and loss-avoidance have been examined in PSZ, neural correlates of uncertainty-driven exploration have not. Understanding neural correlates of uncertainty-driven exploration is an important next step that could reveal insights to how this mechanism of cognitive and negative symptoms manifest at a neural level. We acquired fMRI data from 29 PSZ and 36 controls performing the Temporal Utility Integration decision-making task. Computational analyses estimated parameters corresponding to learning rates for both positive and negative reward prediction errors (RPEs) and the degree to which participates relied on representations of relative uncertainty. Trial-wise estimates of expected value, certainty, and RPEs were generated to model fMRI data. Behaviorally, PSZ demonstrated reduced reward-seeking behavior compared to controls, and negative symptoms were positively correlated with loss-avoidance behavior. This finding of a bias toward loss avoidance learning in PSZ is consistent with previous work. Surprisingly, neither behavioral measures of exploration nor neural correlates of uncertainty in the rlPFC differed significantly between groups. However, we showed that trial-wise estimates of relative uncertainty in the rlPFC distinguished participants who engaged in exploratory behavior from those who did not. rlPFC activation was positively associated with intellectual function. These results further elucidate the nature of reinforcement learning and decision-making in PSZ and healthy volunteers.

Increased sucrose consumption in mice gene-targeted for Vmat2 selectively in NeuroD6-positive neurons of the ventral tegmental area.

Ventral tegmental area (VTA) dopamine (DA) neurons are implicated in reward processing, motivation, reward prediction error, and in substance use disorder. Recent studies have identified distinct neuronal subpopulations within the VTA that can be clustered based on their molecular identity, neurotransmitter profile, physiology, projections and behavioral role. One such subpopulation is characterized by expression of the NeuroD6 gene, and projects primarily to the nucleus accumbens medial shell. We recently showed that optogenetic stimulation of these neurons induces real-time place preference while their targeted deletion of the Vmat2 gene caused altered response to rewarding substances, including ethanol and psychostimulants. Based on these recent findings, we wanted to further investigate the involvement of the NeuroD6-positive VTA subpopulation in reward processing. Using the same NeuroD6Cre+/wt ;Vmat2flox/flox mice as in our prior study, we now addressed the ability of the mice to process sucrose reward. In order to assess appetitive behavior and motivation to obtain sucrose reward, we tested conditional knockout (cKO) and control littermate mice in an operant sucrose self-administration paradigm. We observed that cKO mice demonstrate higher response rates to the operant task and consume more sucrose rewards than control mice. However, their motivation to obtain sucrose is identical to that of control mice. Our results highlight previous observations that appetitive behavior and motivation to obtain rewards can be served by distinct neuronal circuits, and demonstrate that the NeuroD6 VTA subpopulation is involved in mediating the former, but not the latter. Together with previous studies on the NeuroD6 subpopulation, our findings pinpoint the importance of unraveling the molecular and functional role of VTA subpopulations in order to better understand normal behavior and psychiatric disease.

Ventral pallidal regulation of motivated behaviors and reinforcement.

The interconnected nuclei of the ventral basal ganglia have long been identified as key regulators of motivated behavior, and dysfunction of this circuit is strongly implicated in mood and substance use disorders. The ventral pallidum (VP) is a central node of the ventral basal ganglia, and recent studies have revealed complex VP cellular heterogeneity and cell- and circuit-specific regulation of reward, aversion, motivation, and drug-seeking behaviors. Although the VP is canonically considered a relay and output structure for this circuit, emerging data indicate that the VP is a central hub in an extensive network for reward processing and the regulation of motivation that extends beyond classically defined basal ganglia borders. VP neurons respond temporally faster and show more advanced reward coding and prediction error processing than neurons in the upstream nucleus accumbens, and regulate the activity of the ventral mesencephalon dopamine system. This review will summarize recent findings in the literature and provide an update on the complex cellular heterogeneity and cell- and circuit-specific regulation of motivated behaviors and reinforcement by the VP with a specific focus on mood and substance use disorders. In addition, we will discuss mechanisms by which stress and drug exposure alter the functioning of the VP and produce susceptibility to neuropsychiatric disorders. Lastly, we will outline unanswered questions and identify future directions for studies necessary to further clarify the central role of VP neurons in the regulation of motivated behaviors. Significance: Research in the last decade has revealed a complex cell- and circuit-specific role for the VP in reward processing and the regulation of motivated behaviors. Novel insights obtained using cell- and circuit-specific interrogation strategies have led to a major shift in our understanding of this region. Here, we provide a comprehensive review of the VP in which we integrate novel findings with the existing literature and highlight the emerging role of the VP as a linchpin of the neural systems that regulate motivation, reward, and aversion. In addition, we discuss the dysfunction of the VP in animal models of neuropsychiatric disorders.

The modulation of value-driven attentional capture by exploration for reward information.

Previous studies on value-driven attentional capture (VDAC) have demonstrated that the uncertainty of reward value modulates attentional allocation via associative learning. However, it is unclear whether such attentional exploration is executed based on the amount of potential reward information available for refining value prediction or the absolute size of reward prediction error. The present study investigated the effects of reward information (information entropy) and prediction error (variance) on attentional bias while controlling for the influence of the strength of reward association. Participants were instructed to search for either a red or green target circle and respond to the line orientation within the target. Each target color was associated with reward contingencies with different levels of uncertainty. In Experiment 1, one color was paired with a single reward value (zero entropy and variance) and the other with multiple reward values (high entropy and variance). In Experiment 2, one color had a high-entropy, low-variance reward contingency and the other had the inverse. Attentional interference for distractors with high entropy was consistently greater than low or zero entropy distractors. In addition, in Experiment 3, when distractors with an identical level of variance were given, information entropy was observed to modulate the attentional bias toward distractors. Lastly, Experiment 4 revealed that distractors associated with contrasting levels of variance, while information entropy was kept identical, failed to modulate VDAC. These results indicate that value-based attention is primarily allocated to cues that provide maximal information about the reward outcomes and that information entropy is one of the key predictors mediating attentional exploration and associative learning. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Neural responses in macaque prefrontal cortex are linked to strategic exploration.

Humans have been shown to strategically explore. They can identify situations in which gathering information about distant and uncertain options is beneficial for the future. Because primates rely on scarce resources when they forage, they are also thought to strategically explore, but whether they use the same strategies as humans and the neural bases of strategic exploration in monkeys are largely unknown. We designed a sequential choice task to investigate whether monkeys mobilize strategic exploration based on whether information can improve subsequent choice, but also to ask the novel question about whether monkeys adjust their exploratory choices based on the contingency between choice and information, by sometimes providing the counterfactual feedback about the unchosen option. We show that monkeys decreased their reliance on expected value when exploration could be beneficial, but this was not mediated by changes in the effect of uncertainty on choices. We found strategic exploratory signals in anterior and mid-cingulate cortex (ACC/MCC) and dorsolateral prefrontal cortex (dlPFC). This network was most active when a low value option was chosen, which suggests a role in counteracting expected value signals, when exploration away from value should to be considered. Such strategic exploration was abolished when the counterfactual feedback was available. Learning from counterfactual outcome was associated with the recruitment of a different circuit centered on the medial orbitofrontal cortex (OFC), where we showed that monkeys represent chosen and unchosen reward prediction errors. Overall, our study shows how ACC/MCC-dlPFC and OFC circuits together could support exploitation of available information to the fullest and drive behavior towards finding more information through exploration when it is beneficial.

Blunted Expected Reward Value Signals in Binge Alcohol Drinkers.

Alcohol-related morbidities and mortality are highly prevalent, increasing the burden to societies and health systems with 3 million deaths globally each year in young adults directly attributable to alcohol. Cue-induced alcohol craving has been formulated as a type of aberrant associative learning, modeled using temporal difference theory with an expected reward value (ERV) linked to craving. Clinically, although harmful use of alcohol is associated with increased time spent obtaining and using alcohol, it is also associated with self-neglect. The latter implies that the motivational aspects of nonalcohol stimuli are blunted. Using an instrumental learning task with non-alcohol-related stimuli, here, we tested hypotheses that the encoding of cue signals (ERV) predicting reward delivery would be blunted in binge alcohol drinkers in both sexes. We also predicted that for the binge drinking group alone, ratings of problematic alcohol use would correlate with abnormal ERV signals consistent with between groups (i.e., binge drinkers vs controls) abnormalities. Our results support our hypotheses with the ERV (nonalcohol cue) signal blunted in binge drinkers and with the magnitude of the abnormality correlating with ratings of problematic alcohol use. This implies that consistent with hypotheses, the motivational aspects of non-alcohol-related stimuli are blunted in binge drinkers. A better understanding of the mechanisms of harmful alcohol use will, in time, facilitate the development of more effective interventions, which should aim to decrease the motivational value of alcohol and increase the motivational value of non-alcohol-related stimuli.SIGNIFICANCE STATEMENT Allostasis theory predicts specific abnormalities in brain function and subjective experiences that occur when people develop drug problems including addiction. Cue-induced alcohol craving has been formulated as a type of aberrant associative learning, modeled using temporal difference theory with ERV linked to craving. Here, we used an instrumental learning task with non-alcohol-associated stimuli to test hypotheses that the encoding of nonalcohol cue signals (ERV) and reward prediction error signals showed blunting in binge alcohol drinkers. We conclude that fMRI can be used to noninvasively test allostasis and associative learning theory predictions in binge drinkers.

Invariant inhibition to calculate prediction errors?

The ventral tegmental area (VTA) has a pivotal role in motivated behavior. Much of the research on the VTA has focused on the mesocorticolimbic dopamine projections and their role in the computation of a ‘reward prediction error’ (RPE) for reward-guided learning. In a recent study, Zhou et al. report that VTA GABA neurons, the axons of which innervate the ventral pallidum (VP), have a unique role in signaling reward value to the basal ganglia and guiding reward seeking.

Neural mechanism underlying depressive-like state associated with social status loss

Downward social mobility is a well-known mental risk factor for depression, but its neural mechanism remains elusive. Here, by forcing mice to lose against their subordinates in a non-violent social contest, we lower their social ranks stably and induce depressive-like behaviors. These rank-decline-associated depressive-like behaviors can be reversed by regaining social status. Invivo fiber photometry and single-unit electrophysiological recording show that forced loss, but not natural loss, generates negative reward prediction error (RPE). Through the lateral hypothalamus, the RPE strongly activates the brain's anti-reward center, the lateral habenula (LHb). LHb activation inhibits the medial prefrontal cortex (mPFC) that controls social competitiveness and reinforces retreats in contests. These results reveal the core neural mechanisms mutually promoting social status loss and depressive behaviors. The intertwined neuronal signaling controlling mPFC and LHb activities provides a mechanistic foundation for the crosstalk between social mobility and psychological disorder, unveiling a promising target for intervention.