Negative Prediction Errors Research Articles

The expectation-updating mechanism in gratitude: A predictive coding perspective.

The fluctuations in emotions during constant help are unexplained by traditional emotion theories but may align with the predictive coding theory. This theory suggests that individuals tend to form expectations of others' help during social interactions. When outcomes exceed expectations, positive prediction errors are generated, potentially increasing gratitude. Conversely, constant help may build up expectations that surpass outcomes, resulting in negative prediction errors and reduced gratitude. Nevertheless, there is a lack of studies to examine the relationship between prediction errors and gratitude and its underlying mechanism. Here, we conducted two studies. Study 1 consistently found that higher expectations were associated with lower gratitude, when benefactors refused to help, in both reward-gaining and punishment-avoiding tasks. Moreover, prediction errors were positively and reliably linked to gratitude. Study 2 further identified that gratitude dynamically changed through an expectation-updating mechanism. A computational model incorporating predictive coding outperformed traditional theories in predicting the dynamics of gratitude. The findings support predictive coding theory, providing a temporal perspective and a mechanistic understanding of the fluctuations in gratitude, thus having implications for new interventions to improve mental health and well-being. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning

Environmental cues, through Pavlovian learning, become conditioned stimuli that invigorate and guide animals toward rewards. Dopamine (DA) neurons in the ventral tegmental area (VTA) and substantia nigra (SNc) are crucial for this process, via engagement of a reciprocally connected network with their striatal targets. Critically, it remains unknown how dopamine neuron activity itself engages dopamine signals throughout the striatum, across learning. Here, we investigated how optogenetic Pavlovian cue conditioning of VTA or SNc dopamine neurons directs cue-evoked behavior and shapes subregion-specific striatal dopamine dynamics. We used a fluorescent biosensor to monitor dopamine in the nucleus accumbens (NAc) core and shell, dorsomedial striatum (DMS), and dorsolateral striatum (DLS). We demonstrate spatially heterogeneous, learning-dependent dopamine changes across striatal regions. Although VTA stimulation-evoked robust dopamine release in NAc core, shell, and DMS, predictive cues preferentially recruited dopamine release in NAc core, starting early in training, and DMS, late in training. Negative prediction error signals, reflecting a violation in the expectation of dopamine neuron activation, only emerged in the NAc core and DMS. Despite the development of vigorous movement late in training, conditioned dopamine signals did not emerge in the DLS, even during Pavlovian conditioning with SNc dopamine neuron activation, which elicited robust DLS dopamine release. Together, our studies show a broad dissociation in the fundamental prediction and reward-related information generated by VTA and SNc dopamine neuron populations and signaled by dopamine across the striatum. Further, they offer new insight into how larger-scale adaptations across the striatal network emerge during learning to coordinate behavior.

Uncertainty of treatment efficacy moderates placebo effects on reinforcement learning

The placebo-reward hypothesis postulates that positive effects of treatment expectations on health (i.e., placebo effects) and reward processing share common neural underpinnings. Moreover, experiments in humans and animals indicate that reward uncertainty increases striatal dopamine, which is presumably involved in placebo responses and reward learning. Therefore, treatment uncertainty analogously to reward uncertainty may affect updating from rewards after placebo treatment. Here, we address whether different degrees of uncertainty regarding the efficacy of a sham treatment affect reward sensitivity. In an online between-subjects experiment with N = 141 participants, we systematically varied the provided efficacy instructions before participants first received a sham treatment that consisted of listening to binaural beats and then performed a probabilistic reinforcement learning task. We fitted a Q-learning model including two different learning rates for positive (gain) and negative (loss) reward prediction errors and an inverse gain parameter to behavioral decision data in the reinforcement learning task. Our results yielded an inverted-U-relationship between provided treatment efficacy probability and learning rates for gain, such that higher levels of treatment uncertainty, rather than of expected net efficacy, affect presumably dopamine-related reward learning. These findings support the placebo-reward hypothesis and suggest harnessing uncertainty in placebo treatment for recovering reward learning capabilities.

The Causes and Consequences of Drifting Expectations

Awaiting news of uncertain outcomes is distressing because the news might be disappointing. To prevent such disappointments, people often “brace for the worst,” pessimistically lowering expectations before news arrives to decrease the possibility of surprising disappointment (a negative prediction error, or PE). Computational decision-making research commonly assumes that expectations do not drift within trials, yet it is unclear whether expectations pessimistically drift in real-world, high-stakes settings, what factors influence expectation drift, and whether it effectively buffers emotional responses to goal-relevant outcomes. Moreover, individuals learn from PEs to accurately anticipate future outcomes, but it is unknown whether expectation drift also impedes PE-based learning. In a sample of students awaiting exam grades (N = 625), we found that expectations often drift and tend to drift pessimistically. We demonstrate that bracing is preferentially modulated by uncertainty; it transiently buffers the initial emotional impact of negative PEs but impairs PE-based learning, counterintuitively sustaining uncertainty into the future.

Belief Updating during Social Interactions: Neural Dynamics and Causal Role of Dorsomedial Prefrontal Cortex.

In competitive interactions, humans have to flexibly update their beliefs about another person's intentions in order to adjust their own choice strategy, such as when believing that the other may exploit their cooperativeness. Here we investigate both the neural dynamics and the causal neural substrate of belief updating processes in humans. We used an adapted prisoner's dilemma game in which participants explicitly predicted the coplayer's actions, which allowed us to quantify the prediction error between expected and actual behavior. First, in an EEG experiment, we found a stronger medial frontal negativity (MFN) for negative than positive prediction errors, suggesting that this medial frontal ERP component may encode unexpected defection of the coplayer. The MFN also predicted subsequent belief updating after negative prediction errors. In a second experiment, we used transcranial magnetic stimulation (TMS) to investigate whether the dorsomedial prefrontal cortex (dmPFC) causally implements belief updating after unexpected outcomes. Our results show that dmPFC TMS impaired belief updating and strategic behavioral adjustments after negative prediction errors. Taken together, our findings reveal the time course of the use of prediction errors in social decisions and suggest that the dmPFC plays a crucial role in updating mental representations of others' intentions.

The effect of reward prediction errors on subjective affect depends on outcome valence and decision context.

The valence of an individual's emotional response to an event is often thought to depend on their prior expectations for the event: better-than-expected outcomes produce positive affect and worse-than-expected outcomes produce negative affect. In recent years, this hypothesis has been instantiated within influential computational models of subjective affect that assume the valence of affect is driven by reward prediction errors. However, there remain a number of open questions regarding this association. In this project, we investigated the moderating effects of outcome valence and decision context (Experiment 1: free vs. forced choices; Experiment 2: trials with vs. trials without counterfactual feedback) on the effects of reward prediction errors on subjective affect. We conducted two online experiments (N = 300 in total) of general-population samples recruited via Prolific to complete a risky decision-making task with an embedded high-resolution sampling of subjective affect. Hierarchical Bayesian computational modeling revealed that both outcome amount and reward prediction errors influenced subjective affect, but that the effects of reward prediction errors were moderated by both outcome valence and decision context. Specifically, we found evidence that only negative reward prediction errors (worse-than-expected outcomes) influenced subjective affect, with no significant effect of positive reward prediction errors (better-than-expected outcomes). Moreover, these effects were only apparent on trials in which participants made a choice freely (but not on forced-choice trials) and when counterfactual feedback was absent (but not when counterfactual feedback was present). These results deepen our understanding of the effects of reward prediction errors on subjective affect. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Dynamics Learning Rate Bias in Pigeons: Insights from Reinforcement Learning and Neural Correlates

Research in reinforcement learning indicates that animals respond differently to positive and negative reward prediction errors, which can be calculated by assuming learning rate bias. Many studies have shown that humans and other animals have learning rate bias during learning, but it is unclear whether and how the bias changes throughout the entire learning process. Here, we recorded the behavior data and the local field potentials (LFPs) in the striatum of five pigeons performing a probabilistic learning task. Reinforcement learning models with and without learning rate biases were used to dynamically fit the pigeons’ choice behavior and estimate the option values. Furthemore, the correlation between the striatal LFPs power and the model-estimated option values was explored. We found that the pigeons’ learning rate bias shifted from negative to positive during the learning process, and the striatal Gamma (31 to 80 Hz) power correlated with the option values modulated by dynamic learning rate bias. In conclusion, our results support the hypothesis that pigeons employ a dynamic learning strategy in the learning process from both behavioral and neural aspects, providing valuable insights into reinforcement learning mechanisms of non-human animals.

Risk preference as an outcome of evolutionarily adaptive learning mechanisms: An evolutionary simulation under diverse risky environments.

The optimization of cognitive and learning mechanisms can reveal complicated behavioral phenomena. In this study, we focused on reinforcement learning, which uses different learning rules for positive and negative reward prediction errors. We attempted to relate the evolved learning bias to the complex features of risk preference such as domain-specific behavior manifests and the relatively stable domain-general factor underlying behaviors. The simulations of the evolution of the two learning rates under diverse risky environments showed that the positive learning rate evolved on average to be higher than the negative one, when agents experienced both tasks where risk aversion was more rewarding and risk seeking was more rewarding. This evolution enabled agents to flexibly choose more reward behaviors depending on the task type. The evolved agents also demonstrated behavioral patterns described by the prospect theory. Our simulations captured two aspects of the evolution of risk preference: the domain-specific aspect, behavior acquired through learning in a specific context; and the implicit domain-general aspect, corresponding to the learning rates shaped through evolution to adaptively behave in a wide range of environments. These results imply that our framework of learning under the innate constraint may be useful in understanding the complicated behavioral phenomena.

Computational modelling of reinforcement learning and functional neuroimaging of probabilistic reversal for dissociating compulsive behaviours in gambling and cocaine use disorders.

Individuals with cocaine use disorder or gambling disorder demonstrate impairments in cognitive flexibility: the ability to adapt to changes in the environment. Flexibility is commonly assessed in a laboratory setting using probabilistic reversal learning, which involves reinforcement learning, the process by which feedback from the environment is used to adjust behavior. It is poorly understood whether impairments in flexibility differ between individuals with cocaine use and gambling disorders, and how this is instantiated by the brain. We applied computational modelling methods to gain a deeper mechanistic explanation of the latent processes underlying cognitive flexibility across two disorders of compulsivity. We present a re-analysis of probabilistic reversal data from individuals with either gambling disorder (n = 18) or cocaine use disorder (n = 20) and control participants (n = 18), using a hierarchical Bayesian approach. Furthermore, we relate behavioural findings to their underlying neural substrates through an analysis of task-based functional magnetic resonanceimaging (fMRI) data. We observed lower 'stimulus stickiness' in gambling disorder, and report differences in tracking expected values in individuals with gambling disorder compared to controls, with greater activity during reward expected value tracking in the cingulate gyrus and amygdala. In cocaine use disorder, we observed lower responses to positive punishment prediction errors and greater activity following negative punishment prediction errors in the superior frontal gyrus compared to controls. Using a computational approach, we show that individuals with gambling disorder and cocaine use disorder differed in their perseverative tendencies and in how they tracked value neurally, which has implications for psychiatric classification.

Accuracy of Different Lens Power Calculation Formulas in Patients With Mature Cataracts.

Introduction To compare the prediction accuracy of lens power calculation formulas in patients undergoing mature cataract surgery. Methods A total of 90 operations involving the Alcon SA60AT IOL implant (Alcon, Geneva, Switzerland) were analyzed in terms of mean refractive prediction error (PE) and mean absolute prediction error (MAE) using backward calculation in a retrospective design. Results A negative PE was observed in SRK/T, Holladay 1, Holladay 2, Hoffer Q, Haigis, and Emmetropia Verifying Optical (EVO) formulas. In contrast, positive PEs were observed in Barrett Universal II (BAUII), Kane, and Radial Basis Function (RBF) formulas. Negative PE was observed with all formulas, except BAUII, in patients with a shallow anterior chamber depth (ACD). While the SRK/T, Holladay 1, BAU, Kane, and RBF formulas demonstrated positive PE, the Holladay 2, Hoffer Q, Haigis, and EVO formulas indicated negative PE. In patients with deep ACD, positive PE was observed in all formulas, barring Holladay 2 and EVO. No significant differences were identified between the formulas concerning MAE and percentages of 0.25 diopter (D), 0.50 D, 0.75 D, and 1.0 D across all study groups. Conclusion Although the new generation formulas provide very good results, achieving the best with a single formula is still impossible.

Molecularly targetable cell types in mouse visual cortex have distinguishable prediction error responses

Predictive processing postulates the existence of prediction error neurons in cortex. Neurons with both negative and positive prediction error response properties have been identified in layer 2/3 of visual cortex, but whether they correspond to transcriptionally defined subpopulations is unclear. Here we used the activity-dependent, photoconvertible marker CaMPARI2 to tag neurons in layer 2/3 of mouse visual cortex during stimuli and behaviors designed to evoke prediction errors. We performed single-cell RNA-sequencing on these populations and found that previously annotated Adamts2 and Rrad layer 2/3 transcriptional cell types were enriched when photolabeling during stimuli that drive negative or positive prediction error responses, respectively. Finally, we validated these results functionally by designing artificial promoters for use in AAV vectors to express genetically encoded calcium indicators. Thus, transcriptionally distinct cell types in layer 2/3 that can be targeted using AAV vectors exhibit distinguishable negative and positive prediction error responses.

The shadowing effect of initial expectation on learning asymmetry.

Evidence for positivity and optimism bias abounds in high-level belief updates. However, no consensus has been reached regarding whether learning asymmetries exist in more elementary forms of updates such as reinforcement learning (RL). In RL, the learning asymmetry concerns the sensitivity difference in incorporating positive and negative prediction errors (PE) into value estimation, namely the asymmetry of learning rates associated with positive and negative PEs. Although RL has been established as a canonical framework in characterizing interactions between agent and environment, the direction of learning asymmetry remains controversial. Here, we propose that part of the controversy stems from the fact that people may have different value expectations before entering the learning environment. Such a default value expectation influences how PEs are calculated and consequently biases subjects' choices. We test this hypothesis in two learning experiments with stable or varying reinforcement probabilities, across monetary gains, losses, and gain-loss mixed environments. Our results consistently support the model incorporating both asymmetric learning rates and the initial value expectation, highlighting the role of initial expectation in value updating and choice preference. Further simulation and model parameter recovery analyses confirm the unique contribution of initial value expectation in accessing learning rate asymmetry.

Involvement of cortical input to the rostromedial tegmental nucleus in aversion to foot shock.

The rostromedial tegmental nucleus (RMTg) encodes negative reward prediction error (RPE) and plays an important role in guiding behavioral responding to aversive stimuli. Previous research has focused on regulation of RMTg activity by the lateral habenula despite studies revealing RMTg afferents from other regions including the frontal cortex. The current study provides a detailed anatomical and functional analysis of cortical input to the RMTg of male rats. Retrograde tracing uncovered dense cortical input to the RMTg spanning the medial prefrontal cortex, the orbitofrontal cortex and anterior insular cortex. Afferents were most dense in the dorsomedial subregion of the PFC (dmPFC), an area that is also implicated in both RPE signaling and aversive responding. RMTg-projecting dmPFC neurons originate in layer V, are glutamatergic, and collateralize to select brain regions. In-situ mRNA hybridization revealed that neurons in this circuit are predominantly D1 receptor-expressing with a high degree of D2 receptor colocalization. Consistent with cFos induction in this neural circuit during exposure to foot shock and shock-predictive cues, optogenetic stimulation of dmPFC terminals in the RMTg drove avoidance. Lastly, acute slice electrophysiology and morphological studies revealed that exposure to repeated foot shock resulted in significant physiological and structural changes consistent with a loss of top-down modulation of RMTg-mediated signaling. Altogether, these data reveal the presence of a prominent cortico-subcortical projection involved in adaptive behavioral responding to aversive stimuli such as foot shock and provide a foundation for future work aimed at exploring alterations in circuit function in diseases characterized by deficits in cognitive control over reward and aversion.

Periaqueductal gray matter and medial prefrontal cortex reflect negative prediction errors during differential conditioning.

Computational models of associative learning posit that negative prediction errors (PEs) arising from the omission of aversive outcomes weaken aversive Pavlovian associations during differential conditioning and extinction. It is possible that negative PEs may underlie exaggerated conditioned responses to the conditioned stimulus not paired with an aversitve outcome (CS-) during differential conditioning and to the conditioned stimulus originally paired with a aversive outcome (CS+) during extinction in patients with clinical anxiety disorders. Although previous research has demonstrated that manipulations of the periaqueductal gray matter (PAG) interfere with extinction learning in animals, the role of the PAG in processing negative PEs within the human brain is presently unclear. We set out to investigate how PAG responses and connectivity are impacted by negative PEs using ultra-high-field (7 T) functional magnetic resonance imaging and hierarchical Bayesian analysis. During differential conditioning, negative PEs were associated with larger responses within the lateral and dorsolateral PAG and increased connectivity between the dorsolateral PAG and medial areas of Brodmann area 9. Collectively, these results shed light on the association between activity within the PAG and medial prefrontal cortex and the omission of aversive outcomes during Pavlovian learning.

Prediction errors arising from switches between major and minor modes in music: An fMRI study

The major and minor modes in Western music have positive and negative connotations, respectively. The present fMRI study examined listeners’ neural responses to switches between major and minor modes. We manipulated the final chords of J. S. Bach’s keyboard pieces so that each major-mode passage ended with either the major (Major-Major) or minor (Major-Minor) tonic chord, and each minor-mode passage ended with either the minor (Minor-Minor) or major (Minor-Major) tonic chord. If the final major and minor chords have positive and negative reward values respectively, the Major-Minor and Minor-Major stimuli would cause negative and positive reward prediction errors (RPEs) respectively in a listener’s brain. We found that activity in a frontoparietal network was significantly higher for Major-Minor than for Major-Major. Based on previous research, these results support the idea that a major-to-minor switch causes negative RPE. The contrast of Minor-Major minus Minor-Minor yielded activation in the ventral insula and visual cortex, speaking against the idea that a minor-to-major switch causes positive RPE. We discuss our results in relation to executive functions and the emotional connotations of major versus minor modes.

The Human Centromedial Amygdala Contributes to Negative Prediction Error Signaling during Appetitive and Aversive Pavlovian Gustatory Learning.

Prediction error (PE) is the mismatch between a prior expectation and reality, and it lies at the core of associative learning about aversive and appetitive stimuli. Human studies on fear learning have linked the amygdala to aversive PEs. In contrast, the relationship between the amygdala and PE in appetitive settings and stimuli, unlike those that induce fear, has received less research attention. Animal studies show that the amygdala is a functionally heterogeneous structure. Nevertheless, the role of the amygdala nuclei in PE signaling remains unknown in humans. To clarify the role of two subdivisions of the human amygdala, the centromedial amygdala (CMA) and basolateral amygdala (BLA), in appetitive and aversive PE signaling, we used gustatory pavlovian learning involving eating-related naturalistic outcomes. Thirty-eight right-handed individuals (19 females) participated in the study. We found that surprise with neutral feedback when a reward is expected triggers activity within the left and right CMA. When an aversive outcome is expected, surprise with neutral feedback triggers activity only within the left CMA. Notably, the BLA was not activated by those conditions. Thus, the CMA engages in negative PE signaling during appetitive and aversive gustatory pavlovian learning, whereas the BLA is not critical for this process. In addition, PE-related activity within the left CMA during aversive learning is negatively correlated with neuroticism and positively correlated with extraversion. The findings indicate the importance of the CMA in gustatory learning when the value of outcomes changes and have implications for understanding psychological conditions that manifest perturbed processing of negative PEs.SIGNIFICANCE STATEMENT A discrepancy between a prediction and an actual outcome (PE) plays a crucial role in learning. Learning improves when an outcome is more significant than expected (positive PE) and worsens when it is smaller than expected (negative PE). We found that the negative PE during appetitive and aversive taste learning is associated with increased activity of the CMA, which suggests that the CMA controls taste learning. Our findings may have implications for understanding psychological states associated with deficient learning from negative PEs, such as obesity and addictive behaviors.

Individuals with problem gambling and obsessive-compulsive disorder learn through distinct reinforcement mechanisms.

Obsessive-compulsive disorder (OCD) and pathological gambling (PG) are accompanied by deficits in behavioural flexibility. In reinforcement learning, this inflexibility can reflect asymmetric learning from outcomes above and below expectations. In alternative frameworks, it reflects perseveration independent of learning. Here, we examine evidence for asymmetric reward-learning in OCD and PG by leveraging model-based functional magnetic resonance imaging (fMRI). Compared with healthy controls (HC), OCD patients exhibited a lower learning rate for worse-than-expected outcomes, which was associated with the attenuated encoding of negative reward prediction errors in the dorsomedial prefrontal cortex and the dorsal striatum. PG patients showed higher and lower learning rates for better- and worse-than-expected outcomes, respectively, accompanied by higher encoding of positive reward prediction errors in the anterior insula than HC. Perseveration did not differ considerably between the patient groups and HC. These findings elucidate the neural computations of reward-learning that are altered in OCD and PG, providing a potential account of behavioural inflexibility in those mental disorders.

Slow-rising and fast-falling dopaminergic dynamics jointly adjust negative prediction error in the ventral striatum.

The greater the reward expectations are, the more different the brain's physiological response will be. Although it is well-documented that better-than-expected outcomes are encoded quantitatively via midbrain dopaminergic (DA) activity, it has been less addressed experimentally whether worse-than-expected outcomes are expressed quantitatively as well. We show that larger reward expectations upon unexpected reward omissions are associated with the preceding slower rise and following larger decrease (DA dip) in the DA concentration at the ventral striatum of mice. We set up a lever press task on a fixed ratio (FR) schedule requiring five lever presses as an effort for a food reward (FR5). The mice occasionally checked the food magazine without a reward before completing the task. The percentage of this premature magazine entry (PME) increased as the number of lever presses approached five, showing rising expectations with increasing proximity to task completion, and hence greater reward expectations. Fibre photometry of extracellular DA dynamics in the ventral striatum using a fluorescent protein (genetically encoded GPCR activation-based DA sensor: GRABDA2m ) revealed that the slow increase and fast decrease in DA levels around PMEs were correlated with the PME percentage, demonstrating a monotonic relationship between the DA dip amplitude and degree of expectations. Computational modelling of the lever press task implementing temporal difference errors and state transitions replicated the observed correlation between the PME frequency and DA dip amplitude in the FR5 task. Taken together, these findings indicate that the DA dip amplitude represents the degree of reward expectations monotonically, which may guide behavioural adjustment.

The power of the unexpected: Prediction errors enhance stereotype-based learning

Stereotyping is a ubiquitous feature of social cognition, yet surprisingly little is known about how group-related beliefs influence the acquisition of person knowledge. Accordingly, in combination with computational modeling (i.e., Reinforcement Learning Drift Diffusion Model analysis), here we used a probabilistic selection task to explore the extent to which gender stereotypes impact instrumental learning. Several theoretically interesting effects were observed. First, reflecting the impact of cultural socialization on person construal, an expectancy-based preference for stereotype-consistent (vs. stereotype-inconsistent) responses was observed. Second, underscoring the potency of unexpected information, learning rates were faster for counter-stereotypic compared to stereotypic individuals, both for negative and positive prediction errors. Collectively, these findings are consistent with predictive accounts of social perception and have implications for the conditions under which stereotyping can potentially be reduced.

A computational neuroimaging study of reinforcement learning and goal-directed exploration in schizophrenia spectrum disorders.

Prior evidence indicates that negative symptom severity and cognitive deficits, in people with schizophrenia (PSZ), relate to measures of reward-seeking and loss-avoidance behavior (implicating the ventral striatum/VS), as well as uncertainty-driven exploration (reliant on rostrolateral prefrontal cortex/rlPFC). While neural correlates of reward-seeking and loss-avoidance have been examined in PSZ, neural correlates of uncertainty-driven exploration have not. Understanding neural correlates of uncertainty-driven exploration is an important next step that could reveal insights to how this mechanism of cognitive and negative symptoms manifest at a neural level. We acquired fMRI data from 29 PSZ and 36 controls performing the Temporal Utility Integration decision-making task. Computational analyses estimated parameters corresponding to learning rates for both positive and negative reward prediction errors (RPEs) and the degree to which participates relied on representations of relative uncertainty. Trial-wise estimates of expected value, certainty, and RPEs were generated to model fMRI data. Behaviorally, PSZ demonstrated reduced reward-seeking behavior compared to controls, and negative symptoms were positively correlated with loss-avoidance behavior. This finding of a bias toward loss avoidance learning in PSZ is consistent with previous work. Surprisingly, neither behavioral measures of exploration nor neural correlates of uncertainty in the rlPFC differed significantly between groups. However, we showed that trial-wise estimates of relative uncertainty in the rlPFC distinguished participants who engaged in exploratory behavior from those who did not. rlPFC activation was positively associated with intellectual function. These results further elucidate the nature of reinforcement learning and decision-making in PSZ and healthy volunteers.