A ventral striatal prediction error signal in human fear extinction learning

Potential Vulnerabilities of Neuronal Reward, Risk, and Decision Mechanisms to Addictive Drugs

Mechanisms Underlying Dopamine-Mediated Reward Bias in Compulsive Behaviors

Disproportionate reactions to unexpected stimuli in the environment are a cardinal symptom of posttraumatic stress disorder (PTSD). Here, we test whether these heightened responses are associated with disruptions in distinct components of reinforcement learning. Specifically, using functional neuroimaging, a loss-learning task, and a computational model-based approach, we assessed the mechanistic hypothesis that overreactions to stimuli in PTSD arise from anomalous gating of attention during learning (i.e., associability). Behavioral choices of combat-deployed veterans with and without PTSD were fit to a reinforcement learning model, generating trial-by-trial prediction errors (signaling unexpected outcomes) and associability values (signaling attention allocation to the unexpected outcomes). Neural substrates of associability value and behavioral parameter estimates of associability updating, but not prediction error, increased with PTSD during loss learning. Moreover, the interaction of PTSD severity with neural markers of associability value predicted behavioral choices. These results indicate that increased attention-based learning may underlie aspects of PTSD and suggest potential neuromechanistic treatment targets.

SummaryHow the brain uses success and failure to optimize future decisions is a long-standing question in neuroscience. One computational solution involves updating the values of context-action associations in proportion to a reward prediction error. Previous evidence suggests that such computations are expressed in the striatum and, as they are cognitively impenetrable, represent an unconscious learning mechanism. Here, we formally test this by studying instrumental conditioning in a situation where we masked contextual cues, such that they were not consciously perceived. Behavioral data showed that subjects nonetheless developed a significant propensity to choose cues associated with monetary rewards relative to punishments. Functional neuroimaging revealed that during conditioning cue values and prediction errors, generated from a computational model, both correlated with activity in ventral striatum. We conclude that, even without conscious processing of contextual cues, our brain can learn their reward value and use them to provide a bias on decision making.

P02-358 - Prediction error signal correlates with fluid intelligenceand dopamine synthesis across the lifespan

Initially reported in dopamine neurons, neural correlates of prediction errors have now been shown in a variety of areas, including orbitofrontal cortex, ventral striatum, and amygdala. Yet changes in neural activity to an outcome or cues that precede it can reflect other processes. We review the recent literature and show that although activity in dopamine neurons appears to signal prediction errors, similar activity in orbitofrontal cortex, basolateral amygdala, and ventral striatum does not. Instead, increased firing in basolateral amygdala to unexpected outcomes likely reflects attention, whereas activity in orbitofrontal cortex and ventral striatum is unaffected by prior expectations and may provide information on outcome expectancy. These results have important implications for how these areas interact to facilitate learning and guide behavior.

Dysfunctional processing of reward and punishment may play an important role in depression. However, functional magnetic resonance imaging (fMRI) studies have shown heterogeneous results for reward processing in fronto-striatal regions. We examined neural responsivity associated with the processing of reward and loss during anticipation and receipt of incentives and related prediction error (PE) signalling in depressed individuals. Thirty medication-free depressed persons and 28 healthy controls performed an fMRI reward paradigm. Regions of interest analyses focused on neural responses during anticipation and receipt of gains and losses and related PE-signals. Additionally, we assessed the relationship between neural responsivity during gain/loss processing and hedonic capacity. When compared with healthy controls, depressed individuals showed reduced fronto-striatal activity during anticipation of gains and losses. The groups did not significantly differ in response to reward and loss outcomes. In depressed individuals, activity increases in the orbitofrontal cortex and nucleus accumbens during reward anticipation were associated with hedonic capacity. Depressed individuals showed an absence of reward-related PEs but encoded loss-related PEs in the ventral striatum. Depression seems to be linked to blunted responsivity in fronto-striatal regions associated with limited motivational responses for rewards and losses. Alterations in PE encoding might mirror blunted reward- and enhanced loss-related associative learning in depression.

Once a threat no longer exists, extinction of conditioned fear becomes adaptive in order to reduce allotted resources towards cues that no longer predict the threat. In anxiety and stress disorders, fear extinction learning may be affected. Animal findings suggest that the administration of oxytocin (OT) modulates extinction learning in a timepoint-dependent manner, facilitating extinction when administered prior to fear conditioning, but impairing it when administered prior to extinction learning. The aim of the present study was to examine if these findings translate into human research. Using a randomized, double-blind, placebo-controlled, 2-day fear conditioning and extinction learning design, behavioral (self-reported anxiety), physiological (skin conductance response), neuronal (task-based and resting-state functional magnetic resonance imaging), and hormonal (cortisol) data were collected from 124 naturally cycling (taking no hormonal contraceptives) healthy females. When administered prior to conditioning (Day 1), OT, similar to rodent findings, did not affect fear conditioning, but modulated the intrinsic functional connectivity of the anterior insula immediately after fear conditioning. In contrast to animal findings, OT impaired, not facilitated, extinction learning on the next day and increased anterior insula activity. When administered prior to extinction learning (day 2), OT increased the activity in the bilateral middle temporal gyrus, and similar to animal findings, reduced extinction learning. The current findings suggest that intranasal OT impedes fear extinction learning in humans regardless of the timepoint of administration, providing new insights and directions for future translational research and clinical applications.

Fear learning is mediated by a large network of brain structures and the understanding of their roles and interactions is constantly progressing. There is a multitude of anatomical and behavioral evidence on the interconnection of the cerebellar nuclei to other structures in the fear network. Regarding the cerebellar nuclei, we focus on the coupling of the cerebellar fastigial nucleus to the fear network and the relation of the cerebellar dentate nucleus to the ventral tegmental area. Many of the fear network structures that receive direct projections from the cerebellar nuclei are playing a role in fear expression or in fear learning and fear extinction learning. We propose that the cerebellum, via its projections to the limbic system, acts as a modulator of fear learning and extinction learning, using prediction-error signaling and regulation of fear related thalamo-cortical oscillations.

BackgroundPrediction error is the mismatch between expected and obtained outcome, and psychosis has been linked to aberrant striatal prediction error signal. Several lines of evidence indicate alterations of the glutamatergic system to be involved in the pathophysiology of schizophrenia. We have previously reported abnormal thalamic glutamate levels at illness onset in schizophrenia patients driven by increased levels in non-responding patients, and that glutamatergic levels in the thalamus in twins dis- or concordant for psychosis were heritable and associated with the illness. Glutamatergic abnormalities may affect processing of prediction error; however, it remains unresolved if prediction error is affected by antipsychotic treatment, and to which extend treatment effect on prediction error is predicted by glutamatergic levels in patients characterized as responders or non-responders.Here, we explore treatment effects of aripiprazole on striatal prediction error signal in initially antipsychotic-naïve patients characterized as responders and non-responders and relate the findings to thalamic glutamate levels. We hypothesize a different treatment response in prediction error signal in responders and non-responders, and an association to baseline glutamate levels.MethodsThirty-three patients (age 22 ± 4 years) and 33 healthy controls (HC) matched on age and gender underwent functional Magnetic Resonance imaging (fMRI) and magnetic resonance spectroscopy (1H-MRS) (3T) at baseline and after 6 weeks of treatment with aripiprazole.Prediction error related brain activity was examined using a Monetary Incentive Delay Task. Glutamate levels were estimated in the left thalamus and analyzed using LCModel. In patients, symptom severity was assessed with the Positive and Negative Syndrome Scale. The Andreasen criteria defined responders (N=10) and non- responders (N=23).Repeated measures analysis of variance was used to test the effect of time in prediction error signal with group (responders vs non-responders vs HC) as between subject factor and time as within factor. Analysis of variance, two sample t-test and paired sample t-test evaluated group differences at baseline and follow up. In a multiple regression analyses we investigated the influence of baseline glutamate levels, symptom severity and p-aripiprazole on changes in prediction error signal in both responders and non-responders.ResultsRepeated measures analysis of prediction error showed an effect of group (p=0.007) and no effect of time (p=0.29) or interaction (p=0.29). The effect of group was explained by an abnormal increased prediction error signal in responders (p=0.047) and non-responders (p=0.011) compared to HC at baseline, which was normalized at follow up in responders (p=0.94) but not in non-responders (p=0.02) compared to HC. Changes in prediction error signal following treatment were predicted by glutamate levels in non-responders (p=0.03) but not in responders (p=0.85) whereas p-aripiprazole and symptom severity did not predict changes in prediction error signal (all p>0.05).DiscussionThe findings suggest that treatment with a partial dopamine agonist normalizes prediction error signal in patients characterized as responders. Thalamic glutamate seems to play a role in the neural coding of prediction error in patients characterized as non-responders, where increased levels of glutamate in the thalamus seems to predict a less pronounced changes in prediction error signal.

The neural correlates of negative prediction error signaling in human fear conditioning

Why Do Children With Disruptive Behavior Disorders Keep Making Bad Choices?

Classical learning theories predict extinction after the discontinuation of reinforcement through prediction errors. However, placebo hypoalgesia, although mediated by associative learning, has been shown to be resistant to extinction. We tested the hypothesis that this is mediated by the suppression of prediction error processing through the prefrontal cortex (PFC). We compared pain modulation through treatment cues (placebo hypoalgesia, treatment context) with pain modulation through stimulus intensity cues (stimulus context) during functional magnetic resonance imaging in 48 male and female healthy volunteers. During acquisition, our data show that expectations are correctly learned and that this is associated with prediction error signals in the ventral striatum (VS) in both contexts. However, in the nonreinforced test phase, pain modulation and expectations of pain relief persisted to a larger degree in the treatment context, indicating that the expectations were not correctly updated in the treatment context. Consistently, we observed significantly stronger neural prediction error signals in the VS in the stimulus context compared with the treatment context. A connectivity analysis revealed negative coupling between the anterior PFC and the VS in the treatment context, suggesting that the PFC can suppress the expression of prediction errors in the VS. Consistent with this, a participant's conceptual views and beliefs about treatments influenced the pain modulation only in the treatment context. Our results indicate that in placebo hypoalgesia contextual treatment information engages prefrontal conceptual processes, which can suppress prediction error processing in the VS and lead to reduced updating of treatment expectancies, resulting in less extinction of placebo hypoalgesia.SIGNIFICANCE STATEMENT In aversive and appetitive reinforcement learning, learned effects show extinction when reinforcement is discontinued. This is thought to be mediated by prediction errors (i.e., the difference between expectations and outcome). Although reinforcement learning has been central in explaining placebo hypoalgesia, placebo hypoalgesic effects show little extinction and persist after the discontinuation of reinforcement. Our results support the idea that conceptual treatment beliefs bias the neural processing of expectations in a treatment context compared with a more stimulus-driven processing of expectations with stimulus intensity cues. We provide evidence that this is associated with the suppression of prediction error processing in the ventral striatum by the prefrontal cortex. This provides a neural basis for persisting effects in reinforcement learning and placebo hypoalgesia.

“An Incomplete Theory of the Mind”

Facing your fears, or exposure therapy, is an effective psychological intervention for anxiety disorders that is often thought to work through fear extinction learning. Fear extinction learning is a type of associative learning where fear reduces through repeated encounters with a feared situation or stimulus in the absence of aversive outcomes. Laboratory research suggests fear extinction learning is driven by threat prediction errors, defined as when fearful predictions do not eventuate. Threat prediction error and its relationship to exposure therapy outcomes haven't been studied enough in actual therapy settings. It remains unclear whether prediction error and extinction learning are central mechanisms of exposure therapy. We are conducting a longitudinal and observational study of how threat prediction error during exposure in social anxiety disorder (SAD) treatment relates to session-by-session symptom change and treatment outcome in addition to exposure surprise and learning outcome. We aim to recruit 65 adults with a primary diagnosis of SAD through an outpatient psychology clinic. Participants will receive 12 sessions of individual manualized cognitive behavioral therapy (CBT), adapted from an efficacious group protocol, that includes graded exposure. Exposure processes, including self-report measures of anxiety, threat prediction, threat outcomes, surprise, and learning outcome, will be measured with smartphone-based event-contingent ecological momentary assessments (EMAs) of all behavioral experiments completed during treatment. Clinical outcomes include self-reported social anxiety symptoms and social threat appraisals, at each session, post and 3-months after treatment. Prediction error will be operationalized as the mismatch between the threat prediction and threat outcome. The joint effect of threat prediction and threat outcome on session-by-session symptom change, treatment outcome, exposure surprise, and learning outcome will be explored using multilevel modeling. The present study will help determine whether threat prediction error during exposures in SAD treatment is related to theoretically implied clinical outcomes. This would contribute to the larger research aim of clarifying exposure therapy mechanisms.