Reward Learning Task Research Articles

Decoding threat neurocircuitry representations during traumatic memory recall in PTSD.

The neurocircuitry mechanisms underlying recall of traumatic memories remain unclear. This study investigated whether traumatic memory recall engages neurocircuitry representations that mirror activity patterns engaged during generalized threat stimulus processing in Post Traumatic Stress Disorder (PTSD). Multivariate pattern analysis was used to train 3 decoders. A "trauma" decoder was trained on fMRI patterns during idiographic trauma versus neutral narratives in a sample of 73 adult women with PTSD. A separate cohort of 125 adult participants completed a reward and threat learning task, from which "shock" and "reward loss" decoders were trained on neural patterns during threat or reward outcome delivery, respectively. These decoders were then cross-tested on the alternative datasets, allowing analyses of the degree to which traumatic memory recall engaged neurocircuitry representations that overlap with more general aversive stimuli. Decoders were trained and tested in four networks related to salience processing as well bilateral amygdala and hippocampal masks. The shock decoder trained in a midcingulate / posterior insula network demonstrated elevated predictions for shock during traumatic versus neutral memory recall. Similarly, the trauma decoder made elevated predictions about trauma recall during shock versus no shock delivery across multiple networks related to salience processing. There was no overlap between reward loss decoder predictions and trauma memory recall or vice versa. PTSD participants with elevated re-experiencing symptoms demonstrated the highest engagement of shock activity patterns during trauma memory recall. These results suggest that trauma memory recall engages neurocircuitry representations that overlap with threat, specifically painful, stimulus delivery.

Atypical pupil-linked arousal induced by low-risk probabilistic choices, and intolerance of uncertainty in adults with ASD.

Adults with autism spectrum disorder (ASD) experience stress when operating in a probabilistic environment, even if it is familiar, but the underlying mechanisms remain unclear. Their decision-making may be affected by the uncertainty aversion implicated in ASD and associated with increased autonomic arousal. Previous studies have shown that in neurotypical (NT) people, decisions with predictably better outcomes are less stressful and elicit smaller pupil-linked arousal than those involving exploration. Here, in a sample of 46 high-functioning ASD and NT participants, using mixed-effects model analysis, we explored pupil-linked arousal and behavioral performance in a probabilistic reward learning task with a stable advantage of one choice option over the other. We found that subjects with ASD learned and preferred advantageous probabilistic choices at the same rate and to the same extent as NT participants, both in terms of choice ratio and response time. Although both groups exhibited similar predictive behaviors, learning to favor advantageous choices led to increased pupillary arousal for these choices in the ASD group, while it caused a decrease in pupillary arousal in the NT group. Moreover, greater pupil-linked arousal during decisionswithhigherexpected value correlated with greater degree of self-reported intolerance of uncertainty in everyday life. Our results suggest that in a nonvolatile probabilistic environment, objectively good predictive abilities in people with ASD are coupled with elevated physiological stress and subjective uncertainty regarding the decisions with the best possible but still uncertain outcome that contributes to their intolerance of uncertainty.

B - 62 Cognitive Training Targeting Reward Responsiveness Promotes more Efficient Affective Processing in an Interference Go/NoGo Task

Abstract Objective Abnormalities in emotional processing are commonly seen in depression and anxiety, disorders also associated with decreased reward sensitivity i.e., anhedonia. We tested a noninvasive, cognitive training program aimed at boosting reward responsiveness and gaged if it may also improve affective processing efficiency. Methods Individuals (N = 50; 54% Female; 18–65 yo) with low hedonic tone (Positive and Negative Affect Scale/PANAS score < 25) and diagnostic levels of depression and/or anxiety (Patient Health Questionnaire-8/PHQ-8 or General Anxiety Disorder-7/GAD-7 score ≥ 10) who responded to community-posted flyers completed an affective go/no-go (AGN) task before and after completing an exploratory reward learning task, i.e., a multi-arm bandit (MAB). Participants were randomly assigned to a high-reward MAB training with either high variance (HV) or low variance (LV) in reward rates, HV being expected to boost reward maximization. In the AGN, participants saw negative and positive pictures and had to respond as fast as possible to pictures of a given target category (positive or negative). Results Following the HV vs LV training MAB, participants were more accurate (higher likelihood of hits), which was more evident in negative target blocks relative to positive target blocks (odds ratio/OR:0.61, 95% CI = [0.38,0.98]; z = −2.045, p = 0.041). Participants exhibited stronger decreases in hits response times following the HV vs LV training MAB, but across all target blocks (B:-37 ms, 95% CI = [−56,-18]; t = −3.806, p < 0.001). Conclusions A novel cognitive training program aimed at boosting reward sensitivity may help improve patients’ processing of emotionally taxing stimuli. Targeting reward sensitivity is a critical component of improving clinical outcomes in depressed and anxious patients..

College students with depression symptom are more sensitive to task difficulty in reinforcement learning

BackgroundDepression is usually characterized by impairments in reward function, and shows altered motivation to reward in reinforcement learning. This study further explored whether task difficulty affects reinforcement learning in college students with and without depression symptom. MethodsThe depression symptom group (20) and the no depression symptom group (26) completed a probabilistic reward learning task with low, medium, and high difficulty levels, in which task the response bias to reward and the discriminability of reward were analyzed. Additionally, electrophysiological responses to reward and loss feedback were recorded and analyzed while they performed a simple gambling task. ResultsThe depression symptom group showed more response bias to reward than the no depression symptom group when the task was easy and then exhibited more quickly decrease in response bias to reward as task difficulty increased. The no depression symptom group showed a decrease in response bias only in the high-difficulty condition. Further regression analyses showed that, the Feedback-related negativity (FRN) and theta oscillation could predict response bias change in the low-difficulty condition, the FRN and oscillations of theta and delta could predict response bias change in the medium and high-difficulty conditions. LimitationsThe electrophysiological responses to loss and reward were not recorded in the same task as the reinforcement learning behaviors. ConclusionsCollege students with depression symptom are more sensitive to task difficulty during reinforcement learning. The FRN, and oscillations of theta and delta could predict reward leaning behavior.

Electrical brain activations in preadolescents during a probabilistic reward-learning task reflect cognitive processes and behavioral strategy.

Both adults and children learn through feedback which environmental events and choices are associated with higher probability of reward, an ability thought to be supported by the development of fronto-striatal reward circuits. Recent developmental studies have applied computational models of reward learning to investigate such learning in children. However, tasks and measures effective for assaying the cascade of reward-learning neural processes in children have been limited. Using a child-version of a probabilistic reward-learning task while recording event-related-potential (ERP) measures of electrical brain activity, this study examined key processes of reward learning in preadolescents (8-12 years old; n=30), namely: (1) reward-feedback sensitivity, as measured by the early-latency, reward-related, frontal ERP positivity, (2) rapid attentional shifting of processing toward favored visual stimuli, as measured by the N2pc component, and (3) longer-latency attention-related responses to reward feedback as a function of behavioral strategies (i.e., Win-Stay-Lose-Shift), as measured by the central-parietal P300. Consistent with our prior work in adults, the behavioral findings indicate preadolescents can learn stimulus-reward outcome associations, but at varying levels of performance. Neurally, poor preadolescent learners (those with slower learning rates) showed greater reward-related positivity amplitudes relative to good learners, suggesting greater reward-feedback sensitivity. We also found attention shifting towards to-be-chosen stimuli, as evidenced by the N2pc, but not to more highly rewarded stimuli as we have observed in adults. Lastly, we found the behavioral learning strategy (i.e., Win-Stay-Lose-Shift) reflected by the feedback-elicited parietal P300. These findings provide novel insights into the key neural processes underlying reinforcement learning in preadolescents.

Deterioration in cognitive control related mPFC function underlying development of treatment resistance in early psychosis

One third of people with psychosis become antipsychotic treatment-resistant and the underlying mechanisms remain unclear. We investigated whether altered cognitive control function is a factor underlying development of treatment resistance. We studied 50 people with early psychosis at a baseline visit (mean < 2 years illness duration) and follow-up visit (1 year later), when 35 were categorized at treatment-responsive and 15 as treatment-resistant. Participants completed an emotion-yoked reward learning task that requires cognitive control whilst undergoing fMRI and MR spectroscopy to measure glutamate levels from Anterior Cingulate Cortex (ACC). Changes in cognitive control related activity (in prefrontal cortex and ACC) over time were compared between treatment-resistant and treatment-responsive groups and related to glutamate. Compared to treatment-responsive, treatment-resistant participants showed blunted activity in right amygdala (decision phase) and left pallidum (feedback phase) at baseline which increased over time and was accompanied by a decrease in medial Prefrontal Cortex (mPFC) activity (feedback phase) over time. Treatment-responsive participants showed a negative relationship between mPFC activity and glutamate levels at follow-up, no such relationship existed in treatment-resistant participants. Reduced activity in right amygdala and left pallidum at baseline was predictive of treatment resistance at follow-up (67% sensitivity, 94% specificity). The findings suggest that deterioration in mPFC function over time, a key cognitive control region needed to compensate for an initial dysfunction within a social-emotional network, is a factor underlying development of treatment resistance in early psychosis. An uncoupling between glutamate and cognitive control related mPFC function requires further investigation that may present a future target for interventions.

The Affective Bias Test and Reward Learning Assay: Neuropsychological Models for Depression Research and Investigating Antidepressant Treatments in Rodents.

The Affective Bias Test (ABT) quantifies acute changes in affective state based on the affective biases they generate in an associative reward learning task. The Reward Learning Assay (RLA) provides a control assay for the ABT and reward-induced biases generated in this model are sensitive to changes in core affective state. Both tasks involve training animals to associate a specific digging substrate with a food reward. Animals learn to discriminate between two digging substrates placed in ceramic bowls, one rewarded and one unrewarded. In the ABT, the animal learns two independent substrate-reward associations with a fixed reward value following either an affective state or drug manipulation, or under control conditions. Affective biases generated are quantified in a choice test where the animals exhibit a bias (make more choices) for one of the substrates which is specifically related to affective state at the time of learning. The ABT is used to investigate biases generated during learning as well as modulation of biases associated with past experiences. The RLA follows a similar protocol, but the animal remains in the same affective state throughout and a reward-induced bias is generated by pairing one substrate with a higher value reward. The RLA provides a control to determine if drug treatments affect memory retrieval more generally. Studies in depression models and following environmental enrichment suggest that reward-induced biases are sensitive to core changes in affective state. Each task offers different insights into affective processing mechanisms and may help improve the translational validity of animal studies and benefit pre-clinical drug development. © 2024 The Authors. Current Protocols published by Wiley Periodicals LLC. Basic Protocol 1: Bowl digging and discrimination training Basic Protocol 2: The reward learning assay Basic Protocol 3: The affective bias test - new learning Basic Protocol 4: The affective bias test - modulation of affective biases associated with past experiences.

Reward Sensitivity and Noise Contribute to Negative Affective Bias: A Learning Signal Detection Theory Approach in Decision-Making.

In patients with mood disorders, negative affective biases - systematically prioritising and interpreting information negatively - are common. A translational cognitive task testing this bias has shown that depressed patients have a reduced preference for a high reward under ambiguous decision-making conditions. The precise mechanisms underscoring this bias are, however, not yet understood. We therefore developed a set of measures to probe the underlying source of the behavioural bias by testing its relationship to a participant's reward sensitivity, value sensitivity and reward learning rate. One-hundred-forty-eight participants completed three online behavioural tasks: the original ambiguous-cue decision-making task probing negative affective bias, a probabilistic reward learning task probing reward sensitivity and reward learning rate, and a gambling task probing value sensitivity. We modelled the learning task through a dynamic signal detection theory model and the gambling task through an expectation-maximisation prospect theory model. Reward sensitivity from the probabilistic reward task (β = 0.131, p = 0.024) and setting noise from the probabilistic reward task (β = -0.187, p = 0.028) both predicted the affective bias score in a logistic regression. Increased negative affective bias, at least on this specific task, may therefore be driven in part by a combination of reduced sensitivity to rewards and more variable responses.

42. Posttraumatic Stress Disorder (PTSD) is Associated With Abnormal Amygdala and Striatal Signaling of Temporal Prediction Errors During a Passive Reward Learning Task

42. Posttraumatic Stress Disorder (PTSD) is Associated With Abnormal Amygdala and Striatal Signaling of Temporal Prediction Errors During a Passive Reward Learning Task

How False Memory and True Memory Affect Decision Making in Older Adults: A Dissociative Account.

Remembering past rewarding experiences plays a crucial rule in guiding people's decision making in the future. However, as people age, they become less accurate in remembering past events and more susceptible to forming false memories. An important question is how the decline of episodic memory and the increase of false memory may affect older adults' decision-making performance. The current study used a newly developed paradigm in which the Deese-Roediger-McDermott (DRM) false memory paradigm was combined with a reward learning task to create robust false memories of rewarding experiences. Participants learned that some DRM picture lists brought them a monetary reward and some DRM picture lists did not bring reward. Later, their memories were tested and decision-making preferences were measured. We found that older and younger adults had almost equivalent false and true memories under the rewarding context, but older adults showed significantly lower decision-making preferences for lure pictures and rewarded pictures than younger adults. Furthermore, true and false memories were a stronger predictor of decision-making preferences for younger than for older adults. These results together suggest an age-related dissociation between memory and decision making that older adults may be less efficient in using their memory to guide decision making than younger adults. Future research may further investigate its underlying mechanisms and develop potential interventions aiming at strengthening the connection between memory and decision making in older adults to help improve their decision-making performance.

Ghrelin decreases sensitivity to negative feedback and increases prediction-error related caudate activity in humans, a randomized controlled trial

The stomach-derived hormone ghrelin plays not only a role in feeding, starvation, and survival, but it has been suggested to also be involved in the stress response, in neuropsychiatric conditions, and in alcohol and drug use disorders. Mechanisms related to reward processing might mediate ghrelin’s broader effects on complex behaviors, as indicated by animal studies and mostly correlative human studies. Here, using a within-subject double-blind placebo-controlled design with intravenous ghrelin infusion in healthy volunteers (n = 30), we tested whether ghrelin alters sensitivity to reward and punishment in a reward learning task. Parameters were derived from a computational model of participants’ task behavior. The reversal learning task with monetary rewards was performed during functional brain imaging to investigate ghrelin effects on brain signals related to reward prediction errors. Compared to placebo, ghrelin decreased punishment sensitivity (t = −2.448, p = 0.021), while reward sensitivity was unaltered (t = 0.8, p = 0.43). We furthermore found increased prediction-error related activity in the dorsal striatum during ghrelin administration (region of interest analysis: t-values ≥ 4.21, p-values ≤ 0.044). Our results support a role for ghrelin in reward processing that extends beyond food-related rewards. Reduced sensitivity to negative outcomes and increased processing of prediction errors may be beneficial for food foraging when hungry but could also relate to increased risk taking and impulsivity in the broader context of addictive behaviors.

Reinforcement-Learning-Informed Queries Guide Behavioral Change.

Algorithmically defined aspects of reinforcement learning correlate with psychopathology symptoms and change with symptom improvement following cognitive-behavioral therapy (CBT). Separate work in nonclinical samples has shown that varying the structure and statistics of task environments can change learning. Here, we combine these literatures, drawing on CBT-based guided restructuring of thought processes and computationally defined mechanistic targets identified by reinforcement-learning models in depression, to test whether and how verbal queries affect learning processes. Using a parallel-arm design, we tested 1,299 online participants completing a probabilistic reward-learning task while receiving repeated queries about the task environment (11 learning-query arms and one active control arm). Querying participants about reinforcement-learning-related task components altered computational-model-defined learning parameters in directions specific to the target of the query. These effects on learning parameters were consistent across depression-symptom severity, suggesting new learning-based strategies and therapeutic targets for evoking symptom change in mood psychopathology.

Creating false rewarding memories guides novel decision making.

When memories of past rewarding experiences are distorted, are relevant decision-making preferences impacted? Although recent research has demonstrated the important role of episodic memory in value-based decision making, very few have examined the role of false memory in guiding novel decision making. The current study combined the pictorial Deese/Roediger-McDermott false memory paradigm with a reward learning task, where participants learned that items from some related lists gained reward and items from other lists led to no reward. Later, participants' memories and decision-making preferences were tested. With three experiments conducted in three countries, we successfully created false memories of rewarding experiences in which participants falsely remembered seeing a nonpresented lure picture bring them reward thereby confirming our constructive association hypothesis. Such false memories led participants to prefer the lure pictures and respond faster in a follow-up decision-making task, and the more false memories they formed, the higher preferences for the lure items they displayed (Experiment 2). Finally, results were replicated with or without a memory test before the decision-making task, showing that the impact of false memory on decision making was not cued by a memory test (Experiment 3). Our data suggest that the reconstructive nature of memory enables individuals to create new memory episodes to guide decision making in novel situations. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Anxiety associated with perceived uncontrollable stress enhances expectations of environmental volatility and impairs reward learning

Unavoidable stress can lead to perceived lack of control and learned helplessness, a risk factor for depression. Avoiding punishment and gaining rewards involve updating the values of actions based on experience. Such updating is however useful only if action values are sufficiently stable, something that a lack of control may impair. We examined whether self-reported stress uncontrollability during the first wave of the COVID-19 pandemic predicted impaired reward-learning. In a preregistered study during the first-wave of the COVID-19 pandemic, we used self-reported measures of depression, anxiety, uncontrollable stress, and COVID-19 risk from 427 online participants to predict performance in a three-armed-bandit probabilistic reward learning task. As hypothesised, uncontrollable stress predicted impaired learning, and a greater proportion of probabilistic errors following negative feedback for correct choices, an effect mediated by state anxiety. A parameter from the best-fitting hidden Markov model that estimates expected beliefs that the identity of the optimal choice will shift across images, mediated effects of state anxiety on probabilistic errors and learning deficits. Our findings show that following uncontrollable stress, anxiety promotes an overly volatile representation of the reward-structure of uncertain environments, impairing reward attainment, which is a potential path to anhedonia in depression.

Reliability of gamified reinforcement learning in densely sampled longitudinal assessments.

Reinforcement learning is a core facet of motivation and alterations have been associated with various mental disorders. To build better models of individual learning, repeated measurement of value-based decision-making is crucial. However, the focus on lab-based assessment of reward learning has limited the number of measurements and the test-retest reliability of many decision-related parameters is therefore unknown. In this paper, we present an open-source cross-platform application Influenca that provides a novel reward learning task complemented by ecological momentary assessment (EMA) of current mental and physiological states for repeated assessment over weeks. In this task, players have to identify the most effective medication by integrating reward values with changing probabilities to win (according to random Gaussian walks). Participants can complete up to 31 runs with 150 trials each. To encourage replay, in-game screens provide feedback on the progress. Using an initial validation sample of 384 players (9729 runs), we found that reinforcement learning parameters such as the learning rate and reward sensitivity show poor to fair intra-class correlations (ICC: 0.22-0.53), indicating substantial within- and between-subject variance. Notably, items assessing the psychological state showed comparable ICCs as reinforcement learning parameters. To conclude, our innovative and openly customizable app framework provides a gamified task that optimizes repeated assessments of reward learning to better quantify intra- and inter-individual differences in value-based decision-making over time.

Prefrontal cortex state representations shape human credit assignment.

People learn adaptively from feedback, but the rate of such learning differs drastically across individuals and contexts. Here, we examine whether this variability reflects differences in what is learned. Leveraging a neurocomputational approach that merges fMRI and an iterative reward learning task, we link the specificity of credit assignment-how well people are able to appropriately attribute outcomes to their causes-to the precision of neural codes in the prefrontal cortex (PFC). Participants credit task-relevant cues more precisely in social compared vto nonsocial contexts, a process that is mediated by high-fidelity (i.e., distinct and consistent) state representations in the PFC. Specifically, the medial PFC and orbitofrontal cortex work in concert to match the neural codes from feedback to those at choice, and the strength of these common neural codes predicts credit assignment precision. Together this work provides a window into how neural representations drive adaptive learning.

Reward System in Late-Life Depression: a Cross-Sectional Case-Control Study.

Anhedonia, commonly defined as a reduced ability to feel pleasure, is a core clinical symptom of late-life depression (LLD). Deficits in reward processing are hypothesised to be associated with anhedonia. We examined differences in reward sensitivity between patients with LLD and healthy controls and explored the associations between LLD-related symptomatology, global cognition, and the reward system. The reward responsiveness of 63 patients with LLD and 58 healthy controls aged ≥60 years was assessed using the probabilistic reward learning task with an asymmetric reward schedule. Compared with healthy controls, patients with LLD displayed lower response bias and reward learning. Global cognition of all participants was positively correlated with response bias. In patients with LLD, anhedonia severity explained impaired reward learning. A deficit in reward processing is implicated in patients with LLD. Our findings suggest that executive dysfunction and anhedonia contribute to lower sensitivity to reward learning in patients with LLD.

Diminished prospective mental representations of reward mediate reward learning strategies among youth with internalizing symptoms.

Adolescent internalizing symptoms and trauma exposure have been linked with altered reward learning processes and decreased ventral striatal responses to rewarding cues. Recent computational work on decision-making highlights an important role for prospective representations of the imagined outcomes of different choices. This study tested whether internalizing symptoms and trauma exposure among youth impact the generation of prospective reward representations during decision-making and potentially mediate altered behavioral strategies during reward learning. Sixty-one adolescent females with varying exposure to interpersonal violence exposure (n = 31 with histories of physical or sexual assault) and severity of internalizing symptoms completed a social reward learning task during fMRI. Multivariate pattern analyses (MVPA) were used to decode neural reward representations at the time of choice. MVPA demonstrated that rewarding outcomes could accurately be decoded within several large-scale distributed networks (e.g. frontoparietal and striatum networks), that these reward representations were reactivated prospectively at the time of choice in proportion to the expected probability of receiving reward, and that youth with behavioral strategies that favored exploiting high reward options demonstrated greater prospective generation of reward representations. Youth internalizing symptoms, but not trauma exposure characteristics, were negatively associated with both the behavioral strategy of exploiting high reward options as well as the prospective generation of reward representations in the striatum. These data suggest diminished prospective mental simulation of reward as a mechanism of altered reward learning strategies among youth with internalizing symptoms.

Distinct replay signatures for prospective decision-making and memory preservation

Theories of neural replay propose that it supports a range of functions, most prominently planning and memory consolidation. Here, we test the hypothesis that distinct signatures of replay in the same task are related to model-based decision-making ("planning") and memory preservation. We designed a reward learning task wherein participants utilized structure knowledge for model-based evaluation, while at the same time had to maintain knowledge of two independent and randomly alternating task environments. Using magnetoencephalography and multivariate analysis, we first identified temporally compressed sequential reactivation, or replay, both prior to choice and following reward feedback. Before choice, prospective replay strength was enhanced for the current task-relevant environment when a model-based planning strategy was beneficial. Following reward receipt, and consistent with a memory preservation role, replay for the alternative distal task environment was enhanced as a function of decreasing recency of experience with that environment. Critically, these planning and memory preservation relationships were selective to pre-choice and post-feedback periods, respectively. Our results provide support for key theoretical proposals regarding the functional role of replay and demonstrate that the relative strength of planning and memory-related signals are modulated by ongoing computational and task demands.

Striatal dopamine supports reward expectation and learning: A simultaneous PET/fMRI study

Converging evidence from both human neuroimaging and animal studies has supported a model of mesolimbic processing underlying reward learning behaviors, based on the computation of reward prediction errors. However, competing evidence supports human dopamine signaling in the basal ganglia as also contributing to the generation of higher order learning heuristics. Here, we present data from a large (N = 81, 18–30yo), multi-modal neuroimaging study using simultaneously acquired task fMRI, affording temporal resolution of reward system function, and PET imaging with [11C]Raclopride (RAC), assessing striatal dopamine (DA) D2/3 receptor binding, during performance of a probabilistic reward learning task. Both fMRI activation and PET DA measures showed ventral striatum involvement for signaling rewards. However, greater DA release was uniquely associated with learning strategies (i.e., learning rates) that were more task-optimal within the best fitting reinforcement learning model. This DA response was associated with BOLD activation of a network of regions including anterior cingulate cortex, medial prefrontal cortex, thalamus and posterior parietal cortex, primarily during expectation, rather than prediction error, task epochs. Together, these data provide novel, human in vivo evidence that striatal dopaminergic signaling interacts with a network of cortical regions to generate task-optimal learning strategies, rather than representing reward outcomes in isolation.