Probabilistic Decision-making Task Research Articles

Effects of Adult Age and Functioning of the Locus Coeruleus Norepinephrinergic System on Reward-Based Learning.

Age-related impairments in value representations and updating during decision-making and reward-based learning are often related to age-related attenuation in the catecholamine system such as dopamine (DA) and norepinephrine (NE). However, it is unclear to what extent age-related declines in NE functioning in humans affect reward-based decision-making. We conducted a probabilistic decision-making task and applied a Q-learning model to investigate participants' anticipatory values and value sensitivities. Task-related pupil dilations and locus coeruleus (LC) magnetic resonance imaging (MRI) contrast, which served as a potential window of the LC-NE functions, were assessed in younger and older adults. Results showed that in both choice and feedback phases, younger adults' (N = 42, 22 males) pupil dilations negatively correlated with anticipatory values, indicating uncertainty about outcome probabilities. Uncertainty-evoked pupil dilations in older adults (N = 41, 27 males) were smaller, indicating age-related impairments in value estimation and updating. In both age groups, participants who showed a larger uncertainty-evoked pupil dilation exhibited a higher value sensitivity as reflected in the β parameter of the reinforcement Q-learning model. Furthermore, older adults (N = 34, 29 males) showed a lower LC-MRI contrast than younger adults (N = 25, 15 males). The LC-MRI contrast positively correlated with value sensitivity only in older but not in younger adults. These findings suggest that task-related pupillary responses can reflect age-related deficits in value estimation and updating during reward-based decision-making. Our evidence with the LC-MRI contrast further showed the age-related decline of the LC structure in modulating value representations during reward-based learning.SIGNIFICANCE STATEMENT Age-related impairments in value representation and updating during reward-based learning are associated with declines in the catecholamine modulation with age. However, it is unclear how age-related declines in the LC-NE system may affect reward-based learning. Here, we show that compared with younger adults, older adults exhibited reduced uncertainty-induced pupil dilations, suggesting age-related deficits in value estimation and updating. Older adults showed a lower structural MRI of the LC contrast than younger adults, indicating age-related degeneration of the LC structure. The association between the LC-MRI contrast and value sensitivity was only observed in older adults. Our findings may demonstrate a pioneering model to unravel the role of the LC-NE system in reward-based learning in aging.

The Effects of Tinnitus in Probabilistic Learning Tasks: Protocol for an Ecological Momentary Assessment Study.

Chronic tinnitus is an increasing worldwide health concern, causing a significant burden to the health care system each year. The COVID-19 pandemic has seen a further increase in reported cases. For people with tinnitus, symptoms are exacerbated because of social isolation and the elevated levels of anxiety and depression caused by quarantines and lockdowns. Although it has been reported that patients with tinnitus can experience changes in cognitive capabilities, changes in adaptive learning via decision-making tasks for people with tinnitus have not yet been investigated. In this study, we aim to assess state- and trait-related impairments in adaptive learning ability on probabilistic learning tasks among people with tinnitus. Given that performance in such tasks can be quantified through computational modeling methods using a small set of neural-informed model parameters, such approaches are promising in terms of the assessment of tinnitus severity. We will first examine baseline differences in the characterization of decision-making under uncertainty between healthy individuals and people with tinnitus in terms of differences in the parameters of computational models in a cross-sectional experiment. We will also investigate whether these computational markers, which capture characteristics of decision-making, can be used to understand the cognitive impact of tinnitus symptom fluctuations through a longitudinal experimental design. We have developed a mobile app, AthenaCX, to deliver e-consent and baseline tinnitus and psychological assessments as well as regular ecological momentary assessments (EMAs) of perceived tinnitus loudness and a web-based aversive version of a probabilistic decision-making task, which can be triggered based on the participants' responses to the EMA surveys. Computational models will be developed to fit participants' choice data in the task, and cognitive parameters will be estimated to characterize participants' current ability to adapt learning to the change of the simulated environment at each session when the task is triggered. Linear regression analysis will be conducted to evaluate the impacts of baseline tinnitus severity on adapting decision-making performance. Repeated measures linear regression analysis will be used to examine model-derived parameters of decision-making in measuring real-time perceived tinnitus loudness fluctuations. Ethics approval was received in December 2020 from Dublin City University (DCUREC/2021/070). The implementation of the experiments, including both the surveys and the web-based decision-making task, has been prepared. Recruitment flyers have been shared with audiologists, and a video instruction has been created to illustrate to the participants how to participate in the experiment. We expect to finish data collection over 12 months and complete data analysis 6 months after this. The results are expected to be published in December 2023. We believe that EMA with context-aware triggering can facilitate a deeper understanding of the effects of tinnitus symptom severity upon decision-making processes as measured outside of the laboratory. PRR1-10.2196/36583.

P069 Sleep Restriction Impairs Integration of Multiple Information Sources in Probabilistic Decision-Making

Abstract Introduction Sleep loss affects overall decision-making, but its effects on specific mechanisms of decision-making are less well-understood. Here, we examined the effects of sleep restriction (SR) on the ability to integrate multiple information sources on a probabilistic decision-making task. Methods We recruited 41 healthy participants (n=41; age=27.9±6.0years, 20F), 27 of whom underwent well-rested ( WR 4 nights of 9-hours time-in-bed) and SR (3 nights of 3-hours time-in-bed) conditions in counterbalanced order. On the last day of each lab stay, participants completed a probabilistic decision-making task twice – 2-hours after wake (AM) and 12-hours since habitual wake (PM). On each trial, participants were shown two boxes with different proportions of black/white balls. Eight balls were drawn randomly, with replacement, from an unknown box. Participants determined the selected box using easier-to-understand numerical odds and harder-to-understand draw outcome information. Mathematical modelling was used to infer weights placed on each source of information. Results For AM and PM sessions, participants placed significantly greater weights on odds relative to draw information during WR (p<.001 for both). For AM trials, SR significantly reduced weights given to draw information (p=.041), and reduced weights given to odds at marginally significant levels (p=.062). For PM trials, SR reduced weights given to draw information at marginally significant levels (p=.059). Conclusion While well-rested individuals generally rely more on easier-to-understand information, SR impairs decision-making by further reducing individuals’ ability to consider more complex information. This has implications for high-risk professions where SR is common (e.g. doctors) and impaired decision-making may pose serious consequences.

Aversive anticipations modulate electrocortical correlates of decision-making and reward reversal learning, but not behavioral performance.

Predicting the consequences of one’s own decisions is crucial for organizing future behavior. However, when reward contingencies vary frequently, flexible adaptation of decisions is likely to depend on the situation. We examined the effects of an instructed threat context on choice behavior (i.e., reversal learning) and its electrocortical correlates. In a probabilistic decision-making task, 30 participants had to choose between two options that were either contingent on monetary gains or losses. Reward contingencies were reversed after reaching a probabilistic threshold. Decision-making and reversal learning were examined with two contextual background colors, which were instructed as signals for threat-of-shock or safety. Self-report data confirmed the threat context as more unpleasant, arousing, and threatening relative to safety condition. However, against our expectations, behavioral performance was comparable during the threat and safety conditions (i.e., errors-to-criterion, number of reversal, error rates, and choice times). Regarding electrocortical activity, feedback processing changed throughout the visual processing stream. The feedback-related negativity (FRN) reflected expectancy-driven processing (unexpected vs. congruent losses and gains), and the threat-selective P3 component revealed non-specific discrimination of gains vs. losses. Finally, the late positive potentials (LPP) showed strongly valence-specific processing (unexpected and congruent losses vs. gains). Thus, regardless of contextual threat, early and late cortical activity reflects an attentional shift from expectation- to outcome-based feedback processing. Findings are discussed in terms of reward, threat, and reversal-learning mechanisms with implications for emotion regulation and anxiety disorders.

No increased circular inference in adults with high levels of autistic traits or autism.

Autism spectrum disorders have been proposed to arise from impairments in the probabilistic integration of prior knowledge with sensory inputs. Circular inference is one such possible impairment, in which excitation-to-inhibition imbalances in the cerebral cortex cause the reverberation and amplification of prior beliefs and sensory information. Recent empirical work has associated circular inference with the clinical dimensions of schizophrenia. Inhibition impairments have also been observed in autism, suggesting that signal reverberation might be present in that condition as well. In this study, we collected data from 21 participants with self-reported diagnoses of autism spectrum disorders and 155 participants with a broad range of autistic traits in an online probabilistic decision-making task (the fisher task). We used previously established Bayesian models to investigate possible associations between autistic traits or autism and circular inference. There was no correlation between prior or likelihood reverberation and autistic traits across the whole sample. Similarly, no differences in any of the circular inference model parameters were found between autistic participants and those with no diagnosis. Furthermore, participants incorporated information from both priors and likelihoods in their decisions, with no relationship between their weights and psychiatric traits, contrary to what common theories for both autism and schizophrenia would suggest. These findings suggest that there is no increased signal reverberation in autism, despite the known presence of excitation-to-inhibition imbalances. They can be used to further contrast and refine the Bayesian theories of schizophrenia and autism, revealing a divergence in the computational mechanisms underlying the two conditions.

Spared performance but increased uncertainty in schizophrenia: Evidence from a probabilistic decision-making task

Aberrant attribution of salience to in fact little informative events might explain the emergence of positive symptoms in schizophrenia and has been linked to belief uncertainty. Uncertainty is thought to be encoded by neuromodulators, including norepinephrine. However, norepinephrinergic encoding of uncertainty, measured as task-related pupil dilation, has rarely been explored in schizophrenia. Here, we addressed this question by comparing individuals with a disorder from the schizophrenia spectrum to a non-psychiatric control group on behavioral and pupillometric measures in a probabilistic prediction task, where different levels of uncertainty were introduced. Behaviorally, patients performed similar to controls, but their belief uncertainty was higher, particularly when instability of the task environment was high, suggesting an increased sensitivity to this instability. Furthermore, while pupil dilation scaled positively with uncertainty, this was less the case for patients, suggesting aberrant neuromodulatory regulation of neural gain, which may hinder the reduction of uncertainty in the long run. Together, the findings point to abnormal uncertainty processing and norepinephrinergic signaling in schizophrenia, potentially informing future development of both psychopharmacological therapies and psychotherapeutic approaches that deal with the processing of uncertain information.

Individual differences in experienced and observational decision-making illuminate interactions between reinforcement learning and declarative memory

Decision making can be shaped both by trial-and-error experiences and by memory of unique contextual information. Moreover, these types of information can be acquired either by means of active experience or by observing others behave in similar situations. The interactions between reinforcement learning parameters that inform decision updating and memory formation of declarative information in experienced and observational learning settings are, however, unknown. In the current study, participants took part in a probabilistic decision-making task involving situations that either yielded similar outcomes to those of an observed player or opposed them. By fitting alternative reinforcement learning models to each subject, we discerned participants who learned similarly from experience and observation from those who assigned different weights to learning signals from these two sources. Participants who assigned different weights to their own experience versus those of others displayed enhanced memory performance as well as subjective memory strength for episodes involving significant reward prospects. Conversely, memory performance of participants who did not prioritize their own experience over others did not seem to be influenced by reinforcement learning parameters. These findings demonstrate that interactions between implicit and explicit learning systems depend on the means by which individuals weigh relevant information conveyed via experience and observation.

Low Cognitive Impulsivity Is Associated with Better Gain and Loss Learning in a Probabilistic Decision-Making Task.

Many advances have been made over the last decades in describing, on the one hand, the link between reward-based learning and decision-making, and on the other hand, the link between impulsivity and decision-making. However, the association between reward-based learning and impulsivity remains poorly understood. In this study, we evaluated the association between individual differences in loss-minimizing and gain-maximizing behavior in a learning-based probabilistic decision-making task and individual differences in cognitive impulsivity. We found that low cognitive impulsivity was associated both with a better performance minimizing losses and maximizing gains during the task. These associations remained significant after controlling for mathematical skills and gender as potential confounders. We discuss potential mechanisms through which cognitive impulsivity might interact with reward-based learning and decision-making.

Risky decision-making in adolescent girls: The role of pubertal hormones and reward circuitry

Adolescence is a developmental period characterized by a greater tendency to take risks. While the adult literature has shown that sex steroids influence reward-related brain functioning and risk taking, research on the role of these hormones during puberty is limited. In this study, we examined the relation between pubertal hormones and adolescent risk taking using a probabilistic decision-making task. In this task, participants could choose on each trial to play or pass based on explicit information about the risk level and stakes involved in their decision. We administered this task to 58 11-to-13-year-old girls while functional MRI images were obtained to examine reward-related brain processes associated with their risky choices. Results showed that higher testosterone levels were associated with increased risk taking, which was mediated by increased medial orbitofrontal cortex activation. Furthermore, higher estradiol levels were associated with increased nucleus accumbens activation, which in turn related to decreased risk taking. These findings offer potential neuroendocrine mechanisms that can explain why some adolescent girls might engage in more risk taking compared to others.

Cortical Brain Activity Reflecting Attentional Biasing Toward Reward-Predicting Cues Covaries with Economic Decision-Making Performance.

Adaptive choice behavior depends critically on identifying and learning from outcome-predicting cues. We hypothesized that attention may be preferentially directed toward certain outcome-predicting cues. We studied this possibility by analyzing event-related potential (ERP) responses in humans during a probabilistic decision-making task. Participants viewed pairs of outcome-predicting visual cues and then chose to wager either a small (i.e., loss-minimizing) or large (i.e., gain-maximizing) amount of money. The cues were bilaterally presented, which allowed us to extract the relative neural responses to each cue by using a contralateral-versus-ipsilateral ERP contrast. We found an early lateralized ERP response, whose features matched the attention-shift-related N2pc component and whose amplitude scaled with the learned reward-predicting value of the cues as predicted by an attention-for-reward model. Consistently, we found a double dissociation involving the N2pc. Across participants, gain-maximization positively correlated with the N2pc amplitude to the most reliable gain-predicting cue, suggesting an attentional bias toward such cues. Conversely, loss-minimization was negatively correlated with the N2pc amplitude to the most reliable loss-predicting cue, suggesting an attentional avoidance toward such stimuli. These results indicate that learned stimulus-reward associations can influence rapid attention allocation, and that differences in this process are associated with individual differences in economic decision-making performance.

Phasic Dopamine Release in the Rat Nucleus Accumbens Symmetrically Encodes a Reward Prediction Error Term

Making predictions about the rewards associated with environmental stimuli and updating those predictions through feedback is an essential aspect of adaptive behavior. Theorists have argued that dopamine encodes a reward prediction error (RPE) signal that is used in such a reinforcement learning process. Recent work with fMRI has demonstrated that the BOLD signal in dopaminergic target areas meets both necessary and sufficient conditions of an axiomatic model of the RPE hypothesis. However, there has been no direct evidence that dopamine release itself also meets necessary and sufficient criteria for encoding an RPE signal. Further, the fact that dopamine neurons have low tonic firing rates that yield a limited dynamic range for encoding negative RPEs has led to significant debate about whether positive and negative prediction errors are encoded on a similar scale. To address both of these issues, we used fast-scan cyclic voltammetry to measure reward-evoked dopamine release at carbon fiber electrodes chronically implanted in the nucleus accumbens core of rats trained on a probabilistic decision-making task. We demonstrate that dopamine concentrations transmit a bidirectional RPE signal with symmetrical encoding of positive and negative RPEs. Our findings strengthen the case that changes in dopamine concentration alone are sufficient to encode the full range of RPEs necessary for reinforcement learning.

Impairment in risk-sensitive decision-making in older suicide attempters with depression.

Suicidal behavior is a potentially lethal complication of late-life depression. In younger adults, suicide has been linked to abnormal decision-making ability. Given that there are substantial age-related decreases in decision-making ability, and that older adults experience environmental stressors that require effective decision-making, we reasoned that impaired decision-making may be particularly relevant to suicidal behavior in the elderly. We thus compared performance on a probabilistic decision-making task that does not involve working memory ("Cambridge Gamble Task") in four groups of older adults: (1) individuals with major depression and a history of suicide attempt (n = 25), (2) individuals with major depression with active suicidal ideation but no suicide attempt (n = 13), (3) individuals with major depression without suicidality (n = 35), and (4) nondepressed control subjects (n = 22). There was a significant effect of group on quality of decision-making, whereby the suicide attempters exhibited poorer ability to choose the likely outcome, compared with the nonsuicidal depressed and nondepressed comparison subjects. There were no group differences in betting behavior. The suicide attempters differed in several aspects of social problem-solving on a self-report scale. Quality of decision-making was negatively correlated with the score on the impulsive/careless problem-solving subscale. These data suggest that older suicide attempters have a deficit in risk-sensitive decision-making, extending observations in younger adults. More specifically, older suicide attempters seem to neglect outcome probability and make poor choices. These impairments may precipitate and perpetuate suicidal crisis in depressed elders. Identification of decision-making impairment in suicidal elders may help with designing effective interventions.

Understanding the nature and outcome of childhood wheezing

<h3>Introduction</h3> Clinical depression is usually treated in primary care with psychological therapies and antidepressant medication. However, when patients do not respond to at least two or more antidepressants within a depressive episode, they are considered to have treatment resistant depression (TRD). Previous small randomised controlled trials suggested that pramipexole, a dopamine D2/3 receptor agonist, may be effective for treating patients with unipolar and bipolar depression as it is known to influence motivational drive and reward processing. PAX-D will compare the effects of pramipexole vs placebo when added to current antidepressant medication for people with TRD. Additionally, PAX-D will investigate the mechanistic effect of pramipexole on reward sensitivity using a probabilistic decision-making task. <h3>Methods and analysis</h3> PAX-D will assess effectiveness in the short- term (during the first 12 weeks) and in the longer-term (48 weeks) in patients with TRD from the UK. The primary outcome will be change in self-reported depressive symptoms from baseline to week 12 post-randomisation measured using the Quick Inventory of Depressive Symptomatology Self-Report (QIDS-SR16). Performance on the decision-making task will be measured at week 0, week 2 and week 12. Secondary outcomes include anhedonia, anxiety and health economic measures including quality of life, capability, well-being and costs. PAX-D will also assess the adverse effects of pramipexole including impulse control difficulties. <h3>Discussion</h3> Pramipexole is a promising augmentation agent for TRD and may be a useful addition to existing treatment regimes. PAX-D will assess its effectiveness and test for a potential mechanism of action in patients with TRD. <h3>Trial registration number</h3> ISRCTN84666271

Decision-Making Impairments in the Context of Intact Reward Sensitivity in Schizophrenia

Deficits in motivated behavior and decision-making figure prominently in the behavioral syndrome that characterizes schizophrenia and are difficult both to treat and to understand. One explanation for these deficits is that schizophrenia decreases sensitivity to rewards in the environment. An alternate explanation is that sensitivity to rewards is intact but that poor integration of affective with cognitive information impairs the ability to use this information to guide behavior. We tested reward sensitivity with a modified version of an existing signal detection task with asymmetric reinforcement and decision-making with a probabilistic decision-making task in 40 participants with schizophrenia and 26 healthy participants. Results showed normal sensitivity to reward in participants with schizophrenia but differences in choice patterns on the decision-making task. A logistic regression model of the decision-making data showed that participants with schizophrenia differed from healthy participants in the ability to weigh potential outcomes, specifically potential losses, when choosing between competing response options. Deficits in working memory ability accounted for group differences in ability to use potential outcomes during decision-making. These results suggest that the implicit mechanisms that drive reward-based learning are surprisingly intact in schizophrenia but that poor ability to integrate cognitive and affective information when calculating the value of possible choices might hamper the ability to use such information during explicit decision-making.