Probabilistic Reward Learning Research Articles

College students with depression symptom are more sensitive to task difficulty in reinforcement learning

BackgroundDepression is usually characterized by impairments in reward function, and shows altered motivation to reward in reinforcement learning. This study further explored whether task difficulty affects reinforcement learning in college students with and without depression symptom. MethodsThe depression symptom group (20) and the no depression symptom group (26) completed a probabilistic reward learning task with low, medium, and high difficulty levels, in which task the response bias to reward and the discriminability of reward were analyzed. Additionally, electrophysiological responses to reward and loss feedback were recorded and analyzed while they performed a simple gambling task. ResultsThe depression symptom group showed more response bias to reward than the no depression symptom group when the task was easy and then exhibited more quickly decrease in response bias to reward as task difficulty increased. The no depression symptom group showed a decrease in response bias only in the high-difficulty condition. Further regression analyses showed that, the Feedback-related negativity (FRN) and theta oscillation could predict response bias change in the low-difficulty condition, the FRN and oscillations of theta and delta could predict response bias change in the medium and high-difficulty conditions. LimitationsThe electrophysiological responses to loss and reward were not recorded in the same task as the reinforcement learning behaviors. ConclusionsCollege students with depression symptom are more sensitive to task difficulty during reinforcement learning. The FRN, and oscillations of theta and delta could predict reward leaning behavior.

Electrical brain activations in preadolescents during a probabilistic reward-learning task reflect cognitive processes and behavioral strategy.

Both adults and children learn through feedback which environmental events and choices are associated with higher probability of reward, an ability thought to be supported by the development of fronto-striatal reward circuits. Recent developmental studies have applied computational models of reward learning to investigate such learning in children. However, tasks and measures effective for assaying the cascade of reward-learning neural processes in children have been limited. Using a child-version of a probabilistic reward-learning task while recording event-related-potential (ERP) measures of electrical brain activity, this study examined key processes of reward learning in preadolescents (8-12 years old; n=30), namely: (1) reward-feedback sensitivity, as measured by the early-latency, reward-related, frontal ERP positivity, (2) rapid attentional shifting of processing toward favored visual stimuli, as measured by the N2pc component, and (3) longer-latency attention-related responses to reward feedback as a function of behavioral strategies (i.e., Win-Stay-Lose-Shift), as measured by the central-parietal P300. Consistent with our prior work in adults, the behavioral findings indicate preadolescents can learn stimulus-reward outcome associations, but at varying levels of performance. Neurally, poor preadolescent learners (those with slower learning rates) showed greater reward-related positivity amplitudes relative to good learners, suggesting greater reward-feedback sensitivity. We also found attention shifting towards to-be-chosen stimuli, as evidenced by the N2pc, but not to more highly rewarded stimuli as we have observed in adults. Lastly, we found the behavioral learning strategy (i.e., Win-Stay-Lose-Shift) reflected by the feedback-elicited parietal P300. These findings provide novel insights into the key neural processes underlying reinforcement learning in preadolescents.

Reward Sensitivity and Noise Contribute to Negative Affective Bias: A Learning Signal Detection Theory Approach in Decision-Making.

In patients with mood disorders, negative affective biases - systematically prioritising and interpreting information negatively - are common. A translational cognitive task testing this bias has shown that depressed patients have a reduced preference for a high reward under ambiguous decision-making conditions. The precise mechanisms underscoring this bias are, however, not yet understood. We therefore developed a set of measures to probe the underlying source of the behavioural bias by testing its relationship to a participant's reward sensitivity, value sensitivity and reward learning rate. One-hundred-forty-eight participants completed three online behavioural tasks: the original ambiguous-cue decision-making task probing negative affective bias, a probabilistic reward learning task probing reward sensitivity and reward learning rate, and a gambling task probing value sensitivity. We modelled the learning task through a dynamic signal detection theory model and the gambling task through an expectation-maximisation prospect theory model. Reward sensitivity from the probabilistic reward task (β = 0.131, p = 0.024) and setting noise from the probabilistic reward task (β = -0.187, p = 0.028) both predicted the affective bias score in a logistic regression. Increased negative affective bias, at least on this specific task, may therefore be driven in part by a combination of reduced sensitivity to rewards and more variable responses.

Reinforcement-Learning-Informed Queries Guide Behavioral Change

Algorithmically defined aspects of reinforcement learning correlate with psychopathology symptoms and change with symptom improvement following cognitive-behavioral therapy (CBT). Separate work in nonclinical samples has shown that varying the structure and statistics of task environments can change learning. Here, we combine these literatures, drawing on CBT-based guided restructuring of thought processes and computationally defined mechanistic targets identified by reinforcement-learning models in depression, to test whether and how verbal queries affect learning processes. Using a parallel-arm design, we tested 1,299 online participants completing a probabilistic reward-learning task while receiving repeated queries about the task environment (11 learning-query arms and one active control arm). Querying participants about reinforcement-learning-related task components altered computational-model-defined learning parameters in directions specific to the target of the query. These effects on learning parameters were consistent across depression-symptom severity, suggesting new learning-based strategies and therapeutic targets for evoking symptom change in mood psychopathology.

Transdiagnostic inflexible learning dynamics explain deficits in depression and schizophrenia.

Deficits in reward learning are core symptoms across many mental disorders. Recent work suggests that such learning impairments arise by a diminished ability to use reward history to guide behaviour, but the neuro-computational mechanisms through which these impairments emerge remain unclear. Moreover, limited work has taken a transdiagnostic approach to investigate whether the psychological and neural mechanisms that give rise to learning deficits are shared across forms of psychopathology. To provide insight into this issue, we explored probabilistic reward learning in patients diagnosed with major depressive disorder (n = 33) or schizophrenia (n = 24) and 33 matched healthy controls by combining computational modelling and single-trial EEG regression. In our task, participants had to integrate the reward history of a stimulus to decide whether it is worthwhile to gamble on it. Adaptive learning in this task is achieved through dynamic learning rates that are maximal on the first encounters with a given stimulus and decay with increasing stimulus repetitions. Hence, over the course of learning, choice preferences would ideally stabilize and be less susceptible to misleading information. We show evidence of reduced learning dynamics, whereby both patient groups demonstrated hypersensitive learning (i.e. less decaying learning rates), rendering their choices more susceptible to misleading feedback. Moreover, there was a schizophrenia-specific approach bias and a depression-specific heightened sensitivity to disconfirmational feedback (factual losses and counterfactual wins). The inflexible learning in both patient groups was accompanied by altered neural processing, including no tracking of expected values in either patient group. Taken together, our results thus provide evidence that reduced trial-by-trial learning dynamics reflect a convergent deficit across depression and schizophrenia. Moreover, we identified disorder distinct learning deficits.

Subsecond fluctuations in extracellular dopamine encode reward and punishment prediction errors in humans.

In the mammalian brain, midbrain dopamine neuron activity is hypothesized to encode reward prediction errors that promote learning and guide behavior by causing rapid changes in dopamine levels in target brain regions. This hypothesis (and alternatives regarding dopamine's role in punishment-learning) has limited direct evidence in humans. We report intracranial, subsecond measurements of dopamine release in human striatum measured, while volunteers (i.e., patients undergoing deep brain stimulation surgery) performed a probabilistic reward and punishment learning choice task designed to test whether dopamine release encodes only reward prediction errors or whether dopamine release may also encode adaptive punishment learning signals. Results demonstrate that extracellular dopamine levels can encode both reward and punishment prediction errors within distinct time intervals via independent valence-specific pathways in the human brain.

How does apathy impact exploration-exploitation decision-making in older patients with neurocognitive disorders?

Apathy is a pervasive clinical syndrome in neurocognitive disorders, characterized by a quantitative reduction in goal-directed behaviors. The brain structures involved in the physiopathology of apathy have also been connected to the brain structures involved in probabilistic reward learning in the exploration-exploitation dilemma. This dilemma in question involves the challenge of selecting between a familiar option with a more predictable outcome, and another option whose outcome is uncertain and may yield potentially greater rewards compared to the known option. The aim of this study was to combine experimental procedures and computational modeling to examine whether, in older adults with mild neurocognitive disorders, apathy affects performance in the exploration-exploitation dilemma. Through using a four-armed bandit reinforcement-learning task, we showed that apathetic older adults explored more and performed worse than non-apathetic subjects. Moreover, the mental flexibility assessed by the Trail-making test-B was negatively associated with the percentage of exploration. These results suggest that apathy is characterized by an increased explorative behavior and inefficient decision-making, possibly due to weak mental flexibility to switch toward the exploitation of the more rewarding options. Apathetic participants also took longer to make a choice and failed more often to respond in the allotted time, which could reflect the difficulties in action initiation and selection. In conclusion, the present results suggest that apathy in participants with neurocognitive disorders is associated with specific disturbances in the exploration-exploitation trade-off and sheds light on the disturbances in reward processing in patients with apathy.

Anxiety associated with perceived uncontrollable stress enhances expectations of environmental volatility and impairs reward learning

Unavoidable stress can lead to perceived lack of control and learned helplessness, a risk factor for depression. Avoiding punishment and gaining rewards involve updating the values of actions based on experience. Such updating is however useful only if action values are sufficiently stable, something that a lack of control may impair. We examined whether self-reported stress uncontrollability during the first wave of the COVID-19 pandemic predicted impaired reward-learning. In a preregistered study during the first-wave of the COVID-19 pandemic, we used self-reported measures of depression, anxiety, uncontrollable stress, and COVID-19 risk from 427 online participants to predict performance in a three-armed-bandit probabilistic reward learning task. As hypothesised, uncontrollable stress predicted impaired learning, and a greater proportion of probabilistic errors following negative feedback for correct choices, an effect mediated by state anxiety. A parameter from the best-fitting hidden Markov model that estimates expected beliefs that the identity of the optimal choice will shift across images, mediated effects of state anxiety on probabilistic errors and learning deficits. Our findings show that following uncontrollable stress, anxiety promotes an overly volatile representation of the reward-structure of uncertain environments, impairing reward attainment, which is a potential path to anhedonia in depression.

Reward System in Late-Life Depression: a Cross-Sectional Case-Control Study.

Anhedonia, commonly defined as a reduced ability to feel pleasure, is a core clinical symptom of late-life depression (LLD). Deficits in reward processing are hypothesised to be associated with anhedonia. We examined differences in reward sensitivity between patients with LLD and healthy controls and explored the associations between LLD-related symptomatology, global cognition, and the reward system. The reward responsiveness of 63 patients with LLD and 58 healthy controls aged ≥60 years was assessed using the probabilistic reward learning task with an asymmetric reward schedule. Compared with healthy controls, patients with LLD displayed lower response bias and reward learning. Global cognition of all participants was positively correlated with response bias. In patients with LLD, anhedonia severity explained impaired reward learning. A deficit in reward processing is implicated in patients with LLD. Our findings suggest that executive dysfunction and anhedonia contribute to lower sensitivity to reward learning in patients with LLD.

Striatal dopamine supports reward expectation and learning: A simultaneous PET/fMRI study

Converging evidence from both human neuroimaging and animal studies has supported a model of mesolimbic processing underlying reward learning behaviors, based on the computation of reward prediction errors. However, competing evidence supports human dopamine signaling in the basal ganglia as also contributing to the generation of higher order learning heuristics. Here, we present data from a large (N = 81, 18–30yo), multi-modal neuroimaging study using simultaneously acquired task fMRI, affording temporal resolution of reward system function, and PET imaging with [11C]Raclopride (RAC), assessing striatal dopamine (DA) D2/3 receptor binding, during performance of a probabilistic reward learning task. Both fMRI activation and PET DA measures showed ventral striatum involvement for signaling rewards. However, greater DA release was uniquely associated with learning strategies (i.e., learning rates) that were more task-optimal within the best fitting reinforcement learning model. This DA response was associated with BOLD activation of a network of regions including anterior cingulate cortex, medial prefrontal cortex, thalamus and posterior parietal cortex, primarily during expectation, rather than prediction error, task epochs. Together, these data provide novel, human in vivo evidence that striatal dopaminergic signaling interacts with a network of cortical regions to generate task-optimal learning strategies, rather than representing reward outcomes in isolation.

Humans combine value learning and hypothesis testing strategically in multi-dimensional probabilistic reward learning.

Realistic and complex decision tasks often allow for many possible solutions. How do we find the correct one? Introspection suggests a process of trying out solutions one after the other until success. However, such methodical serial testing may be too slow, especially in environments with noisy feedback. Alternatively, the underlying learning process may involve implicit reinforcement learning that learns about many possibilities in parallel. Here we designed a multi-dimensional probabilistic active-learning task tailored to study how people learn to solve such complex problems. Participants configured three-dimensional stimuli by selecting features for each dimension and received probabilistic reward feedback. We manipulated task complexity by changing how many feature dimensions were relevant to maximizing reward, as well as whether this information was provided to the participants. To investigate how participants learn the task, we examined models of serial hypothesis testing, feature-based reinforcement learning, and combinations of the two strategies. Model comparison revealed evidence for hypothesis testing that relies on reinforcement-learning when selecting what hypothesis to test. The extent to which participants engaged in hypothesis testing depended on the instructed task complexity: people tended to serially test hypotheses when instructed that there were fewer relevant dimensions, and relied more on gradual and parallel learning of feature values when the task was more complex. This demonstrates a strategic use of task information to balance the costs and benefits of the two methods of learning.

Dopamine and reward-related vigor in younger and older adults

Vigor reflects how motivated people are to respond to stimuli. We previously showed that, on average, humans are more vigorous when a higher rate of reward is available, and that this relationship is modulated by the dopamine precursor levodopa. Dopamine signaling and probabilistic reward learning deteriorate across the adult life span, and thus, the relationship between vigor and reward may also change in aging. We tested this assertion and assessed whether it correlates with D1 dopamine receptor availability, measured using Positron Emission Tomography. We registered response times of 30 older and 30 younger participants during an oddball discrimination task where rewards varied systematically between trials. The average reward rate had a similar impact on vigor in both age groups. There was a weak positive association between ventral striatal dopamine receptor availability and the effect of average reward rate on response time. Overall, the effect of reward on response vigor was similar in younger and older adults, and weakly correlated with dopamine D1 receptor availability.

Pupillary response in reward processing in adults with major depressive disorder in remission.

Major depressive disorder (MDD) is associated with impaired reward processing and reward learning. The literature is inconclusive regarding whether these impairments persist after remission. The current study examined reward processing during a probabilistic learning task in individuals in remission from MDD (n = 19) and never depressed healthy controls (n = 31) matched for age and sex. The outcome measures were pupil dilation (an indirect index of noradrenergic activity and arousal) and computational modeling parameters. Participants completed two versions (facial/nonfacial feedback) of probabilistic reward learning task with changing contingencies. Pupil dilation was measured with a corneal reflection eye tracker. The hypotheses and analysis plan were preregistered. Healthy controls had larger pupil dilation following losses than gains (p <.001), whereas no significant difference between outcomes was found in individuals with a history of MDD, resulting in an interaction between group and outcome (β = 0.81, SE = 0.34, t = 2.37, p = .018). The rMDD group also achieved lower mean score at the last trial (t[46.77] = 2.12, p = .040) as well as a smaller proportion of correct choices (t[46.70] = 2.09, p = .041) compared with healthy controls. Impaired reward processing may persist after remission from MDD and could constitute a latent risk factor for relapse. Measuring pupil dilation in a reward learning task is a promising method for identifying reward processing abnormalities linked to MDD. The task is simple and noninvasive, which makes it feasible for clinical research.

P242. Anxiety Associated with Perceived Lack of Control Over Stress During the COVID-19 Pandemic Impairs Reward Learning

BackgroundUnavoidable stress can lead to perceived lack of control and learned helplessness, a risk factor for depression. Avoiding punishment and gaining rewards involve updating the values of actions based on experience. We examined whether self-reported stress uncontrollability during the first-wave of the COVID-19 pandemic predicted impaired reward-learning.MethodsIn a preregistered study, 427 online participants completed a three-armed-bandit probabilistic reward-learning task to maximise monetary reward. Participants chose between three composite-stimuli comprising three images each. Choosing the target image, which changed every 20-30 trials without warning, resulted in reward on 80% of trials (20% otherwise). Participants completed self-reported measures of depression, anxiety, stress uncontrollability, and risk of COVID-19, which we used to predict accuracy across trials. We tested various reinforcement learning and hidden Markov models (HMMs) to examine cognitive mechanisms underlying the effect of perceived uncontrollability on choices. We hypothesized that stress uncontrollability would predict worse reward-learning.ResultsHigher state anxiety and stress uncontrollability separately predicted impaired reward-learning (z=-2.66, p=.008; z=-3.05, p=.002). The effect of stress uncontrollability was mediated by anxiety (β=0.61, p=.052). State anxiety was positively associated with a parameter from the winning HMM that estimates how much beliefs about the identity of the target diffuse among images (r=.12, p=.0152).ConclusionsGreater perceived stress uncontrollability during the COVID-19 pandemic was associated with impaired reward-learning, and this effect was explained by current anxiety. These results suggest that anxiety promotes imprecision in the representation of the reward structure of the environment, mediating the observed effect of lack of control on reward-learning.Supported BySwedish Research CouncilKeywordsStress Controllability, Reward Learning, Computational Modelling, State Anxiety BackgroundUnavoidable stress can lead to perceived lack of control and learned helplessness, a risk factor for depression. Avoiding punishment and gaining rewards involve updating the values of actions based on experience. We examined whether self-reported stress uncontrollability during the first-wave of the COVID-19 pandemic predicted impaired reward-learning. Unavoidable stress can lead to perceived lack of control and learned helplessness, a risk factor for depression. Avoiding punishment and gaining rewards involve updating the values of actions based on experience. We examined whether self-reported stress uncontrollability during the first-wave of the COVID-19 pandemic predicted impaired reward-learning. MethodsIn a preregistered study, 427 online participants completed a three-armed-bandit probabilistic reward-learning task to maximise monetary reward. Participants chose between three composite-stimuli comprising three images each. Choosing the target image, which changed every 20-30 trials without warning, resulted in reward on 80% of trials (20% otherwise). Participants completed self-reported measures of depression, anxiety, stress uncontrollability, and risk of COVID-19, which we used to predict accuracy across trials. We tested various reinforcement learning and hidden Markov models (HMMs) to examine cognitive mechanisms underlying the effect of perceived uncontrollability on choices. We hypothesized that stress uncontrollability would predict worse reward-learning. In a preregistered study, 427 online participants completed a three-armed-bandit probabilistic reward-learning task to maximise monetary reward. Participants chose between three composite-stimuli comprising three images each. Choosing the target image, which changed every 20-30 trials without warning, resulted in reward on 80% of trials (20% otherwise). Participants completed self-reported measures of depression, anxiety, stress uncontrollability, and risk of COVID-19, which we used to predict accuracy across trials. We tested various reinforcement learning and hidden Markov models (HMMs) to examine cognitive mechanisms underlying the effect of perceived uncontrollability on choices. We hypothesized that stress uncontrollability would predict worse reward-learning. ResultsHigher state anxiety and stress uncontrollability separately predicted impaired reward-learning (z=-2.66, p=.008; z=-3.05, p=.002). The effect of stress uncontrollability was mediated by anxiety (β=0.61, p=.052). State anxiety was positively associated with a parameter from the winning HMM that estimates how much beliefs about the identity of the target diffuse among images (r=.12, p=.0152). Higher state anxiety and stress uncontrollability separately predicted impaired reward-learning (z=-2.66, p=.008; z=-3.05, p=.002). The effect of stress uncontrollability was mediated by anxiety (β=0.61, p=.052). State anxiety was positively associated with a parameter from the winning HMM that estimates how much beliefs about the identity of the target diffuse among images (r=.12, p=.0152). ConclusionsGreater perceived stress uncontrollability during the COVID-19 pandemic was associated with impaired reward-learning, and this effect was explained by current anxiety. These results suggest that anxiety promotes imprecision in the representation of the reward structure of the environment, mediating the observed effect of lack of control on reward-learning. Greater perceived stress uncontrollability during the COVID-19 pandemic was associated with impaired reward-learning, and this effect was explained by current anxiety. These results suggest that anxiety promotes imprecision in the representation of the reward structure of the environment, mediating the observed effect of lack of control on reward-learning.

Reward learning in unmedicated women with bulimia nervosa: A pilot investigation

Bulimia nervosa (BN) is characterized by recurrent engagement in eating disorder behaviors despite negative consequences, potentially reflecting aberrant stimulus-response or reward-learning processes. Indeed, frontostriatal circuitry involved in reward learning is altered in persons with BN and preliminary research suggests reward learning is impaired in persons with BN. Additional research on reward learning in BN and its association with eating disorder symptom expression is warranted to further the field's understanding of potential pathophysiological mechanisms of BN. To this end, the probabilistic reward learning task (PRLT) was administered to unmedicated women with BN (n = 15) and demographically matched psychiatrically healthy women (n = 18). Contrary to our hypotheses, results demonstrated that women with BN showed greater reward learning during the PRLT relative to healthy comparison women when covarying for symptoms of depression, social anxiety, and mania. Exploratory analyses showed that binge-eating frequency was inversely associated with reward learning in women with BN; however, results should be interpreted with caution due to the small sample size. Together, results suggest that women with BN do not have deficits in implicit reward learning. Given the preliminary nature of this investigation, larger-scale studies are needed to further examine reward learning in current BN and could compare reward learning using general (e.g., monetary) and disorder-specific (e.g., food) reinforcers. Further work is needed to confirm the inverse association between reward learning and binge eating.

Challenges and Strategies for Current Classifications of Depressive Disorders: Proposal for Future Diagnostic Standards.

The Diagnostic and Statistical Manual of Mental Disorder, Fourth Edition (DSM-IV) was revised based on a combination of a categorical and a dimensional approach such that in the DSM, Fifth Edition (DSM-5), depressive disorders have been separated as a distinctive disease entity from bipolar disorders, consistent with the deconstruction of Kraepelinian dualism. Additionally, the diagnostic thresholds of depressive disorders may be reduced due to the addition of "hopelessness" to the subjective descriptors of depressed mood and the removal of the "bereavement exclusion." Manic/hypomanic, psychotic, and anxious symptoms in major depressive disorder (MDD) and other depressive disorders are described using the transdiagnostic specifiers of "with mixed features," "with psychotic features," and "with anxious distress," respectively. Additionally, due to the polythetic and operational characteristics of the DSM-5 diagnostic criteria, the heterogeneity of MDD is inevitable. Thus, 227 different symptom combinations fulfill the DSM-5 diagnostic criteria for MDD. This heterogeneity of MDD is criticized in view of the Wittgensteinian analogy of language game. Depression subtypes determined by disturbances in monoamine levels and the severity of the disease have been identified in the literature. According to a review of the Gottesman and Gould criteria, neuroticism, morning cortisol, cortisol awakening response, asymmetry in frontal cortical activity on electroencephalography (EEG), and probabilistic reward learning, among other variables, are evidenced as endophenotypes for depressive disorders. Network analysis has been proposed as a potential method to compliment the limitations of current diagnostic criteria and to explore the pathways between depressive symptoms, as well as to identify novel and interesting relationships between depressive symptoms. Based on the literature on network analysis in this field, no differences in the centrality index of the DSM and non-DSM symptoms were repeatedly present among patients with MDD. Furthermore, MDD and other depressive syndromes include two of the Research Domain Criteria (RDoC), including the Loss construct within the Negative Valence Systems domains and various Reward constructs within the Positive Valence Systems domain.

Cortical thickness correlates of probabilistic reward learning in young adults

While impairments in reward learning have been observed in depression, individual differences also exist among healthy normative populations and these differences have been parametrically mapped onto naturally occurring inter-individual variability in neuroanatomy. To date, alterations of cortical thickness as a function of reward learning have not been studied. Structural MRI and behaviourally assessedrobabilistic reward learning data from 103 participants (nmale = 35, nfemale = 68, age = 19.0 ± 1.0 years) from the Duke Neurogenetics Study were used in our analysis. The relationship between reward learning and vertex-wise cortical thickness was tested with a multiple linear regression including gender, age, and reward task stimulus discriminability as nuisance variables. Reward learning was significantly, positively associated with cortical thickness in the right medial superior frontal cortex (t = 4.469, pFWER = .048). This is the first study exploring the relationship between cortical thickness and probabilistic reward learning. It implicates a novel region in reward learning and can thereby inform future mechanistic studies.

Autistic traits are related to worse performance in a volatile reward learning task despite adaptive learning rates.

Recent theories propose that autism is characterized by an impairment in determining when to learn and when not. Here, we investigated this hypothesis by estimating learning rates (i.e. the speed with which one learns) in three different environments that differed in rule stability and uncertainty. We found that neurotypical participants with more autistic traits performed worse in a volatile environment (with unstable rules), as they chose less often for the most rewarding option. Exploratory analyses indicated that performance was specifically worse when reward rules were opposite to those initially learned for participants with more autistic traits. However, there were no differences in the adjustment of learning rates between participants with more versus less autistic traits. Together, these results suggest that performance in volatile environments is lower in participants with more autistic traits, but that this performance difference cannot be unambiguously explained by an impairment in adjusting learning rates.

Aberrant computational mechanisms of social learning and decision-making in schizophrenia and borderline personality disorder.

Psychiatric disorders are ubiquitously characterized by debilitating social impairments. These difficulties are thought to emerge from aberrant social inference. In order to elucidate the underlying computational mechanisms, patients diagnosed with major depressive disorder (N = 29), schizophrenia (N = 31), and borderline personality disorder (N = 31) as well as healthy controls (N = 34) performed a probabilistic reward learning task in which participants could learn from social and non-social information. Patients with schizophrenia and borderline personality disorder performed more poorly on the task than healthy controls and patients with major depressive disorder. Broken down by domain, borderline personality disorder patients performed better in the social compared to the non-social domain. In contrast, controls and major depressive disorder patients showed the opposite pattern and schizophrenia patients showed no difference between domains. In effect, borderline personality disorder patients gave up a possible overall performance advantage by concentrating their learning in the social at the expense of the non-social domain. We used computational modeling to assess learning and decision-making parameters estimated for each participant from their behavior. This enabled additional insights into the underlying learning and decision-making mechanisms. Patients with borderline personality disorder showed slower learning from social and non-social information and an exaggerated sensitivity to changes in environmental volatility, both in the non-social and the social domain, but more so in the latter. Regarding decision-making the modeling revealed that compared to controls and major depression patients, patients with borderline personality disorder and schizophrenia showed a stronger reliance on social relative to non-social information when making choices. Depressed patients did not differ significantly from controls in this respect. Overall, our results are consistent with the notion of a general interpersonal hypersensitivity in borderline personality disorder and schizophrenia based on a shared computational mechanism characterized by an over-reliance on beliefs about others in making decisions and by an exaggerated need to make sense of others during learning specifically in borderline personality disorder.

A role for neurogenesis in probabilistic reward learning.

Rewards are often unreliable and optimal choice requires behavioral flexibility and learning about the probabilistic nature of uncertain rewards. Probabilistic learning occurs over multiple trials, often without conscious knowledge, and is traditionally associated with striatal function. While the hippocampus is classically recognized for its role in memory for individual experiences, recent work indicates that it is also involved in probabilistic forms of learning but little is known about the features that support such learning. We hypothesized that adult neurogenesis may be involved, because adult-born neurons contribute to both learning and reward-related behaviors. To test this, we used an appetitive probabilistic reversal learning task where a correct lever is rewarded with 80% probability and an incorrect lever is rewarded with 20% probability. Behavioral flexibility was assessed by switching correct-incorrect lever identities after 8 consecutive correct choices. Transgenic male rats that lacked adult neurogenesis displayed an initial deficit in discriminating the correct and incorrect levers, but they were not impaired at reversing behavior when the reward contingencies switched. When reward was withheld after a correct lever choice, neurogenesis-deficient rats were more likely to choose the incorrect lever on the subsequent trial. Also, rats with intact neurogenesis were more sensitive to reward at the incorrect lever. Differences were not observed in control transgenic rats that had intact neurogenesis. These results identify a novel role for neurogenesis in learning about uncertain, probabilistic rewards. Altered sensitivity to reward and negative feedback furthermore implicates neurogenesis in cognitive phenotypes associated with mood disorders such as depression. (PsycInfo Database Record (c) 2020 APA, all rights reserved).