Impaired Reinforcement Learning Research Articles

Decomposition of Reinforcement Learning Deficits in Disordered Gambling via Drift Diffusion Modeling and Functional Magnetic Resonance Imaging.

Gambling disorder is associated with deficits in reward-based learning, but the underlying computational mechanisms are still poorly understood. Here, we examined this issue using a stationary reinforcement learning task in combination with computational modeling and functional resonance imaging (fMRI) in individuals that regular participate in gambling (n = 23, seven fulfilled one to three DSM 5 criteria for gambling disorder, sixteen fulfilled four or more) and matched controls (n = 23). As predicted, the gambling group exhibited substantially reduced accuracy, whereas overall response times (RTs) were not reliably different between groups. We then used comprehensive modeling using reinforcement learning drift diffusion models (RLDDMs) in combination with hierarchical Bayesian parameter estimation to shed light on the computational underpinnings of this performance deficit. In both groups, an RLDDM in which both non-decision time and decision threshold (boundary separation) changed over the course of the experiment accounted for the data best. The model showed good parameter and model recovery, and posterior predictive checks revealed that, in both groups, the model accurately reproduced the evolution of accuracies and RTs over time. Modeling revealed that, compared to controls, the learning impairment in the gambling group was linked to a more rapid reduction in decision thresholds over time, and a reduced impact of value-differences on the drift rate. The gambling group also showed shorter non-decision times. FMRI analyses replicated effects of prediction error coding in the ventral striatum and value coding in the ventro-medial prefrontal cortex, but there was no credible evidence for group differences in these effects. Taken together, our findings show that reinforcement learning impairments in disordered gambling are linked to both maladaptive decision threshold adjustments and a reduced consideration of option values in the choice process.

Unraveling the neuropsychological underpinnings of self-regulation problems in individuals convicted of sexual offenses against children: A look into reinforcement learning

Self-regulation problems are critically involved in the onset and the maintenance of sexual offending behavior against children. Studying the neuropsychological underpinnings of these problems could help deepen our understanding of this contributing factor and, thus, of sexual offending behavior. Whereas most studies have examined executive functioning in relation to self-regulation problems in individuals convicted of sexual offenses against children (ISOCs), this review aimed to provide an overview of what is known about another process that is involved in self-regulation, that is reinforcement learning. The results of this review suggested that ISOCs are impaired in their ability to acquire and reverse stimulus-reward and stimulus-punishment associations relative to nonoffender controls, but similar to a control group of individuals convicted of nonsexual violent offenses. These reinforcement learning impairments were found to be more pronounced in nonpedophilic ISOCs than in pedophilic ISOCs. By paving the way towards a deeper understanding of the self-regulation problems seen in ISOCs, this review can help guide treatment strategies for ISOCs.

The Dopamine System in Mediating Alcohol Effects in Humans.

Brain-imaging studies show that the development and maintenance of alcohol use disorder (AUD) is determined by a complex interaction of different neurotransmitter systems and multiple psychological factors. In this context, the dopaminergic reinforcement system appears to be of fundamental importance. We focus on the excitatory and depressant effects of acute versus chronic alcohol intake and its impact on dopaminergic neurotransmission. Furthermore, we describe alterations in dopaminergic neurotransmission as associated with symptoms of alcohol dependence. We specifically focus on neuroadaptations to chronic alcohol consumption and their effect on central processing of alcohol-associated and reward-related stimuli. Altered reward processing, complex conditioning processes, impaired reinforcement learning, and increased salience attribution to alcohol-associated stimuli enable alcohol cues to drive alcohol seeking and consumption. Finally, we will discuss how the neurobiological and neurochemical mechanisms of alcohol-associated alterations in reward processing and learning can interact with stress, cognition, and emotion processing.

Reward-based reinforcement learning is altered among individuals with a history of major depressive disorder and psychomotor retardation symptoms

Reward-based reinforcement learning impairments are common in major depressive disorder, but it is unclear which aspects of reward-based reinforcement learning are disrupted in remitted major depression (rMDD). Given that the neurobiological substrates that implement reward-based RL are also strongly implicated in psychomotor retardation (PmR), the present study sought to test whether reward-based reinforcement learning is altered in rMDD individuals with a history of PmR. Three groups of individuals (1) rMDD with past PmR (PmR+, N = 34), (2) rMDD without past PmR (PmR-, N = 44), and (3) healthy controls (N = 90) completed a reward-based reinforcement learning task. Computational modeling was applied to test for group differences in model-derived parameters – specifically, learning rates and reward sensitivity. Compared to controls, rMDD PmR + exhibited lower learning rates, but not reduced reward sensitivity. By contrast, rMDD PmR- did not significantly differ from controls on either of the model-derived parameters. Follow-up analyses indicated that the results were not due to current psychopathology symptoms. Results indicate that a history of PmR predicts altered reward-based reinforcement learning in rMDD. Abnormal reward-related reinforcement learning may reflect a scar of past depressive episodes that contained psychomotor symptoms, or a trait-like deficit that preceded these episodes.

Delaying feedback compensates for impaired reinforcement learning in developmental dyslexia

A theoretical framework suggests that developmental dyslexia is characterized by abnormalities in brain structures underlying the procedural learning and memory systems while the declarative learning and memory systems are presumed to remain intact or even enhanced (Procedural Deficit Hypothesis). This notion has been supported by a substantial body of research, which focused on each system independently. However, less attention has been paid to interactions between these memory systems which may provide insights as to learning situations and conditions in which learning in dyslexia can be improved. The current study was undertaken to examine these important but unresolved issues. To this end, probabilistic reinforcement learning and episodic memory tasks were examined in participants with dyslexia and neurotypicals simultaneously within a single task. Feedback timing presentation was manipulated, building on prior research indicating that delaying feedback timing shifts striatal-based probabilistic learning, to become more hippocampal-dependent. It was hypothesized that if the procedural learning and memory systems are impaired in dyslexia, performance will be impaired under conditions that encourage procedural memory engagement (immediate feedback trials) but not under conditions that promote declarative memory processing (long delayed feedback trials). It was also predicted that the ability to incidentally acquire episodic information would be preserved in dyslexia. The results supported these predictions. Participants with dyslexia were impaired in probabilistic learning of cue-outcome associations compared to neurotypicals in an immediate feedback condition, but not when feedback on choices was presented after a long delay. Furthermore, participants with dyslexia demonstrated similar performance to neurotypicals in a task requiring incidental episodic memory formation. These findings attest to a dissociation between procedural-based and declarative-based learning in developmental dyslexia within a single task, a finding that adds discriminative validity to the Procedural Deficit Hypothesis. Just as important, the present findings suggest that training conditions designed to shift the load from midbrain/striatal systems to declarative memory mechanisms have the potential to compensate for impaired learning in developmental dyslexia.

Reduced sensitivity to affiliation and psychopathic traits.

Psychopathy is characterized by affective and interpersonal deficits, deviant lifestyle, and antisocial behaviors. Much research has been dedicated to understanding the impairments in reinforcement learning, fear conditioning, and sensitivity to threat, distress, or fear in others, which are thought to underpin psychopathic traits. Fewer studies have examined deficits in affiliative processes, which could provide insight into mechanisms giving rise to the impairments in social bonding, closeness with others, and cooperation that also characterize individuals high on psychopathy. The current study examined whether reduced sensitivity to affiliation was related to psychopathic traits among 407 adults from the community (female, 59%). Sensitivity to affiliation was modeled as a latent construct capturing item-level variance shared across 4 measures that assessed sensitivity to emotional and physical cues of affiliation, including stimuli presented as videos or images, and via self-report ratings about sensitivity to positive affiliative and affective cues. Results indicated that lower sensitivity to affiliation was related to higher total psychopathy scores. In particular, in models parsing the overlap of psychopathy factor and facet scores, lower sensitivity to affiliation was uniquely related to higher Factor 1 and affective facet scores, as well as higher scores on a measure of callous-unemotional traits. Our findings provide support for the existence of important socioaffiliative and motivational deficits that may underpin the affective features of psychopathy and speak to the potential to target such mechanisms in interventions and treatments to reduce psychopathic traits. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Impaired Learning From Negative Feedback in Stimulant Use Disorder: Dopaminergic Modulation.

BackgroundDrug-induced alterations to the dopamine system in stimulant use disorder (SUD) are hypothesized to impair reinforcement learning (RL). Computational modeling enables the investigation of the latent processes of RL in SUD patients, which could elucidate the nature of their impairments.MethodsWe investigated RL in 44 SUD patients and 41 healthy control participants using a probabilistic RL task that assesses learning from reward and punishment separately. In an independent sample, we determined the modulatory role of dopamine in RL following a single dose of the dopamine D2/3 receptor antagonist amisulpride (400 mg) and the agonist pramipexole (0.5 mg) in a randomised, double-blind, placebo-controlled, crossover design. We analyzed task performance using computational modelling and hypothesized that RL impairments in SUD patients would be differentially modulated by a dopamine D2/3 receptor antagonist and agonist.ResultsComputational analyses in both samples revealed significantly reduced learning rates from punishment in SUD patients compared with healthy controls, whilst their reward learning rates were not measurably impaired. In addition, the dopaminergic receptor agents modulated RL parameters differentially in both groups. Both amisulpride and pramipexole impaired RL parameters in healthy participants, but ameliorated learning from punishment in SUD patients.ConclusionOur findings suggest that RL impairments seen in SUD patients are associated with altered dopamine function.

Single Exposure to Cocaine Impairs Reinforcement Learning by Potentiating the Activity of Neurons in the Direct Striatal Pathway in Mice.

Plasticity in the glutamatergic synapses on striatal medium spiny neurons (MSNs) is not only essential for behavioral adaptation but also extremely vulnerable to drugs of abuse. Modulation on these synapses by even a single exposure to an addictive drug may interfere with the plasticity required by behavioral learning and thus produce impairment. In the present work, we found that the negative reinforcement learning, escaping mild foot-shocks by correct nose-poking, was impaired by a single in vivo exposure to 20 mg/kg cocaine 24 h before the learning in mice. Either a single exposure to cocaine or reinforcement learning potentiates the glutamatergic synapses on MSNs expressing the striatal dopamine 1 (D1) receptor (D1-MSNs). However, 24 h after the cocaine exposure, the potentiation required for reinforcement learning was disrupted. Specific manipulation of the activity of striatal D1-MSNs in D1-cre mice demonstrated that activation of these MSNs impaired reinforcement learning in normal D1-cre mice, but inhibition of these neurons reversed the reinforcement learning impairment induced by cocaine. The results suggest that cocaine potentiates the activity of direct pathway neurons in the dorsomedial striatum and this potentiation might disrupt the potentiation produced during and required for reinforcement learning.

Reinforcement learning abnormalities in the attenuated psychosis syndrome and first episode psychosis

Prior studies indicate that chronic schizophrenia (SZ) is associated with a specific profile of reinforcement learning abnormalities. These impairments are characterized by: 1) reductions in learning rate, and 2) impaired Go learning and intact NoGo learning. Furthermore, each of these deficits are associated with greater severity of negative symptoms, consistent with theoretical perspectives positing that avolition and anhedonia are associated with impaired value representation. However, it is unclear whether these deficits extend to earlier phases of psychotic illness and when individuals are unmedicated. Two studies were conducted to examine reinforcement learning deficits in earlier phases of psychosis and in high risk patients. In study 1, participants included 35 participants with first episode psychosis (FEP) with limited antipsychotic medication exposure and 25 healthy controls (HC). Study 2 included 17 antipsychotic naïve individuals who were at clinical high-risk for psychosis (CHR) (i.e., attenuated psychosis syndrome) and 18 matched healthy controls (HC). In both studies, participants completed the Temporal Utility Integration Task, a measure of probabilistic reinforcement learning that contained Go and NoGo learning blocks. FEP displayed impaired Go and NoGo learning. In contrast, CHR did not display impairments in Go or NoGo learning. Impaired Go learning was not significantly associated with clinical outcomes in the CHR or FEP samples. Findings provide new evidence for areas of spared and impaired reinforcement learning in early phases of psychosis.

Reliability and Replicability of Implicit and Explicit Reinforcement Learning Paradigms in People With Psychotic Disorders.

Motivational deficits in people with psychosis may be a result of impairments in reinforcement learning (RL). Therefore, behavioral paradigms that can accurately measure these impairments and their change over time are essential. We examined the reliability and replicability of 2 RL paradigms (1 implicit and 1 explicit, each with positive and negative reinforcement components) given at 2 time points to healthy controls (n = 75), and people with bipolar disorder (n = 62), schizoaffective disorder (n = 60), and schizophrenia (n = 68). Internal consistency was acceptable (mean α = 0.78 ± 0.15), but test-retest reliability was fair to low (mean intraclass correlation = 0.33 ± 0.25) for both implicit and explicit RL. There were no clear effects of practice for these tasks. Largely, performance on these tasks shows intact implicit and impaired explicit RL in psychosis. Symptom presentation did not relate to performance in any robust way. Our findings replicate previous literature showing spared implicit RL and impaired explicit reinforcement in psychosis. This suggests typical basal ganglia dopamine release, but atypical recruitment of the orbitofrontal and dorsolateral prefrontal cortices. However, we found that these tasks have only fair to low test-retest reliability and thus may not be useful for assessing change over time in clinical trials.

Blunted reward prediction error signals in internet gaming disorder.

Internet gaming disorder (IGD) is a type of behavioural addictions. One of the key features of addiction is the excessive exposure to addictive objectives (e.g. drugs) reduces the sensitivity of the brain reward system to daily rewards (e.g. money). This is thought to be mediated via the signals expressed as dopaminergic reward prediction error (RPE). Emerging evidence highlights blunted RPE signals in drug addictions. However, no study has examined whether IGD also involves alterations in RPE signals that are observed in other types of addictions. To fill this gap, we used functional magnetic resonance imaging data from 45 IGD and 42 healthy controls (HCs) during a reward-related prediction-error task and utilised a psychophysiological interaction (PPI) analysis to characterise the underlying neural correlates of RPE and related functional connectivity. Relative to HCs, IGD individuals showed impaired reinforcement learning, blunted RPE signals in multiple regions of the brain reward system, including the right caudate, left orbitofrontal cortex (OFC), and right dorsolateral prefrontal cortex (DLPFC). Moreover, the PPI analysis revealed a pattern of hyperconnectivity between the right caudate, right putamen, bilateral DLPFC, and right dorsal anterior cingulate cortex (dACC) in the IGD group. Finally, linear regression suggested that the connection between the right DLPFC and right dACC could significantly predict the variation of RPE signals in the left OFC. These results highlight disrupted RPE signalling and hyperconnectivity between regions of the brain reward system in IGD. Reinforcement learning deficits may be crucial underlying characteristics of IGD pathophysiology.

Dopamine is associated with prioritization of reward-associated memories in Parkinson's disease.

Patients with Parkinson's disease have reduced reward sensitivity related to dopaminergic neuron loss, which is associated with impairments in reinforcement learning. Increasingly, however, dopamine-dependent reward signals are recognized to play an important role beyond reinforcement learning. In particular, it has been shown that reward signals mediated by dopamine help guide the prioritization of events for long-term memory consolidation. Meanwhile, studies of memory in patients with Parkinson's disease have focused on overall memory capacity rather than what is versus what isn't remembered, leaving open questions about the effect of dopamine replacement on the prioritization of memories by reward and the time-dependence of this effect. The current study sought to fill this gap by testing the effect of reward and dopamine on memory in patients with Parkinson's disease. We tested the effect of dopamine modulation and reward on two forms of long-term memory: episodic memory for neutral objects and memory for stimulus-value associations. We measured both forms of memory in a single task, adapting a standard task of reinforcement learning with incidental episodic encoding events of trial-unique objects. Objects were presented on each trial at the time of feedback, which was either rewarding or not. Memory for the trial-unique images and for the stimulus-value associations, and the influence of reward on both, was tested immediately after learning and 2 days later. We measured performance in Parkinson's disease patients tested either ON or OFF their dopaminergic medications and in healthy older control subjects. We found that dopamine was associated with a selective enhancement of memory for reward-associated images, but that it did not influence overall memory capacity. Contrary to predictions, this effect did not differ between the immediate and delayed memory tests. We also found that while dopamine had an effect on reward-modulated episodic memory, there was no effect of dopamine on memory for stimulus-value associations. Our results suggest that impaired prioritization of cognitive resource allocation may contribute to the early cognitive deficits of Parkinson's disease.

S79. REINFORCEMENT LEARNING ABNORMALITIES IN INDIVIDUALS AT CLINICAL HIGH RISK AND IN THE FIRST EPISODE OF PSYCHOSIS

BackgroundPrior studies indicate that chronic schizophrenia (SZ) is associated with a specific profile of reinforcement learning abnormalities. These impairments are characterized by: 1) reductions in learning rate, and 2) impaired Go learning and intact NoGo learning. Furthermore, each of these deficits are associated with greater severity of negative symptoms, consistent with theoretical perspectives positing that avolition and anhedonia are associated with deficits in generating, updating, and maintaining mental representations of reward value hat are needed to guide decision-making. However, it is unclear whether these deficits extend to earlier phases of psychotic illness and when individuals are unmedicated.MethodsTwo studies were conducted to examine reinforcement learning deficits in earlier phases of psychosis. In study 1, participants included 35 participants with first episode psychosis (FEP) and 25 healthy controls (HC). Study 2 included 17 antipsychotic naïve individuals who met criteria for attenuated psychosis syndrome (APS) (i.e., those with a prodromal syndrome) and 18 matched healthy controls (HC). In both studies, participants completed the Temporal Utility Integration Task, a measure of probabilistic reinforcement learning that contained Go and NoGo learning blocks. Participants in the clinical groups also completed neuropsychological testing and standard clinical interviews designed to determine symptom severity and diagnosis.ResultsFEP displayed impaired Go learning and intact NoGo learning. In contrast, APS did not display impairments in Go or NoGo learning at the group level. Negative symptoms were not significantly associated with reinforcement learning in APS participants. However, greater impairments in Go learning were associated with increased cross-sectional risk for conversion on the NAPLS risk calculator score in the APS group.DiscussionFindings provide new evidence for areas of spared and impaired reinforcement learning in early phases of psychosis. Similar to chronic SZ, FEP was associated with impaired Go learning, and intact NoGo learning. Reinforcement learning is more spared in those at clinical high-risk, except those at greatest risk for conversion, where Go learning deficits are more pronounced. These findings suggest that reinforcement learning deficits may emerge early among those who are at clinical high risk for developing psychosis and that they are already pronounced by illness onset in the first episode. Importantly, these reinforcement learning deficits do not appear to be a byproduct of illness chronicity or antipsychotic medication use, but rather a consequence of the illness itself.

So Many Are "Few," but so Few Are Also "Few" - Reduced Semantic Flexibility in bvFTD Patients.

The processing of quantifier words such as “many” or “few” is a complex operation supported by a plastic fronto-parietal network predominantly in the left hemisphere. The internal reference criterion defining a quantifier (e.g., ≥50% for “many”) can be modified in a learning paradigm. Most interestingly, changing the criterion for one quantifier also leads to a change in the criterion for the untrained quantifier, i.e., a semantic restructuring effect, which is supported by Broca’s region in the left inferior frontal cortex. Here, we applied this paradigm to patients with the behavioral variant of fronto-temporal dementia (bvFTD) because they suffer from loss of cognitive flexibility, reduced ability to process quantities and their values, impaired reinforcement learning, and language comprehension deficits. The question was whether the patients would be able to perform the task, show direct learning of the new quantifier meanings, and exhibit cognitive flexibility in terms of semantic restructuring. Eleven bvFTD patients took part in two behavioral experiments. In Experiment 1, in a first baseline block, each individual’s criterion for “many” and “few” was assessed. In block 2, subjects received feedback about their decisions. Contrary to their initial notion, a proportion of 40% yellow circles was reinforced as “many.” In block 3, the effect of this training on their judgments of “many” and “few” was re-assessed. The group of bvFTD patients showed a learning effect for the new criterion trained for the quantifier “many,” but failed to generalize this criterion shift to the other quantifier “few.” Experiment 2 was similar to Experiment 1, but the patients were trained in Block 2 to judge 60% of circles as “few,” with no training for “many.” Again, there was an average learning effect for the trained quantifier “few” over the entire group, but no generalization to “many.” Since the patients were still able to perform the task and showed learning of “many” to direct feedback, the data suggest that the generalization process, rather than initial learning, is more vulnerable to fronto-temporal degeneration.

Impairments in reinforcement learning do not explain enhanced habit formation in cocaine use disorder

RationaleDrug addiction has been suggested to develop through drug-induced changes in learning and memory processes. Whilst the initiation of drug use is typically goal-directed and hedonically motivated, over time, drug-taking may develop into a stimulus-driven habit, characterised by persistent use of the drug irrespective of the consequences. Converging lines of evidence suggest that stimulant drugs facilitate the transition of goal-directed into habitual drug-taking, but their contribution to goal-directed learning is less clear. Computational modelling may provide an elegant means for elucidating changes during instrumental learning that may explain enhanced habit formation.ObjectivesWe used formal reinforcement learning algorithms to deconstruct the process of appetitive instrumental learning and to explore potential associations between goal-directed and habitual actions in patients with cocaine use disorder (CUD).MethodsWe re-analysed appetitive instrumental learning data in 55 healthy control volunteers and 70 CUD patients by applying a reinforcement learning model within a hierarchical Bayesian framework. We used a regression model to determine the influence of learning parameters and variations in brain structure on subsequent habit formation.ResultsPoor instrumental learning performance in CUD patients was largely determined by difficulties with learning from feedback, as reflected by a significantly reduced learning rate. Subsequent formation of habitual response patterns was partly explained by group status and individual variation in reinforcement sensitivity. White matter integrity within goal-directed networks was only associated with performance parameters in controls but not in CUD patients.ConclusionsOur data indicate that impairments in reinforcement learning are insufficient to account for enhanced habitual responding in CUD.

Beyond Motor Noise: Considering Other Causes of Impaired Reinforcement Learning in Cerebellar Patients.

Motor and reinforcement learning have been classically linked to functionally independent brain networks centered on the cerebellum and the basal ganglia, respectively. In a recent study published in eNeuro , Therrien et al. (2018) showed that increasing motor noise in healthy subjects disrupts reinforcement learning. However, this impairment remained well below that detected in cerebellar patients even when motor noise in healthy subjects was adjusted to match that observed in the patients. This suggests that impaired reinforcement learning following cerebellar damage cannot be solely accounted for by altered motor noise in these patients. Based on recent anatomic and functional evidence, we argue that the cerebellum may directly contribute to reinforcement learning, consistent with its tight connections with the basal ganglia. The ability to adapt to changes occurring in the environment is a fundamental feature of human behavior, which relies on both sensory and reward feedbacks. On the one hand, the role of sensory feedback has been largely considered by studying how motor commands adapt to visual perturbations (e.g., a visuomotor rotation), a process called error-based learning (Shadmehr et al., 2010; Wolpert et al., 2011; Kim et al., 2018; Roemmich and Bastian, 2018). This type of motor learning involves the computation of sensory prediction errors (SPEs), namely, the difference between predicted and actual sensory outcome (Tseng et al., 2007; Schlerf and Ivry, 2012; Shadmehr, 2017, 2018). On the other hand, the role of reward feedback has been mostly investigated in tasks that require learning what action to select or not, by updating reward predictions based on previous experience, a process named reinforcement learning (Lee et al., 2012; Derosiere et al., 2017a,b; Gershman and Daw, 2017; O’Doherty et al., 2017). A central …

Reinforcement learning in child molesters.

Child molesters form a heterogeneous group, but one generally shared characteristic is maladaptive, rigid behaviour. Impairments in reinforcement learning may explain these maladaptive tendencies, but this has not been systematically investigated. Further, it is not known if such impairments vary with subtype of child molesters. To investigate the presence of impairments in reinforcement learning among child molesters and to test for differences in patterns of impairment with subtype. A group of 59 child molesters was recruited from several prisons in a two-stage screening process, the first using records and the second interview; a comparison group of 33 offenders who had never committed a sex offence and who denied paedophile ideation was similarly recruited; 36 nonoffender comparison men were recruited by social media and word of mouth. Each was asked to perform a probabilistic reversal learning task, in which stimulus-outcome contingencies had to be learned. Child molesters, as a group, made significantly more errors on the probabilistic reversal learning task than the nonoffenders; the comparison offenders and the nonoffenders gained similar scores, although findings may have been confounded by older age in the child molester group. Nonpaedophilic child molesters had significantly worse scores than paedophilic child molesters. Child molesters, especially those not diagnosed with paedophilia, have deficits during both the acquisition and reversal of contingencies, suggesting reinforcement learning deficits that may undermine their capacity to benefit maximally from therapy without preliminary work to repair those deficits, possibly in conjunction with extending the offender programmes. Testing before programme entry would enable accurate targeting of scarce resources in this respect.

Motivational deficits in schizophrenia relate to abnormalities in cortical learning rate signals.

Individuals from across the psychosis spectrum display impairments in reinforcement learning. In some individuals, these deficits may result from aberrations in reward prediction error (RPE) signaling, conveyed by dopaminergic projections to the ventral striatum (VS). However, there is mounting evidence that VS RPE signals are relatively intact in medicated people with schizophrenia (PSZ). We hypothesized that, in PSZ, reinforcement learning deficits often are not related to RPE signaling per se but rather their impact on learning and behavior (i.e., learning rate modulation), due to dysfunction in anterior cingulate and dorsomedial prefrontal cortex (dmPFC). Twenty-six PSZ and 23 healthy volunteers completed a probabilistic reinforcement learning paradigm with occasional, sudden, shifts in contingencies. Using computational modeling, we found evidence of an impairment in trial-wise learning rate modulation (α) in PSZ before and after a reinforcement contingency shift, expressed most in PSZ with more severe motivational deficits. In a subsample of 22 PSZ and 22 healthy volunteers, we found little evidence for between-group differences in VS RPE and dmPFC learning rate signals, as measured with fMRI. However, a follow-up psychophysiological interaction analysis revealed decreased dmPFC-VS connectivity concurrent with learning rate modulation, most prominently in individuals with the most severe motivational deficits. These findings point to an impairment in learning rate modulation in PSZ, leading to a reduced ability to adjust task behavior in response to unexpected outcomes. At the level of the brain, learning rate modulation deficits may be associated with decreased involvement of the dmPFC within a greater RL network.

Feedback delay impaired reinforcement learning: Principal components analysis of Reward Positivity

An immediate feedback after action facilitated reinforcement learning in dynamically varying environments. With several seconds delay, a series of event-related potential (ERP) studies have recently conducted to explore how delayed feedback influences learning processes and corresponding brain activities by measuring the Reward Positivity and N170 component. However, it remains unclear how does our brain process a feedback that is delayed longer and interrupted by other trials. In the present study, participants were asked to undertake a time-estimation task in two different conditions. Feedback was presented right after their actions in the immediate feedback condition, while it was presented after another five trials in the delayed feedback condition. By recording feedback related activities, we aim to test whether, or not, delayed feedback impairs reinforcement learning, the Reward Positivity and N170 amplitude. The behavioural results show that delayed feedback can reduce behavioural adjustment efficiency from trial-to-trial. To reduce component overlapping, we adopted the temporospatial principal components analysis (PCA) to separate the Reward Positivity from other ERP components. Results indicate that the Reward Positivity is decreased in the delayed feedback condition compared to the immediate feedback condition, however, no difference of N170 amplitude is found between the two conditions. These results indicate that delayed feedback impairs reinforcement learning process in terms of behavioural adjustment and brain activities even though these feedbacks are truly associated with participants’ previous actions.

Striatal dopamine release and impaired reinforcement learning in adults with 22q11.2 deletion syndrome

22q11.2 deletion syndrome (22q11DS) is a genetic disorder caused by a microdeletion on chromosome 22q11.2 and associated with an increased risk for developing psychosis. The catechol-O-methyltransferase (COMT) gene is located in the deleted region and involved in dopamine (DA) breakdown. Impaired reinforcement learning (RL) is a recurrent feature in psychosis and thought to be related to abnormal striatal DA function. This study aims to examine RL and the potential association with striatal DA-ergic neuromodulation in 22q11DS. Twelve non-psychotic adults with 22q11DS and 16 healthy controls (HC) were included. A dopamine D2/3 receptor [18F]fallypride positron emission tomography (PET) scan was acquired while participants performed a modified version of the probabilistic stimulus selection task. RL-task performance was significantly worse in 22q11DS compared to HC. There were no group difference in striatal nondisplaceable binding potential (BPND) and task-induced DA release. In HC, striatal task-induced DA release was positively associated with task performance, but no such relation was found in 22q11DS subjects. Moreover, higher caudate nucleus task-induced DA release was found in COMT Met hemizygotes relative to Val hemizygotes. This study is the first to show impairments in RL in 22q11DS. It suggests that potentially motivational impairments are not only present in psychosis, but also in this genetic high risk group. These deficits may be underlain by abnormal striatal task-induced DA release, perhaps as a consequence of COMT haplo-insufficiency.