Reinforcement Learning Deficits Research Articles

Disentangling negative reinforcement, working memory, and deductive reasoning deficits in elevated BMI

Neuropsychological data suggest that being overweight or obese is associated with a tendency to perseverate behavior despite negative feedback. This deficit might be observed due to other cognitive factors, such as working memory (WM) deficits or decreased ability to deduce model-based strategies when learning by trial-and-error. In the present study, a group of subjects with overweight or obesity (Ow/Ob, n = 30) was compared to normal-weight individuals (n = 42) in a modified Reinforcement Learning (RL) task. The task was designed to control WM effects on learning by manipulating cognitive load and to foster model-based learning via deductive reasoning. Computational modelling and analysis were conducted to isolate parameters related to RL mechanisms, WM use, and model-based learning (deduction parameter). Results showed that subjects with Ow/Ob had a higher number of perseverative errors and used a weaker deduction mechanism in their performance than control individuals, indicating impairments in negative reinforcement and model-based learning, whereas WM impairments were not responsible for deficits in RL. The present data suggests that obesity is associated with impairments in negative reinforcement and model-based learning.

Computational Modeling Differentiates Learning Rate From Reward Sensitivity Deficits Produced by Early-Life Adversity in a Rodent Touchscreen Probabilistic Reward Task

BackgroundExposure to adversity, including unpredictable environments, during early life is associated with neuropsychiatric illness in adulthood. One common factor in this sequela is anhedonia, the loss of responsivity to previously reinforcing stimuli. To accelerate the development of new treatment strategies for anhedonic disorders induced by early-life adversity, animal models have been developed to capture critical features of early-life stress and the behavioral deficits that such stressors induce. We have previously shown that rats exposed to the limited bedding and nesting protocol exhibited blunted reward responsivity in the probabilistic reward task, a touchscreen-based task reverse translated from human studies. MethodsTo test the quantitative limits of this translational platform, we examined the ability of Bayesian computational modeling and probability analyses identical to those optimized in previous human studies to quantify the putative mechanisms that underlie these deficits with precision. Specifically, 2 parameters that have been shown to independently contribute to probabilistic reward task outcomes in patient populations, reward sensitivity and learning rate, were extracted, as were trial-by-trial probability analyses of choices as a function of the preceding trial. ResultsSignificant deficits in reward sensitivity, but not learning rate, contributed to the anhedonic phenotypes in rats exposed to early-life adversity. ConclusionsThe current findings confirm and extend the translational value of these rodent models by verifying the effectiveness of computational modeling in distinguishing independent features of reward sensitivity and learning rate that complement the probabilistic reward task’s signal detection end points. Together, these metrics serve to objectively quantify reinforcement learning deficits associated with anhedonic phenotypes.

Decomposition of Reinforcement Learning Deficits in Disordered Gambling via Drift Diffusion Modeling and Functional Magnetic Resonance Imaging.

Gambling disorder is associated with deficits in reward-based learning, but the underlying computational mechanisms are still poorly understood. Here, we examined this issue using a stationary reinforcement learning task in combination with computational modeling and functional resonance imaging (fMRI) in individuals that regular participate in gambling (n = 23, seven fulfilled one to three DSM 5 criteria for gambling disorder, sixteen fulfilled four or more) and matched controls (n = 23). As predicted, the gambling group exhibited substantially reduced accuracy, whereas overall response times (RTs) were not reliably different between groups. We then used comprehensive modeling using reinforcement learning drift diffusion models (RLDDMs) in combination with hierarchical Bayesian parameter estimation to shed light on the computational underpinnings of this performance deficit. In both groups, an RLDDM in which both non-decision time and decision threshold (boundary separation) changed over the course of the experiment accounted for the data best. The model showed good parameter and model recovery, and posterior predictive checks revealed that, in both groups, the model accurately reproduced the evolution of accuracies and RTs over time. Modeling revealed that, compared to controls, the learning impairment in the gambling group was linked to a more rapid reduction in decision thresholds over time, and a reduced impact of value-differences on the drift rate. The gambling group also showed shorter non-decision times. FMRI analyses replicated effects of prediction error coding in the ventral striatum and value coding in the ventro-medial prefrontal cortex, but there was no credible evidence for group differences in these effects. Taken together, our findings show that reinforcement learning impairments in disordered gambling are linked to both maladaptive decision threshold adjustments and a reduced consideration of option values in the choice process.

Differential deficits in social versus monetary reinforcement learning in schizophrenia: Associations with facial emotion recognition.

Despite evidence that individuals with schizophrenia (SZ) have an intact desire for social relationships, they have small social networks and report high levels of loneliness. Difficulty with reinforcement learning (RL), the ability to update behavior based on feedback, may inhibit the formation and maintenance of social relationships in SZ. However, impaired RL in SZ has largely been demonstrated via monetary tasks. Thus, it remains unclear whether SZ are similarly impaired in social and monetary RL, or whether social-specific factors may further inhibit their ability to learn from social feedback. Thirty-one individuals with SZ and 31 healthy controls (HCs) participated in a RL paradigm to test hypotheses about social versus monetary RL. SZ exhibited impaired RL compared to HCs in both social and monetary tasks. Further, a Group × Task interaction demonstrated that SZ was more impaired when learning from social than monetary reinforcement, F(1, 59) = 5.99, p = .017. This differential deficit to social RL was not accounted for by reported pleasure from social feedback, which did not differ between groups. Instead, SZ had poorer emotion recognition than HCs, t(1, 60) = 4.80, p < .001, particularly for negative emotions, and controlling for this eliminated the differential social RL impairment. These results suggest the possibility that difficulty recognizing social cues, especially those indicating negative feedback, may relate to a reduced ability to learn from others' feedback. Thus, future research could elucidate whether targeting these emotion recognition difficulties in treatment could serve as a potential mechanism for improving social functioning in SZ. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Maternal selenium dietary supplementation alters sociability and reinforcement learning deficits induced by in utero exposure to maternal immune activation in mice

Maternal immune activation (MIA) during pregnancy increases the risk for the unborn foetus to develop neurodevelopmental conditions such as autism spectrum disorder and schizophrenia later in life. MIA mouse models recapitulate behavioural and biological phenotypes relevant to both conditions, and are valuable models to test novel treatment approaches. Selenium (Se) has potent anti-inflammatory properties suggesting it may be an effective prophylactic treatment against MIA. The aim of this study was to determine if Se supplementation during pregnancy can prevent adverse effects of MIA on offspring brain and behaviour in a mouse model.Selenium was administered via drinking water (1.5 ppm) to pregnant dams from gestational day (GD) 9 to birth, and MIA was induced at GD17 using polyinosinic:polycytidylic acid (poly-I:C, 20 mg/kg via intraperitoneal injection). Foetal placenta and brain cytokine levels were assessed using a Luminex assay and brain elemental nutrients assessed using inductively coupled plasma- mass spectrometry. Adult offspring were behaviourally assessed using a reinforcement learning paradigm, the three-chamber sociability test and the open field test. MIA elevated placental IL-1β and IL-17, and Se supplementation successfully prevented this elevation. MIA caused an increase in foetal brain calcium, which was prevented by Se supplement. MIA caused in offspring a female-specific reduction in sociability, which was recovered by Se, and a male-specific reduction in social memory, which was not recovered by Se. Exposure to poly-I:C or selenium, but not both, reduced performance in the reinforcement learning task. Computational modelling indicated that this was predominantly due to increased exploratory behaviour, rather than reduced rate of learning the location of the food reward.This study demonstrates that while Se may be beneficial in ameliorating sociability deficits caused by MIA, it may have negative effects in other behavioural domains. Caution in the use of Se supplementation during pregnancy is therefore warranted.

Supplemental Material for Differential Deficits in Social Versus Monetary Reinforcement Learning in Schizophrenia: Associations With Facial Emotion Recognition

Supplemental Material for Differential Deficits in Social Versus Monetary Reinforcement Learning in Schizophrenia: Associations With Facial Emotion Recognition

Validation of a touchscreen probabilistic reward task for mice: A reverse-translated assay with cross-species continuity.

The Probabilistic Reward Task (PRT) is a laboratory-based technique used to objectively quantify responsivity to reward. The PRT was initially designed to identify reinforcement learning deficits in clinical populations and subsequently was reverse-translated for use in preclinical studies with rats and monkeys. In this task, subjects make visual discriminations and asymmetric probabilistic contingencies are arranged such that correct responses to one stimulus (rich) are reinforced more often than correct responses to the other (lean). Numerous studies have demonstrated that healthy subjects reliably develop a response bias toward the richly rewarded stimulus, whereas humans with anhedonia and laboratory animals with a history of chronic stress exhibit a blunted response bias. This is important because anhedonia, the loss of responsivity to previously rewarding stimuli, is a behavioral phenotype that is a cardinal feature of multiple neuropsychiatric conditions and is without approved pharmacotherapeutic options. To aid in addressing this critical treatment gap, this report describes validation of the first PRT designed for mice, which are a commonly utilized species in preclinical research toward neuropsychiatric medications development. Results reveal orderly psychophysical functions in response to asymmetric probabilistic contingencies in mice, with signal detection outcomes comparable to previous PRT findings in humans, rats, and monkeys. Taken together, such robust cross-species continuity in task performance confirms that the mouse is well-positioned to serve in bidirectional research efforts between human and animal laboratories. These efforts may accelerate the development of treatment options for anhedonia in the different neuropsychiatric conditions in which it is prominent.

Leveraging Decision Science to Characterize Depression

This brief review examines the potential to use decision science to objectively characterize depression. We provide a brief overview of the existing literature examining different domains of decision-making in depression. Because this overview highlights the specific role of reinforcement learning as an important decision process affected in the disorder, we then introduce reinforcement learning modeling and explain how this approach has identified specific reinforcement learning deficits in depression. We conclude with ideas for future research at the intersection of decision science and depression, emphasizing the potential for decision science to help uncover underlying mechanisms and targets for the treatment of depression.

Impaired Punishment Learning in Conduct Disorder

Conduct disorder (CD) has been associated with deficits in the use of punishment to guide reinforcement learning (RL) and decision making. This may explain the poorly planned and often impulsive antisocial and aggressive behavior in affected youths. Here, we used a computational modeling approach to examine differences in RL abilities between CD youths and typically developing controls (TDCs). Specifically, we tested 2 competing hypotheses that RL deficits in CD reflect either reward dominance (also known as reward hypersensitivity) or punishment insensitivity (also known as punishment hyposensitivity). The study included 92 CD youths and 130 TDCs (aged 9-18 years, 48% girls) who completed a probabilistic RL task with reward, punishment, and neutral contingencies. Using computational modeling, we investigated the extent to which the 2 groups differed in their learning abilities to obtain reward and/or to avoid punishment. RL model comparisons showed that a model with separate learning rates per contingency explained behavioral performance best. Importantly, CD youths showed lower learning rates than TDCs specifically for punishment, whereas learning rates for reward and neutral contingencies did not differ. Moreover, callous-unemotional (CU) traits did not correlate with learning rates in CD. CD youths have a highly selective impairment in probabilistic punishment learning, regardless of their CU traits, whereas reward learning appears to be intact. In summary, our data suggest punishment insensitivity rather than reward dominance in CD. Clinically, the use of punishment-based intervention techniques to achieve effective discipline in patients with CD may be a less helpful strategy than reward-based techniques.

Perceived chronic stress and impulsivity are associated with reduced learning about the costs and benefits of actions

Many psychiatric conditions have been linked with deficits in cost-benefit reinforcement learning. However, previous results have been mixed, partly due to significant symptom heterogeneity within distinct psychiatric conditions and symptom overlap between them, making it difficult to disentangle whether alterations in cost-benefit reinforcement learning are condition- or symptom-specific. Here, we investigate whether transdiagnostic (sub-)clinical symptoms and risk factors for psychopathology are associated with reinforcement learning and cost-benefit integration. For this reason, we use an online cost-benefit reinforcement learning task in combination with self-rated measures of common transdiagnostic factors (perceived chronic stress, anhedonia, impulsivity, energy/fatigue) in 360 subjects (18–46 years old) with(out) a diagnosis of a psychiatric condition. Increased chronic stress and impulsivity were associated with poorer reinforcement learning, independent of whether participants were learning to minimize costs (physical effort) or maximize benefits (monetary rewards). These associations were selectively driven by a reduction in learning from positive and negative reinforcement, not punishment. The use of mobile phone (compared to laptop/PC) was also associated with lower performance accuracy. Data and scripts are available (https://osf.io/w3mvq/). Our work emphasizes the importance of chronic stress and impulsivity as potential drivers of altered motivation and goal-directed behavior beyond diagnostic labels, in addition to methodological challenges associated with data collection via online platforms.

Instrumental learning and behavioral persistence in children with attention-deficit/hyperactivity-disorder: does reinforcement frequency matter?

Prominent theoretical accounts of attention-deficit/hyperactivity-disorder (ADHD) hypothesize that reinforcement learning deficits underlie symptoms of ADHD. The Dynamic Developmental Theory and the Dopamine Transfer Deficit hypothesis assume impairments in both the acquisition and extinction of behavior, especially when learning occurs under partial (non-continuous) reinforcement, and subsequently the Partial Reinforcement Extinction Effect (PREE). Few studies have evaluated instrumental learning in ADHD and the results are inconsistent. The current study investigates instrumental learning under partial and continuous reinforcement schedules and subsequent behavioral persistence when reinforcement is withheld (extinction) in children with and without ADHD. Large well-defined samples of children with ADHD (n = 93) and typically developing (TD) children (n = 73) completed a simple instrumental learning task. The children completed acquisition under continuous (100%) or partial (20%) reinforcement, followed by a 4-min extinction phase. Two-way (diagnosis by condition) ANOVAs evaluated responses needed to reach the learning criterion during acquisition, and target and total responses during extinction. Children with ADHD required more trials to reach criterion compared to TD children under both continuous and partial reinforcement. After partial reinforcement, children with ADHD executed fewer target responses during extinction than TD children. Children with ADHD executed more responses than TD children during extinction, irrespective of learning condition. The findings demonstrate general difficulties in instrumental learning in ADHD, that is, slower learning irrespective of reinforcement schedule. They also show faster extinction following learning under partial reinforcement in those with ADHD, that is, a diminished PREE. Children with ADHD executed more responses during extinction. Results are theoretically important, with clinical implications for understanding and managing learning difficulties in those with ADHD, as they suggest poorer reinforcement learning and lower behavioral persistence.

Reinforcement learning deficits exhibited by postnatal PCP-treated rats enable deep neural network classification.

The ability to appropriately update the value of a given action is a critical component of flexible decision making. Several psychiatric disorders, including schizophrenia, are associated with impairments in flexible decision making that can be evaluated using the probabilistic reversal learning (PRL) task. The PRL task has been reverse-translated for use in rodents. Disrupting glutamate neurotransmission during early postnatal neurodevelopment in rodents has induced behavioral, cognitive, and neuropathophysiological abnormalities relevant to schizophrenia. Here, we tested the hypothesis that using the NMDA receptor antagonist phencyclidine (PCP) to disrupt postnatal glutamatergic transmission in rats would lead to impaired decision making in the PRL. Consistent with this hypothesis, compared to controls the postnatal PCP-treated rats completed fewer reversals and exhibited disruptions in reward and punishment sensitivity (i.e., win-stay and lose-shift responding, respectively). Moreover, computational analysis of behavior revealed that postnatal PCP-treatment resulted in a pronounced impairment in the learning rate throughout PRL testing. Finally, a deep neural network (DNN) trained on the rodent behavior could accurately predict the treatment group of subjects. These data demonstrate that disrupting early postnatal glutamatergic neurotransmission impairs flexible decision making and provides evidence that DNNs can be trained on behavioral datasets to accurately predict the treatment group of new subjects, highlighting the potential for DNNs to aid in the diagnosis of schizophrenia.

Reinforcement learning abnormalities in the attenuated psychosis syndrome and first episode psychosis

Prior studies indicate that chronic schizophrenia (SZ) is associated with a specific profile of reinforcement learning abnormalities. These impairments are characterized by: 1) reductions in learning rate, and 2) impaired Go learning and intact NoGo learning. Furthermore, each of these deficits are associated with greater severity of negative symptoms, consistent with theoretical perspectives positing that avolition and anhedonia are associated with impaired value representation. However, it is unclear whether these deficits extend to earlier phases of psychotic illness and when individuals are unmedicated. Two studies were conducted to examine reinforcement learning deficits in earlier phases of psychosis and in high risk patients. In study 1, participants included 35 participants with first episode psychosis (FEP) with limited antipsychotic medication exposure and 25 healthy controls (HC). Study 2 included 17 antipsychotic naïve individuals who were at clinical high-risk for psychosis (CHR) (i.e., attenuated psychosis syndrome) and 18 matched healthy controls (HC). In both studies, participants completed the Temporal Utility Integration Task, a measure of probabilistic reinforcement learning that contained Go and NoGo learning blocks. FEP displayed impaired Go and NoGo learning. In contrast, CHR did not display impairments in Go or NoGo learning. Impaired Go learning was not significantly associated with clinical outcomes in the CHR or FEP samples. Findings provide new evidence for areas of spared and impaired reinforcement learning in early phases of psychosis.

Are methamphetamine users compulsive? Faulty reinforcement learning, not inflexibility, underlies decision making in people with methamphetamine use disorder.

Methamphetamine use disorder involves continued use of the drug despite negative consequences. Such 'compulsivity' can be measured by reversal learning tasks, which involve participants learning action-outcome task contingencies (acquisition-contingency) and then updating their behaviour when the contingencies change (reversal). Using these paradigms, animal models suggest that people with methamphetamine use disorder (PwMUD) may struggle to avoid repeating actions that were previously rewarded but are now punished (inflexibility). However, difficulties in learning task contingencies (reinforcement learning) may offer an alternative explanation, with meaningful treatment implications. We aimed to disentangle inflexibility and reinforcement learning deficits in 35 PwMUD and 32 controls with similar sociodemographic characteristics, using novel trial-by-trial analyses on a probabilistic reversal learning task. Inflexibility was defined as (a) weaker reversal phase performance, compared with the acquisition-contingency phases, and (b) persistence with the same choice despite repeated punishments. Conversely, reinforcement learning deficits were defined as (a) poor performance across both acquisition-contingency and reversal phases and (b) inconsistent postfeedback behaviour (i.e., switching after reward). Compared with controls, PwMUD exhibited weaker learning (odds ratio [OR]=0.69, 95% confidence interval [CI] [0.63-0.77], p<.001), though no greater accuracy reduction during reversal. Furthermore, PwMUD were more likely to switch responses after one reward/punishment (OR=0.83, 95% CI [0.77-0.89], p<.001; OR=0.82, 95% CI [0.72-0.93], p=.002) but just as likely to switch after repeated punishments (OR=1.03, 95% CI [0.73-1.45], p=.853). These results indicate that PwMUD's reversal learning deficits are driven by weaker reinforcement learning, not inflexibility.

Reward Learning Through the Lens of RDoC: a Review of Theory, Assessment, and Empirical Findings in the Eating Disorders.

Reward-related processes may represent important transdiagnostic factors underlying eating pathology. Using the NIMH Research Domain Criteria as a guide, the current article reviews theories, behavioral and self-report assessments, and empirical findings related to reward learning in the eating disorders. Data from behavioral tasks suggest deficits in reinforcement learning, which may become more pronounced with increasing disorder severity and duration. Self-report data strongly implicate positive eating and thinness/restriction expectancies (an element of reward prediction error) in the onset and maintenance of eating pathology. Finally, self-report measures of habit strength demonstrate relationships with eating pathology and illness duration; however, behavioral task data do not support relationships between eating pathology and a propensity towards general habit development. Existing studies are limited, but provide preliminary support for the presence of abnormal reward learning in eating disorders. Continued research is needed to address identified gaps in the literature.

Blunted reward prediction error signals in internet gaming disorder.

Internet gaming disorder (IGD) is a type of behavioural addictions. One of the key features of addiction is the excessive exposure to addictive objectives (e.g. drugs) reduces the sensitivity of the brain reward system to daily rewards (e.g. money). This is thought to be mediated via the signals expressed as dopaminergic reward prediction error (RPE). Emerging evidence highlights blunted RPE signals in drug addictions. However, no study has examined whether IGD also involves alterations in RPE signals that are observed in other types of addictions. To fill this gap, we used functional magnetic resonance imaging data from 45 IGD and 42 healthy controls (HCs) during a reward-related prediction-error task and utilised a psychophysiological interaction (PPI) analysis to characterise the underlying neural correlates of RPE and related functional connectivity. Relative to HCs, IGD individuals showed impaired reinforcement learning, blunted RPE signals in multiple regions of the brain reward system, including the right caudate, left orbitofrontal cortex (OFC), and right dorsolateral prefrontal cortex (DLPFC). Moreover, the PPI analysis revealed a pattern of hyperconnectivity between the right caudate, right putamen, bilateral DLPFC, and right dorsal anterior cingulate cortex (dACC) in the IGD group. Finally, linear regression suggested that the connection between the right DLPFC and right dACC could significantly predict the variation of RPE signals in the left OFC. These results highlight disrupted RPE signalling and hyperconnectivity between regions of the brain reward system in IGD. Reinforcement learning deficits may be crucial underlying characteristics of IGD pathophysiology.

Selective Mesoaccumbal Pathway Inactivation Affects Motivation but Not Reinforcement-Based Learning in Macaques

Rodent studies have demonstrated the role of the mesoaccumbal circuit in reinforcement-based learning. Importantly, however, while phasic activity of the ventral tegmental area (VTA) contributes to reinforcement learning, rodent evidence suggests that slow changes in tonic VTA activity and associated accumbal dopamine release help regulate motivational behavior. Nonetheless, the consequences of sustained blockage of the mesoaccumbal circuit for motivation and reinforcement learning have not yet been examined in primates. Using a double-infection viral vector technique, we demonstrate that selective, unidirectional, and reversible blockage of the primarily dopaminergic mesoaccumbal circuit in monkeys increased network-level functional connectivity, especially in fronto-temporal cortex. These global network changes were not associated with deficits in reinforcement learning during an object discrimination reversal task. In contrast, sustained mesoaccumbal inactivation greatly reduced motivation for performing a motivation-based decision-making task. Thus, the mesoaccumbal pathway in primates is critical for high-effort motivation but not for all forms of reinforcement-based learning.

Dynamic changes of brain networks during feedback-related processing of reinforcement learning in schizophrenia

Previous studies have reported that schizophrenia (SZ) patients showed selective reinforcement learning deficits and abnormal feedback-related event-related potential (ERP) components. However, how the brain networks and their topological properties evolve over time during transient feedback-related cognition processing in SZ patients has not been investigated so far. In this paper, using publicly available feedback-related ERP data which were recorded from SZ patients and healthy controls (HC) when they performed a reinforcement learning task, we carried out an event-related network analysis where topology of brain functional networks was characterized with some graph measures including clustering coefficient (C), global efficiency (Eglobal) and local efficiency (Elocal) on a millisecond timescale. Our results showed that the brain functional networks displayed rapid rearrangements of topological properties during transient feedback-related cognition process for both two groups. More importantly, we found that SZ patients exhibited significantly reduced theta-band (time window of 170–350 ms after stimuli onset) brain functional connectivity strength, Eglobal, Elocal and C in response to negative feedback stimuli compared to HC group. The network based statistic (NBS) analysis detected one significantly decreased theta-band subnetwork in SZ patients mainly involving in frontal-occipital and temporal-occipital connections compared to HC group. In addition, clozapine treatment seemed to greatly reduce theta-band power and topological measures of brain networks in SZ patients. Finally, the theta-band power, graph measures and functional connectivity were extracted to train a support vector machine classifier for classification of HC from SZ, or Cloz + SZ or Cloz- SZ, and a relatively good classification accuracy of 84.48%, 89.47% and 78.26% was obtained, respectively. The above results suggested a less optimal organization of theta-band brain network in SZ patients, and studying the topological parameters of brain networks evolve over time during transient feedback-related processing could be useful for understanding the pathophysiologic mechanisms underlying reinforcement learning deficits in SZ patients.

S79. REINFORCEMENT LEARNING ABNORMALITIES IN INDIVIDUALS AT CLINICAL HIGH RISK AND IN THE FIRST EPISODE OF PSYCHOSIS

BackgroundPrior studies indicate that chronic schizophrenia (SZ) is associated with a specific profile of reinforcement learning abnormalities. These impairments are characterized by: 1) reductions in learning rate, and 2) impaired Go learning and intact NoGo learning. Furthermore, each of these deficits are associated with greater severity of negative symptoms, consistent with theoretical perspectives positing that avolition and anhedonia are associated with deficits in generating, updating, and maintaining mental representations of reward value hat are needed to guide decision-making. However, it is unclear whether these deficits extend to earlier phases of psychotic illness and when individuals are unmedicated.MethodsTwo studies were conducted to examine reinforcement learning deficits in earlier phases of psychosis. In study 1, participants included 35 participants with first episode psychosis (FEP) and 25 healthy controls (HC). Study 2 included 17 antipsychotic naïve individuals who met criteria for attenuated psychosis syndrome (APS) (i.e., those with a prodromal syndrome) and 18 matched healthy controls (HC). In both studies, participants completed the Temporal Utility Integration Task, a measure of probabilistic reinforcement learning that contained Go and NoGo learning blocks. Participants in the clinical groups also completed neuropsychological testing and standard clinical interviews designed to determine symptom severity and diagnosis.ResultsFEP displayed impaired Go learning and intact NoGo learning. In contrast, APS did not display impairments in Go or NoGo learning at the group level. Negative symptoms were not significantly associated with reinforcement learning in APS participants. However, greater impairments in Go learning were associated with increased cross-sectional risk for conversion on the NAPLS risk calculator score in the APS group.DiscussionFindings provide new evidence for areas of spared and impaired reinforcement learning in early phases of psychosis. Similar to chronic SZ, FEP was associated with impaired Go learning, and intact NoGo learning. Reinforcement learning is more spared in those at clinical high-risk, except those at greatest risk for conversion, where Go learning deficits are more pronounced. These findings suggest that reinforcement learning deficits may emerge early among those who are at clinical high risk for developing psychosis and that they are already pronounced by illness onset in the first episode. Importantly, these reinforcement learning deficits do not appear to be a byproduct of illness chronicity or antipsychotic medication use, but rather a consequence of the illness itself.

Reinforcement learning as an intermediate phenotype in psychosis? Deficits sensitive to illness stage but not associated with polygenic risk of schizophrenia in the general population

BackgroundSchizophrenia is a complex disorder in which the causal relations between risk genes and observed clinical symptoms are not well understood and the explanatory gap is too wide to be clarified without considering an intermediary level. Thus, we aimed to test the hypothesis of a pathway from molecular polygenic influence to clinical presentation occurring via deficits in reinforcement learning. MethodsWe administered a reinforcement learning task (Go/NoGo) that measures reinforcement learning and the effect of Pavlovian bias on decision making. We modelled the behavioural data with a hierarchical Bayesian approach (hBayesDM) to decompose task performance into its underlying learning mechanisms. Study 1 included controls (n = 29, F|M = 0.81), At Risk Mental State for psychosis (ARMS, n = 23, F|M = 0.35) and FEP (First-episode psychosis, n = 26, F|M = 0.18). Study 2 included healthy adolescents (n = 735, F|M = 1.06), 390 of whom had their polygenic risk scores for schizophrenia (PRSs) calculated. ResultsPatients with FEP showed significant impairments in overriding Pavlovian conflict, a lower learning rate and a lower sensitivity to both reward and punishment. Less widespread deficits were observed in ARMS. PRSs did not significantly predict performance on the task in the general population, which only partially correlated with measures of psychopathology. ConclusionsReinforcement learning deficits are observed in first episode psychosis and, to some extent, in those at clinical risk for psychosis, and were not predicted by molecular genetic risk for schizophrenia in healthy individuals. The study does not support the role of reinforcement learning as an intermediate phenotype in psychosis.