Probabilistic Associative Learning Research Articles

Altered Reinforcement Learning from Reward and Punishment in Anorexia Nervosa: Evidence from Computational Modeling.

Anorexia nervosa (AN) is associated with altered sensitivity to reward and punishment. Few studies have investigated whether this results in aberrant learning. The ability to learn from rewarding and aversive experiences is essential for flexibly adapting to changing environments, yet individuals with AN tend to demonstrate cognitive inflexibility, difficulty set-shifting and altered decision-making. Deficient reinforcement learning may contribute to repeated engagement in maladaptive behavior. This study investigated learning in AN using a probabilistic associative learning task that separated learning of stimuli via reward from learning via punishment. Forty-two individuals with Diagnostic and Statistical Manual of Mental Disorders (DSM)-5 restricting-type AN were compared to 38 healthy controls (HCs). We applied computational models of reinforcement learning to assess group differences in learning, thought to be driven by violations in expectations, or prediction errors (PEs). Linear regression analyses examined whether learning parameters predicted BMI at discharge. AN had lower learning rates than HC following both positive and negative PE (p<.02), and were less likely to exploit what they had learned. Negative PE on punishment trials predicted lower discharge BMI (p<.001), suggesting individuals with more negative expectancies about avoiding punishment had the poorest outcome. This is the first study to show lower rates of learning in AN following both positive and negative outcomes, with worse punishment learning predicting less weight gain. An inability to modify expectations about avoiding punishment might explain persistence of restricted eating despite negative consequences, and suggests that treatments that modify negative expectancy might be effective in reducing food avoidance in AN.

The Computational, Pharmacological, and Physiological Determinants of Sensory Learning under Uncertainty

SummaryThe ability to represent and respond to uncertainty is fundamental to human cognition and decision-making. Noradrenaline (NA) is hypothesized to play a key role in coordinating the sensory, learning, and physiological states necessary to adapt to a changing world, but direct evidence for this is lacking in humans. Here, we tested the effects of attenuating noradrenergic neurotransmission on learning under uncertainty. We probed the effects of the β-adrenergic receptor antagonist propranolol (40 mg) using a between-subjects, double-blind, placebo-controlled design. Participants performed a probabilistic associative learning task, and we employed a hierarchical learning model to formally quantify prediction errors about cue-outcome contingencies and changes in these associations over time (volatility). Both unexpectedness and noise slowed down reaction times, but propranolol augmented the interaction between these main effects such that behavior was influenced more by prior expectations when uncertainty was high. Computationally, this was driven by a reduction in learning rates, with people slower to update their beliefs in the face of new information. Attenuating the global effects of NA also eliminated the phasic effects of prediction error and volatility on pupil size, consistent with slower belief updating. Finally, estimates of environmental volatility were predicted by baseline cardiac measures in all participants. Our results demonstrate that NA underpins behavioral and computational responses to uncertainty. These findings have important implications for understanding the impact of uncertainty on human biology and cognition.

S61. COMPUTATIONAL MODELLING OF VISUAL MOTION PERCEPTION AND ITS ASSOCIATION WITH SCHIZOTYPAL TRAITS

BackgroundPsychotic symptoms might be explained by disturbances of information processing due to errors of inference during neural coding, and hierarchical models could advance our understanding of how impaired functioning at different levels of the processing hierarchy are associated with psychotic symptoms. However, in order to examine to what extent such alterations are temporary or stable, the psychometric reliability and validity of the measurements need to be established. Individual differences in visual perception were measured by responses to uncertain stimuli presented during a probabilistic associative learning task. Our novel contributions are the measurement of cross-modal (visual and acoustic) associative learning and the assessment of the psychometric properties of indicators derived from a perceptual decision task: we evaluate its internal consistency, test-retest reliability and external validity as shown by associations with schizotypal traits.MethodsParticipants (32 healthy individuals, 13 men, age (SD) = 27.4 (9.4)) performed a perceptual decision task twice with one week delay. They were asked to indicate the direction of perceived motion of disambiguous and ambiguous visual stimuli (640 trials), which were preceded by visual and acoustic cues that were probabilistically associated with the motion direction and were congruent (both predict the same motion) or incongruent (cues predict different motion). Schizotypal traits were measured with the short version of the Oxford-Liverpool Inventory of Feelings and Experiences (O-LIFE) questionnaire, which showed good internal consistency and test-retest reliability (Cronbach’s alpha: 0.71 – 0.83 for subscales, test-retest correlation for Cognitive Disorganization: r = 0.84, and Unusual Experiences: r = 0.79).ResultsWe found a significant difference in response reaction times between stimuli with high and low probability (t = -2.037; p = 0.044). Acoustic cues predicted the decision significantly higher in case of ambiguous stimuli in both sessions (1. t=4.19, p<0.001; 2: t=3.46, p=0.002). Congruency of visual and acoustic cue pairs had no significant effect on response times for ambiguous stimuli. Reaction times and bias towards reliance on auditory cues during perceptual decision making under uncertainty showed stability over the two sessions (test-retest rho’s ranging from 0.56 – 0.72). Cognitive Disorganization scores showed weak negative correlation with response time under uncertainty (session 1: r= -0.24, session 2: r= -0.28), Unusual Experiences scores showed weak negative correlation with the bias towards reliance on auditory cues (session1: r= -0.21, session 2: r= -0.19). We did not find relationship between general response speed and any O-LIFE subscale scores.DiscussionThe results show some intraindividual stability of individual differences in perceptual decision making as measured by our paradigm. Participants with higher schiztypal scores tend to have slower response speed under uncertainty and greater bias towards reliance on auditory cues in a small healthy sample which implies it might be useful to measure these variables in clinical population and evaluate the effectiveness of therapeutic interventions or illness progression in follow-up studies.The presented preliminary results derived from descriptive statistics of the behavioral data. Our research group is currently working on fitting a trial-by-trial hierarchical computational model - which includes the representation of uncertainty - to find more detailed individual differences, e.g. the time course of parameter changes while learning in a visual perception task.

Altered Reinforcement Learning From Reward and Punishment in Anorexia Nervosa: Evidence From Computational Modeling

OBJECTIVES Anorexia nervosa (AN) is associated with altered sensitivity to reward and punishment. Few studies have investigated whether this results in aberrant learning. The ability to learn from rewarding and aversive experiences is essential for flexibly adapting to changing environments, yet individuals with AN tend to demonstrate cognitive inflexibility, difficulty set-shifting and altered decision-making. Deficient reinforcement learning may contribute to repeated engagement in maladaptive behavior. METHODS This study investigated learning in AN using a probabilistic associative learning task that separated learning of stimuli via reward from learning via punishment. Forty-two individuals with Diagnostic and Statistical Manual of Mental Disorders (DSM)-5 restricting-type AN were compared to 38 healthy controls (HCs). We applied computational models of reinforcement learning to assess group differences in learning, thought to be driven by violations in expectations, or prediction errors (PEs). Linear regression analyses examined whether learning parameters predicted BMI at discharge. RESULTS AN had lower learning rates than HC following both positive and negative PE (p < .02), and were less likely to exploit what they had learned. Negative PE on punishment trials predicted lower discharge BMI (p < .001), suggesting individuals with more negative expectancies about avoiding punishment had the poorest outcome. CONCLUSIONS This is the first study to show lower rates of learning in AN following both positive and negative outcomes, with worse punishment learning predicting less weight gain. An inability to modify expectations about avoiding punishment might explain persistence of restricted eating despite negative consequences, and suggests that treatments that modify negative expectancy might be effective in reducing food avoidance in AN.

A Computational Role for Top-Down Modulation from Frontal Cortex in Infancy.

Recent findings have shown that full-term infants engage in top-down sensory prediction, and these predictions are impaired as a result of premature birth. Here, we use an associative learning model to uncover the neuroanatomical origins and computational nature of this top-down signal. Infants were exposed to a probabilistic audiovisual association. We find that both groups (full term, preterm) have a comparable stimulus-related response in sensory and frontal lobes and track prediction error in their frontal lobes. However, preterm infants differ from their full-term peers in weaker tracking of prediction error in sensory regions. We infer that top-down signals from the frontal lobe to the sensory regions carry information about prediction error. Using computational learning models and comparing neuroimaging results from full-term and preterm infants, we have uncovered the computational content of top-down signals in young infants when they are engaged in a probabilistic associative learning.

Probabilistic associative learning suffices for learning the temporal structure of multiple sequences.

From memorizing a musical tune to navigating a well known route, many of our underlying behaviors have a strong temporal component. While the mechanisms behind the sequential nature of the underlying brain activity are likely multifarious and multi-scale, in this work we attempt to characterize to what degree some of this properties can be explained as a consequence of simple associative learning. To this end, we employ a parsimonious firing-rate attractor network equipped with the Hebbian-like Bayesian Confidence Propagating Neural Network (BCPNN) learning rule relying on synaptic traces with asymmetric temporal characteristics. The proposed network model is able to encode and reproduce temporal aspects of the input, and offers internal control of the recall dynamics by gain modulation. We provide an analytical characterisation of the relationship between the structure of the weight matrix, the dynamical network parameters and the temporal aspects of sequence recall. We also present a computational study of the performance of the system under the effects of noise for an extensive region of the parameter space. Finally, we show how the inclusion of modularity in our network structure facilitates the learning and recall of multiple overlapping sequences even in a noisy regime.

Social noise interferes with learning in a volatile environment

To learn through feedback, feedback should be reliable. However, if feedback is blurred by irrelevant social information, learning in a volatile environment, which requires fast learning and adaptation, might be disturbed. In this study, we investigated how feedback with social noise interferes with learning in a volatile environment by designing a probabilistic associative learning task in which the association probability changes dynamically, and the outcome was randomly blurred by an emotional face with incongruent valence. Learning in this situation was modelled by HGF-S such that emotionally incongruent feedback induces perceptual uncertainty called social noise. The Bayesian model comparison showed that the HGF-S model explains the subjects’ behaviour well, and the simulation showed that social noise interrupts both learning the association probability and the volatility. Furthermore, the learning interruption influenced the subsequent decision. Finally, we found that the individual difference in how the same emotionally incongruent feedback induces social noise in varying degrees was related to the differences in event-related desynchronization induced by happy and sad faces in the right anterior insula, which encodes the degree of emotional feeling. These results advance our understanding of how feedback with emotional interference affects learning.

Selective Estrogen Receptor Modulation Increases Hippocampal Activity during Probabilistic Association Learning in Schizophrenia.

People with schizophrenia show probabilistic association learning impairment in conjunction with abnormal neural activity. The selective estrogen receptor modulator (SERM) raloxifene preserves neural activity during memory in healthy older men and improves memory in schizophrenia. Here, we tested the extent to which raloxifene modifies neural activity during learning in schizophrenia. Nineteen people with schizophrenia participated in a twelve-week randomized, double-blind, placebo-controlled, cross-over adjunctive treatment trial of the SERM raloxifene administered orally at 120 mg daily to assess brain activity during probabilistic association learning using functional magnetic resonance imaging (fMRI). Raloxifene improved probabilistic association learning and significantly increased fMRI BOLD activity in the hippocampus and parahippocampal gyrus relative to placebo. A separate region of interest confirmatory analysis in 21 patients vs 36 healthy controls showed a positive association between parahippocampal neural activity and learning in patients, but no such relationship in the parahippocampal gyrus of healthy controls. Thus, selective estrogen receptor modulation by raloxifene concurrently increases activity in the parahippocampal gyrus and improves probabilistic association learning in schizophrenia. These results support a role for estrogen receptor modulation of mesial temporal lobe neural activity in the remediation of learning disabilities in both men and women with schizophrenia.

Error-monitoring During Complex Probabilistic Association Learning in Adults with ADHD

Error-monitoring During Complex Probabilistic Association Learning in Adults with ADHD

Behavioural and neural deficits in adults with Attention Deficit/Hyperactivity Disorder (ADHD) throughout complex probabilistic association learning

Behavioural and neural deficits in adults with Attention Deficit/Hyperactivity Disorder (ADHD) throughout complex probabilistic association learning

Peripheral BDNF: a candidate biomarker of healthy neural activity during learning is disrupted in schizophrenia.

Brain-derived neurotrophic factor (BDNF) is an important regulator of synaptogenesis and synaptic plasticity underlying learning. However, a relationship between circulating BDNF levels and brain activity during learning has not been demonstrated in humans. Reduced brain BDNF levels are found in schizophrenia and functional neuroimaging studies of probabilistic association learning in schizophrenia have demonstrated reduced activity in a neural network that includes the prefrontal and parietal cortices and the caudate nucleus. We predicted that brain activity would correlate positively with peripheral BDNF levels during probabilistic association learning in healthy adults and that this relationship would be altered in schizophrenia. Twenty-five healthy adults and 17 people with schizophrenia or schizo-affective disorder performed a probabilistic association learning test during functional magnetic resonance imaging (fMRI). Plasma BDNF levels were measured by enzyme-linked immunosorbent assay (ELISA). We found a positive correlation between circulating plasma BDNF levels and brain activity in the parietal cortex in healthy adults. There was no relationship between plasma BDNF levels and task-related activity in the prefrontal, parietal or caudate regions in schizophrenia. A direct comparison of these relationships between groups revealed a significant diagnostic difference. This is the first study to show a relationship between peripheral BDNF levels and cortical activity during learning, suggesting that plasma BDNF levels may reflect learning-related brain activity in healthy humans. The lack of relationship between plasma BDNF and task-related brain activity in patients suggests that circulating blood BDNF may not be indicative of learning-dependent brain activity in schizophrenia.

BDNF val66met genotype and schizotypal personality traits interact to influence probabilistic association learning

The brain derived neurotrophic factor (BDNF) val66met polymorphism rs6265 influences learning and may represent a risk factor for schizophrenia. Healthy people with high schizotypal personality traits display cognitive deficits that are similar to but not as severe as those observed in schizophrenia and they can be studied without confounds of antipsychotics or chronic illness. How genetic variation in BDNF may impact learning in individuals falling along the schizophrenia spectrum is unknown. We predicted that schizotypal personality traits would influence learning and that schizotypal personality-based differences in learning would vary depending on the BDNF val66met genotype. Eighty-nine healthy adults completed the Schizotypal Personality Questionnaire (SPQ) and a probabilistic association learning test. Blood samples were genotyped for the BDNF val66met polymorphism. An ANOVA was performed with BDNF genotype (val homozygotes and met-carriers) and SPQ score (high/low) as grouping variables and probabilistic association learning as the dependent variable. Participants with low SPQ scores (fewer schizotypal personality traits) showed significantly better learning than those with high SPQ scores. BDNF met-carriers displaying few schizotypal personality traits performed best, whereas BDNF met-carriers displaying high schizotypal personality traits performed worst. Thus, the BDNF val66met polymorphism appears to influence probabilistic association learning differently depending on the extent of schizotypal personality traits displayed.

Poster #S33 REDUCED DEACTIVATION IN MEDIAL PREFRONTAL CORTEX DURING AN INNER SPEECH TASK IN SCHIZOPHRENIA PATIENTS WITH AUDITORY VERBAL HALLUCINATIONS

prefrontal and parietal cortex, and the caudate nucleus in schizophrenia. We predicted that brain activity would positively correlate with peripheral BDNF levels during probabilistic association learning in healthy adults and that this relationship would be altered in schizophrenia. Methods: Twenty-five healthy adults and 17 people with schizophrenia or schizoaffective disorder performed a probabilistic association learning test during functional magnetic resonance imaging (fMRI). Plasma BDNF levels were measured by ELISA. Results: We found a positive correlation between circulating plasma BDNF levels and brain activity in the right parietal cortex in healthy adults. There was no relationship between plasma BDNF levels and task-related activity in the prefrontal, parietal or caudate regions in schizophrenia. A direct comparison of these relationships between groups revealed a significant difference. Discussion: This is the first study to show a relationship between peripheral BDNF levels and cortical activity during learning suggesting that plasma BDNF levels may reflect brain activity in healthy humans. The lack of relationship between plasma BDNF and task related brain activity in patients demonstrates that circulating blood BDNF is not related to frontal-parietal-striatal activity during learning in schizophrenia.

Dopaminergic therapy removal differentially effects learning in schizophrenia and Parkinson’s disease

Studies of patients with Parkinson’s disease receiving dopamimetics report conflicting evidence for early learning of probabilistic cue–outcome associations that elicits frontal–striatal activity. Previous studies of probabilistic association learning in patients with schizophrenia administered antipsychotics have displayed conflicting evidence for normal and abnormal learning. The role of dopaminergic treatment (dopamimetic versus dopamine antagonistic) effects on probabilistic association learning in these diseases that directly impact the dopamine system is not fully understood. The current study examined the effects of dopaminergic therapies on probabilistic association learning in 13 patients with schizophrenia and 8 patients with Parkinson’s disease under two conditions: after withdrawal from dopaminergic treatment and following administration of appropriate dopaminergic treatment. Medication order was counterbalanced in both groups. Patients with Parkinson’s disease failed to demonstrate any significant improvement over 150 trials, under both conditions (receiving or withdrawn from dopamimetics). Patients with schizophrenia withdrawn from antipsychotics displayed significant improvement during later trials only. These results demonstrate an effect of dopamine (DA) signaling on probabilistic association learning in that: (1) dopamine replacement therapy in Parkinson’s disease is insufficient to significantly improve probabilistic association learning and (2) DA receptor blockade impairs and removal of DA receptor blockade significantly improves frontal–striatal-dependent probabilistic association learning in schizophrenia, which is a novel finding and is opposite to the effects shown following removal of DA receptor blockade on other cognitive domains reported previously.

Impaired acquisition rates of probabilistic associative learning in frontotemporal dementia is associated with fronto-striatal atrophy.

Frontotemporal dementia (FTD) is classically considered to be a neurodegenerative disease with cortical changes. Recent structural imaging findings, however, highlight that subcortical and in particular striatal regions are also affected in the FTD syndrome. The influence of striatal pathology on cognitive and behavioural changes in FTD is virtually unexplored. In the current study we employ the Weather Prediction Task (WPT), a probabilistic learning task which taps into striatal dysfunction, in a group of FTD patients. We also regressed the patients' behavioural performance with their grey matter atrophy via voxel-based morphometry (VBM) to identify the grey matter contributions to WPT performance in FTD. Based on previous studies we expected to see striatal and frontal atrophy to be involved in impaired probabilistic learning. Our behavioural results show that patients performed on a similar level to controls overall, however, there was a large variability of patient performance in the first 30 trials of the task, which are critical in the acquisition of the probabilistic learning rules. A VBM analysis covarying the performance for the first 30 trials across participants showed that atrophy in striatal but also frontal brain regions correlated with WPT performance in these trials. Closer inspection of performance across the first 30 trials revealed a subgroup of FTD patients that performed significantly poorly than the remaining patients and controls on the WPT, despite achieving the same level of probabilistic learning as the other groups in later trials. Additional VBM analyses revealed that the subgroup of FTD patients with poor early probabilistic learning in the first 30 trials showed greater striatal atrophy compared to the remaining FTD patients and controls. These findings suggest that the integrity of fronto-striatal regions is important for probabilistic learning in FTD, with striatal integrity in particular, determining the acquisition learning rate. These findings will therefore have implications for developing an easily administered version of the probabilistic learning task which can be used by clinicians to assess striatal functioning in neurodegenerative syndromes.

Functional specialization within the striatum along both the dorsal/ventral and anterior/posterior axes during associative learning via reward and punishment

The goal of the present study was to elucidate the role of the human striatum in learning via reward and punishment during an associative learning task. Previous studies have identified the striatum as a critical component in the neural circuitry of reward-related learning. It remains unclear, however, under what task conditions, and to what extent, the striatum is modulated by punishment during an instrumental learning task. Using high-resolution functional magnetic resonance imaging (fMRI) during a reward- and punishment-based probabilistic associative learning task, we observed activity in the ventral putamen for stimuli learned via reward regardless of whether participants were correct or incorrect (i.e., outcome). In contrast, activity in the dorsal caudate was modulated by trials that received feedback--either correct reward or incorrect punishment trials. We also identified an anterior/posterior dissociation reflecting reward and punishment prediction error estimates. Additionally, differences in patterns of activity that correlated with the amount of training were identified along the anterior/posterior axis of the striatum. We suggest that unique subregions of the striatum--separated along both a dorsal/ventral and anterior/posterior axis--differentially participate in the learning of associations through reward and punishment.

Probabilistic association learning in frontotemporal dementia and schizophrenia

IntroductionRecent neuropsychological studies show substantial cognitive deficits in patients with frontotemporal dementia (FTD). Schizophrenia (SC) overlaps in terms of neurobehavioural symptoms with FTD. Probabilistic association learning, which is thought to assay fronto-striatal function, is well documented to elicit impairment in SC and has not been investigated in FTD to date; this study compared FTD, SC and a healthy comparison group on probabilistic association learning to determine the extent to which FTD patients were similar in performance to SC patients. MethodsTwenty FTD patients, 24 SC patients and 26 healthy controls were assessed using the probabilistic association learning weather prediction test. FTD patients were also divided into behavioural and language variants for comparison to the healthy group. ResultsFTD patients were impaired during probabilistic association learning in comparison to healthy controls. There was no difference in performance between the FTD and SC groups. FTD behavioural variants performed significantly worse than the healthy comparison group, while FTD language variants did not differ from the healthy comparison group. ConclusionsThis study provides the first evidence for impaired probabilistic association learning in FTD which is of an equivalent degree to that seen in SC. These results support recent structural neuroimaging studies showing fronto-striatal abnormalities in FTD and suggest that fronto-striatal dysfunction may contribute to cognitive deficits in a significant proportion of people with FTD.

Transcranial direct current stimulation influences probabilistic association learning in schizophrenia

Schizophrenia is associated with heterogeneity in symptoms, cognition and treatment response. Probabilistic association learning, involving a gradual learning of cue–outcome associations, activates a frontal-striatal network in healthy adults. Studies of probabilistic association learning in schizophrenia have shown frontal-striatal dysfunction although considerable heterogeneity in performance has also been reported. Anodal transcranial direct current stimulation (tDCS) to the dorsolateral prefrontal cortex has been shown to improve probabilistic association learning in healthy adults. The aim of the current study was to determine the extent to which anodal tDCS to the left dorsolateral prefrontal cortex would reverse probabilistic association learning deficits in schizophrenia. Prior to tDCS, 20 people with schizophrenia performed an initial baseline assessment without stimulation. Anodal tDCS was administered continuously for 20 min at an intensity of 2.0 mA to the left dorsolateral prefrontal cortex in a single-blind, counterbalanced, sham-controlled, cross-over design while participants performed 150 trials of a probabilistic association learning test. Although anodal tDCS failed to improve probabilistic association learning based on the whole sample performance, greater variance in the active relative to the sham conditions suggested a subset of people may respond to treatment. Further correlation, regression and cluster analyses revealed differential effects of baseline performance on active tDCS and sham treatment and that there was a subset of people with schizophrenia who displayed improvement with tDCS suggesting that anodal tDCS to the dorsolateral prefrontal cortex may facilitate access to existing prefrontal cortex neural reserves in people with schizophrenia who show adequate capacity to learn at baseline.

Frontal and Parietal Contributions to Probabilistic Association Learning

Neuroimaging studies have shown both dorsolateral prefrontal (DLPFC) and inferior parietal cortex (iPARC) activation during probabilistic association learning. Whether these cortical brain regions are necessary for probabilistic association learning is presently unknown. Participants' ability to acquire probabilistic associations was assessed during disruptive 1 Hz repetitive transcranial magnetic stimulation (rTMS) of the left DLPFC, left iPARC, and sham using a crossover single-blind design. On subsequent sessions, performance improved relative to baseline except during DLPFC rTMS that disrupted the early acquisition beneficial effect of prior exposure. A second experiment examining rTMS effects on task-naive participants showed that neither DLPFC rTMS nor sham influenced naive acquisition of probabilistic associations. A third experiment examining consecutive administration of the probabilistic association learning test revealed early trial interference from previous exposure to different probability schedules. These experiments, showing disrupted acquisition of probabilistic associations by rTMS only during subsequent sessions with an intervening night's sleep, suggest that the DLPFC may facilitate early access to learned strategies or prior task-related memories via consolidation. Although neuroimaging studies implicate DLPFC and iPARC in probabilistic association learning, the present findings suggest that early acquisition of the probabilistic cue-outcome associations in task-naive participants is not dependent on either region.