Probabilistic Learning Task Research Articles

Acute stress impairs reward positivity effect in probabilistic learning.

Decision making based on feedback learning requires a series of cognitive processes, including estimating the probability of particular outcomes and modulating expectations between expected versus actual outcomes. It has been suggested that stress affects decision making and subsequent processing of feedback valence and magnitude. However, less is known about the effect of acute stress on reward expectancy. In the current study, participants performed a probabilistic learning task, in which they learned an association between response and feedback within different reward expectancy trials (30% and 70%) under the conditions of stress (threat of shock) and safety (no shock). We recorded event-related potentials (ERPs) to measure the reward positivity (RewP) which reflects reward prediction error signals during feedback processing. Behavioral data indicated that participants performed better in the 70% reward trials than in the 30% reward trials. However, no significant difference was observed between stress and safe conditions. Importantly, ERP results indicated that unexpected feedback elicited larger RewP than did expected feedback and this expectancy effect of RewP was reduced in the stress relative to safe condition. Moreover, the correlations between RewP and choice accuracy (70% - 30% reward condition) in the safe and stress conditions were in a similar pattern; yet, only in the stress condition the correlation reached significantly. This may indicate that blunted RewP was associated with impaired performance at an individual level. Our study provides ERP evidence that acute stress affects brain responses to reward prediction error processing, which may explain various abnormal learning behaviors associated with stress-related disorders.

Separating Probability and Reversal Learning in a Novel Probabilistic Reversal Learning Task for Mice

The exploration/exploitation tradeoff – pursuing a known reward vs. sampling from lesser known options in the hope of finding a better payoff – is a fundamental aspect of learning and decision making. In humans, this has been studied using multi-armed bandit tasks. The same processes have also been studied using simplified probabilistic reversal learning (PRL) tasks with binary choices. Our investigations suggest that protocols previously used to explore PRL in mice may prove beyond their cognitive capacities, with animals performing at a no-better-than-chance level. We sought a novel probabilistic learning task to improve behavioral responding in mice, whilst allowing the investigation of the exploration/exploitation tradeoff in decision making. To achieve this, we developed a two-lever operant chamber task with levers corresponding to different probabilities (high/low) of receiving a saccharin reward, reversing the reward contingencies associated with levers once animals reached a threshold of 80% responding at the high rewarding lever. We found that, unlike in existing PRL tasks, mice are able to learn and behave near optimally with 80% high/20% low reward probabilities. Altering the reward contingencies towards equality showed that some mice displayed preference for the high rewarding lever with probabilities as close as 60% high/40% low. Additionally, we show that animal choice behavior can be effectively modelled using reinforcement learning (RL) models incorporating learning rates for positive and negative prediction error, a perseveration parameter, and a noise parameter. This new decision task, coupled with RL analyses, advances access to investigate the neuroscience of the exploration/exploitation tradeoff in decision making.

Feedback timing modulates interactions between feedback processing and memory encoding: Evidence from event-related potentials.

Feedback-based learning relies on a procedural learning system driven by reward prediction errors (RPEs). The processing of temporally delayed feedback is supported by brain structures associated with declarative memory processes, but it is still unknown how delayed feedback processing and memory encoding interact. In this study, a subsequent memory paradigm was employed to investigate how the incidental encoding of feedback pictures presented with a short (SD, 500 ms) or long (LD, 6500 ms) delay in a probabilistic learning task affects the event-related potential (ERP) correlate of RPEs (i.e., the feedback-related negativity; FRN). In an ensuing test phase, a surprise recognition memory test for the feedback pictures was conducted. FRN amplitudes measured in the feedback-locked ERPs recorded during the learning phase (FRNpeak) and in the negative minus positive feedback difference wave (FRNdiff) were compared for subsequently remembered and forgotten feedback pictures. Feedback processing as reflected in the FRNpeak was diminished for remembered LD feedback pictures, indicating that delayed feedback processing and memory encoding competed for similar neural processing resources. As evidenced by large FRNdiff amplitudes in the SD condition, the evaluation of shortly delayed feedback strongly relied on the procedural learning system. A complementary model-based single trial analysis was conducted to validate models of the functional significance of the FRN. Consistent with previous studies, feedback-locked N170 and P300 amplitudes were sensitive to feedback delay. In the test phase, memory for LD feedback pictures was better than for SD pictures and accompanied by a late old-new effect, presumably reflecting extended recollective processing.

Comparison of conventional and rapid-acting antidepressants in a rodent probabilistic reversal learning task.

Deficits in reward processing are a central feature of major depressive disorder with patients exhibiting decreased reward learning and altered feedback sensitivity in probabilistic reversal learning tasks. Methods to quantify probabilistic learning in both rodents and humans have been developed, providing translational paradigms for depression research. We have utilised a probabilistic reversal learning task to investigate potential differences between conventional and rapid-acting antidepressants on reward learning and feedback sensitivity. We trained 12 rats in a touchscreen probabilistic reversal learning task before investigating the effect of acute administration of citalopram, venlafaxine, reboxetine, ketamine or scopolamine. Data were also analysed using a Q-learning reinforcement learning model to understand the effects of antidepressant treatment on underlying reward processing parameters. Citalopram administration decreased trials taken to learn the first rule and increased win-stay probability. Reboxetine decreased win-stay behaviour while also decreasing the number of rule changes animals performed in a session. Venlafaxine had no effect. Ketamine and scopolamine both decreased win-stay probability, number of rule changes performed and motivation in the task. Insights from the reinforcement learning model suggested that reboxetine led animals to choose a less optimal strategy, while ketamine decreased the model-free learning rate. These results suggest that reward learning and feedback sensitivity are not differentially modulated by conventional and rapid-acting antidepressant treatment in the probabilistic reversal learning task.

Grant Report on Social Reward Learning in Schizophrenia †.

We report on the ongoing R21 project “Social Reward Learning in Schizophrenia”. Impairments in social cognition are a hallmark of schizophrenia. However, little work has been done on social reward learning deficits in schizophrenia. The overall goal of the project is to assess social reward learning in schizophrenia. A probabilistic reward learning (PRL) task is being used in the MRI scanner to evaluate reward learning to negative and positive social feedback. Monetary reward learning is used as a comparison to assess specificity. Behavioral outcomes and brain areas, included those involved in reward, are assessed in patients with schizophrenia or schizoaffective disorder and controls. It is also critical to determine whether decreased expected value (EV) of social stimuli and/or reward prediction error (RPE) learning underlie social reward learning deficits to inform potential treatment pathways. Our central hypothesis is that the pattern of social learning deficits is an extension of a more general reward learning impairment in schizophrenia and that social reward learning deficits critically contribute to deficits in social motivation and pleasure. We hypothesize that people with schizophrenia will show impaired behavioral social reward learning compared to controls, as well as decreased ventromedial prefrontal cortex (vmPFC) EV signaling at time of choice and decreased striatal RPE signaling at time of outcome, with potentially greater impairment to positive than negative feedback. The grant is in its second year. It is hoped that this innovative approach may lead to novel and more targeted treatment approaches for social cognitive impairments, using cognitive remediation and/or brain stimulation.

Intact Reinforcement Learning But Impaired Attentional Control During Multidimensional Probabilistic Learning in Older Adults.

To efficiently learn optimal behavior in complex environments, humans rely on an interplay of learning and attention. Healthy aging has been shown to independently affect both of these functions. Here, we investigate how reinforcement learning and selective attention interact during learning from trial and error across age groups. We acquired behavioral and fMRI data from older and younger adults (male and female) performing two probabilistic learning tasks with varying attention demands. Although learning in the unidimensional task did not differ across age groups, older adults performed worse than younger adults in the multidimensional task, which required high levels of selective attention. Computational modeling showed that choices of older adults are better predicted by reinforcement learning than Bayesian inference, and that older adults rely more on reinforcement learning-based predictions than younger adults. Conversely, a higher proportion of younger adults' choices was predicted by a computationally demanding Bayesian approach. In line with the behavioral findings, we observed no group differences in reinforcement-learning related fMRI activation. Specifically, prediction-error activation in the nucleus accumbens was similar across age groups, and numerically higher in older adults. However, activation in the default mode was less suppressed in older adults in for higher attentional task demands, and the level of suppression correlated with behavioral performance. Our results indicate that healthy aging does not significantly impair simple reinforcement learning. However, in complex environments, older adults rely more heavily on suboptimal reinforcement-learning strategies supported by the ventral striatum, whereas younger adults use attention processes supported by cortical networks.SIGNIFICANCE STATEMENT Changes in the way that healthy human aging affects how we learn to optimally behave are not well understood; it has been suggested that age-related declines in dopaminergic function may impair older adult's ability to learn from reinforcement. In the present fMRI experiment, we show that learning and nucleus accumbens activation in a simple unidimensional reinforcement-learning task was not significantly affected by age. However, in a more complex multidimensional task, older adults showed worse performance and relied more on reinforcement-learning strategies than younger adults, while failing to disengage their default-mode network during learning. These results imply that older adults are only impaired in reinforcement learning if they additionally need to learn which dimensions of the environment are currently important.

Positive social feedback alters emotional ratings and reward valuation of neutral faces.

Evaluation of facial and vocal emotional cues is vital in social interactions but can be highly influenced by characteristics of the observer, such as sex, age, and symptoms of affective disorders. Our evaluations of others' emotional expressions are likely to change as we get to know them and anticipate how they are likely to behave. However, the role of associative learning in the evaluation of social cues remains poorly understood. In this study, we investigated whether emotional ratings (valence and arousal) and reward valuation ("liking" and "wanting" measures) of neutral facial expressions can be altered through associative learning. We also examined whether emotional ratings and reward valuation varied with symptoms of anxiety and depression, disorders known to impair socio-affective functioning. Participants (N = 324) were young adults, ranging in scores across dimensions of depression and anxiety symptoms: "general distress" (common to depression and anxiety), "anhedonia-apprehension" (more specific to depression), and "fears" (more specific to anxiety). They rated neutral faces and completed a probabilistic learning task that paired images of neutral faces with positive or negative social feedback. Results demonstrated that pairing neutral faces with positive social feedback increased ratings of arousal, valence, and reward valuation (both "liking" and "wanting"). Pairing neutral faces with negative feedback reduced valence ratings and reduced "wanting," but did not impact arousal ratings or "liking." Symptoms of general distress were associated with negative bias in valence ratings, symptoms of anhedonia-apprehension were associated with reduced "wanting," and symptoms of fears were associated with altered accuracy over trials. Notably, the association between general distress and negative bias was reduced following the associative learning task. This suggests that disrupted evaluation of social cues can be improved through brief training.

Reinforcement learning in a probabilistic learning task without time constraints

Reinforcement learning in a probabilistic learning task without time constraints

Familiality of behavioral flexibility and response inhibition deficits in autism spectrum disorder (ASD)

BackgroundDiminished cognitive control, including reduced behavioral flexibility and behavioral response inhibition, has been repeatedly documented in autism spectrum disorder (ASD). We evaluated behavioral flexibility and response inhibition in probands and their parents using a family trio design to determine the extent to which these cognitive control impairments represent familial traits associated with ASD.MethodsWe examined 66 individuals with ASD (probands), 135 unaffected biological parents, and 76 typically developing controls. Participants completed a probabilistic reversal learning task (PRL) and a stop-signal task (SST) to assess behavioral flexibility and response inhibition respectively. Rates of PRL and SST errors were examined across groups, within families, and in relation to clinical and subclinical traits of ASD. Based on prior findings that subclinical broader autism phenotypic (BAP) traits may co-segregate within families and reflect heritable risk factors, we also examined whether cognitive control deficits were more prominent in families in which parents showed BAP features (BAP+).ResultsProbands and parents each showed increased rates of PRL and SST errors relative to controls. Error rates across tasks were not related. SST error rates inter-correlated among probands and their parents. PRL errors were more severe in BAP+ parents and their children relative to BAP− parents and their children. For probands of BAP+ parents, PRL and SST error rates were associated with more severe social-communication abnormalities and repetitive behaviors, respectively.ConclusionReduced behavioral flexibility and response inhibition are present among probands and their unaffected parents, but represent unique familial deficits associated with ASD that track with separate clinical issues. Specifically, behavioral response inhibition impairments are familial in ASD and manifest independently from parental subclinical features. In contrast, behavioral flexibility deficits are selectively present in families with BAP characteristics, suggesting they co-segregate in families with parental subclinical social, communication, and rigid personality traits. Together, these findings provide evidence that behavioral flexibility and response inhibition impairments track differentially with ASD risk mechanisms and related behavioral traits.

Aberrant probabilistic reinforcement learning in first-degree relatives of individuals with bipolar disorder

BackgroundMotivational dysregulation represents a core vulnerability factor for bipolar disorder. Whether this also comprises aberrant learning of stimulus-reinforcer contingencies is less clear. MethodsTo answer this question, we compared healthy first-degree relatives of individuals with bipolar disorder (n = 42) known to convey an increased risk of developing a bipolar spectrum disorder and healthy individuals (n = 97). Further, we investigated the effects of the behavioral activation system (BAS) on reinforcement learning across the entire sample. All participants were assessed with a probabilistic learning task that distinguishes learning from positive and negative feedback. Main outcome measures included choice frequencies and learning rate parameters generated by computational reinforcement learning algorithms. ResultsFirst-degree relatives choose more rewarding stimuli more consistently and showed marginally reduced learning rates from unexpected negative feedback. Further, first-degree relatives had lower BAS scores than controls, which were negatively associated with learning rates from unexpected negative feedback. LimitationsHowever as probands also reported other mental disorders such as Attention-Deficit/Hyperactivity Disorder and substance abuse among their first-degree relatives, we cannot know, whether these findings are specific to the risk for bipolar disorder. ConclusionThe behavior of first-degree relatives of individuals with bipolar disorder, who also display increased BAS sensitivity, is less influenced by unexpected negative feedback. This reduced learning from unexpected negative feedback biases subsequent choices towards stimuli with higher probabilities for a reward. In sum, our results confirm the role of aberrant reinforcement learning in the pathophysiology of bipolar disorder.

Amphetamine disrupts haemodynamic correlates of prediction errors in nucleus accumbens and orbitofrontal cortex

In an uncertain world, the ability to predict and update the relationships between environmental cues and outcomes is a fundamental element of adaptive behaviour. This type of learning is typically thought to depend on prediction error, the difference between expected and experienced events and in the reward domain that has been closely linked to mesolimbic dopamine. There is also increasing behavioural and neuroimaging evidence that disruption to this process may be a cross-diagnostic feature of several neuropsychiatric and neurological disorders in which dopamine is dysregulated. However, the precise relationship between haemodynamic measures, dopamine and reward-guided learning remains unclear. To help address this issue, we used a translational technique, oxygen amperometry, to record haemodynamic signals in the nucleus accumbens (NAc) and orbitofrontal cortex (OFC), while freely moving rats performed a probabilistic Pavlovian learning task. Using a model-based analysis approach to account for individual variations in learning, we found that the oxygen signal in the NAc correlated with a reward prediction error, whereas in the OFC it correlated with an unsigned prediction error or salience signal. Furthermore, an acute dose of amphetamine, creating a hyperdopaminergic state, disrupted rats’ ability to discriminate between cues associated with either a high or a low probability of reward and concomitantly corrupted prediction error signalling. These results demonstrate parallel but distinct prediction error signals in NAc and OFC during learning, both of which are affected by psychostimulant administration. Furthermore, they establish the viability of tracking and manipulating haemodynamic signatures of reward-guided learning observed in human fMRI studies by using a proxy signal for BOLD in a freely behaving rodent.

Modeling the Effect of Speed Emphasis in Probabilistic Category Learning

For many decisions in life, there is uncertainty about the outcome even if the stimulus is a familiar one. Under these circumstances, people often rely upon previous experience to make a choice. In general, more accurate responding is achieved when one considers decision-relevant information for longer. There are practical limits, however, to how long one can deliberate, and so decision-makers must balance decision speed against accuracy. Traditionally, the overall speed with which people respond has been understood in terms of changes in the evidence threshold used for decision-making. However, more recent findings suggest that speed pressure may also affect the quality of information people recruit for making decisions. The current study investigated the effects of emphasizing either response speed or response accuracy on decision-making in a probabilistic category learning task. Strikingly, conclusions from two model-based analyses of performance depended on whether data were analyzed at the level of individuals or the group average. For the group-level analyses, speed pressure selectively influenced evidence threshold. For the individual-level analyses, speed pressure affected both evidence threshold as well as the quality of information driving decision-making for four out of six participants. Our findings are consistent with recent results that identify common brain areas associated with setting evidence thresholds decision-making and representing category-level information in trial-by-trial associative learning.

A role for 5-HT4 receptors in human learning and memory.

5-HT4 receptor stimulation has pro-cognitive and antidepressant-like effects in animal experimental studies; however, this pharmacological approach has not yet been tested in humans. Here we used the 5-HT4 receptor partial agonist prucalopride to assess the translatability of these effects and characterise, for the first time, the consequences of 5-HT4 receptor activation on human cognition and emotion. Forty one healthy volunteers were randomised, double-blind, to a single dose of prucalopride (1 mg) or placebo in a parallel group design. They completed a battery of cognitive tests measuring learning and memory, emotional processing and reward sensitivity. Prucalopride increased recall of words in a verbal learning task, increased the accuracy of recall and recognition of words in an incidental emotional memory task and increased the probability of choosing a symbol associated with a high likelihood of reward or absence of loss in a probabilistic instrumental learning task. Thus acute prucalopride produced pro-cognitive effects in healthy volunteers across three separate tasks. These findings are a translation of the memory enhancing effects of 5-HT4 receptor agonism seen in animal studies, and lend weight to the idea that the 5-HT4 receptor could be an innovative target for the treatment of cognitive deficits associated with depression and other neuropsychiatric disorders. Contrary to the effects reported in animal models, prucalopride did not reveal an antidepressant profile in human measures of emotional processing.

Reward and punishment learning in schizophrenia and bipolar disorder

Prior studies on reward learning deficits in psychiatric disorders have used probabilistic learning tasks, making it unclear whether impairment is due to the probabilistic nature of the task rather than reward processing. In this study, we tested probabilistic vs. deterministic reward and punishment learning in healthy controls and three patient groups: schizophrenia (SZ), psychotic bipolar disorder (BD), and nonpsychotic BD. Experimental results show that reward learning was impaired in patients with SZ and patients with psychotic BD in the probabilistic learning task compared to patients with nonpsychotic BD and healthy controls. In contrast, punishment learning in the probabilistic task was impaired in patients with nonpsychotic BD compared to the other patient groups and healthy controls. There were no significant differences among all groups in the deterministic learning task scores. We also found that Hamilton Depression Scale scores negatively correlated with probabilistic learning performance. Our data may suggest that reward learning impairment may be due to the nature of the task as well as subtype of BD.

Neural mechanisms of reinforcement learning under mortality threat

ABSTRACTReinforcement learning – to adjust behaviors in response to feedback regarding reward and punishment – is pivotal to our survival. The present work investigated whether and how reinforcement learning is affected by thoughts of mortality that endanger one’s survival. We recorded electroencephalographic while adults performed a probabilistic learning task that required a forced-choice between two visual patterns for monetary reward for different beneficiaries (i.e., self, stranger, or no one) followed by reward or no-reward feedback. We found that verbal reminders of mortality (vs. negative emotion) enlarged an early positive component (P1) at the occipital electrodes but decreased a late positive potential (LPP) at the frontocentral electrodes in response to learning stimuli. While no-reward feedback relative to reward feedback stimuli elicited a feedback-related negativity (FRN) and increased non-phase locked theta band (4–8 Hz) activity at the frontocentral electrodes during reward learning for all beneficiaries, verbal reminders of mortality (vs. negative emotion) significantly reduced the FRN amplitude but failed to modulate the theta band activity. These results suggest that mortality salience enhances early attentional processing but dampens late cognitive evaluation of the learning stimuli during reinforcement learning. Moreover, mortality salience decreases the neural sensitivity to feedback signaling the absence of monetary reward.

Btbd3 expression regulates compulsive-like and exploratory behaviors in mice

BTB/POZ domain-containing 3 (BTBD3) was identified as a potential risk gene in the first genome-wide association study of obsessive-compulsive disorder (OCD). BTBD3 is a putative transcription factor implicated in dendritic pruning in developing primary sensory cortices. We assessed whether BTBD3 also regulates neural circuit formation within limbic cortico-striato-thalamo-cortical circuits and behaviors related to OCD in mice. Behavioral phenotypes associated with OCD that are measurable in animals include compulsive-like behaviors and reduced exploration. We tested Btbd3 wild-type, heterozygous, and knockout mice for compulsive-like behaviors including cage-mate barbering, excessive wheel-running, repetitive locomotor patterns, and reduced goal-directed behavior in the probabilistic learning task (PLT), and for exploratory behavior in the open field, digging, and marble-burying tests. Btbd3 heterozygous and knockout mice showed excessive barbering, wheel-running, impaired goal-directed behavior in the PLT, and reduced exploration. Further, chronic treatment with fluoxetine, but not desipramine, reduced barbering in Btbd3 wild-type and heterozygous, but not knockout mice. In contrast, Btbd3 expression did not alter anxiety-like, depression-like, or sensorimotor behaviors. We also quantified dendritic morphology within anterior cingulate cortex, mediodorsal thalamus, and hippocampus, regions of high Btbd3 expression. Surprisingly, Btbd3 knockout mice only showed modest increases in spine density in the anterior cingulate, while dendritic morphology was unaltered elsewhere. Finally, we virally knocked down Btbd3 expression in whole, or just dorsal, hippocampus during neonatal development and assessed behavior during adulthood. Whole, but not dorsal, hippocampal Btbd3 knockdown recapitulated Btbd3 knockout phenotypes. Our findings reveal that hippocampal Btbd3 expression selectively modulates compulsive-like and exploratory behavior.

Probability learning and feedback processing in dyslexia: A performance and heart rate analysis.

Recent studies suggest that individuals with dyslexia may be impaired in probability learning and performance monitoring. These observations are consistent with findings indicating atypical neural activations in frontostriatal circuits in the brain, which are important for associative learning. The current study further examined probability learning and performance monitoring in adult individuals with dyslexia (n=23) and typical readers (n=31) using two varieties of a typical probabilistic learning task. In addition to performance measures, we measured heart rate, focusing on cardiac slowing with negative feedback as a manifestation of the automatic performance monitoring system. One task required participants to learn associations between artificial script and speech sounds and the other task required them to learn associations between geometric forms and bird sounds. Corrective feedback (informative or random) was provided in both tasks. Performance results indicated that individuals with dyslexia and typical readers learned the associations equally well in contrast to expectations. We found the typical cardiac response associated with feedback processing consisting of a heart rate slowing with the presentation of the feedback and a return to baseline thereafter. Interestingly, the heart rate slowing associated with feedback was less pronounced and the return to baseline was delayed in individuals with dyslexia relative to typical readers. These findings were interpreted in relation to current theorizing of performance monitoring linking the salience network in the brain to autonomic functioning.

Heterogeneity in Disordered Gambling: Decision-Making and Impulsivity in Gamblers Grouped by Preferred Form.

Background: Previous research has indicated that disordered gamblers display deficits in impulsivity and risky decision-making, compared to healthy control groups. However, disordered gamblers are not a homogenous group, and differences in performance on neurocognitive tasks may be related to the form of gambling in which an individual chooses to engage. The present study used neurocognitive tasks and questionnaire measures to ascertain group differences in gamblers grouped by preferred form of gambling. Method: Treatment-seeking pathological gamblers from the National Problem Gambling Clinic, London (n = 101), completed a neurocognitive assessment comprising the Cambridge gamble task (CGT), the stop-signal task (SST), a probabilistic reversal learning task (PRL), and the Kirby Monetary Choice Questionnaire, as well as questionnaire measures of gambling severity, impulsivity, depression, and anxiety. Analyses compared gamblers who favored fixed-odds betting terminals (FOBTs) (the modal form) to gamblers who preferred other forms of gambling (non-FOBT). Results: The FOBT group showed impaired decision-making under risk on the CGT compared to the non-FOBT group, choosing the likely option less on more uncertain decisions. The FOBT group made fewer perseverative errors on the PRL task, had lower depression and anxiety scores, and were less likely to have a family history of problem gambling than the non-FOBT group. Discussion: Decision-making and cognitive flexibility differences between gamblers grouped by gambling type supports preferred form as an important source of heterogeneity in gambling disorder. Decision-making strategies and risk attitudes should be considered when approaching cognition-focused treatment strategies, allowing interventions to be targeted at specific cognitive deficits.

Is there more room to improve? The lifespan trajectory of procedural learning and its relationship to the between- and within-group differences in average response times.

Characterizing the developmental trajectories of cognitive functions such as learning, memory and decision making across the lifespan faces fundamental challenges. Cognitive functions typically encompass several processes that can be differentially affected by age. Methodological issues also arise when comparisons are made across age groups that differ in basic performance measures, such as in average response times (RTs). Here we focus on procedural learning–a fundamental cognitive function that underlies the acquisition of cognitive, social, and motor skills–and demonstrate how disentangling subprocesses of learning and controlling for differences in average RTs can reveal different developmental trajectories across the human lifespan. Two hundred-seventy participants aged between 7 and 85 years performed a probabilistic sequence learning task that enabled us to separately measure two processes of procedural learning, namely general skill learning and statistical learning. Using raw RT measures, in between-group comparisons, we found a U-shaped trajectory with children and older adults exhibiting greater general skill learning compared to adolescents and younger adults. However, when we controlled for differences in average RTs (either by using ratio scores or focusing on a subsample of participants with similar average speed), only children (but not older adults) demonstrated superior general skill learning consistently across analyses. Testing the relationship between average RTs and general skill learning within age groups shed light on further age-related differences, suggesting that general skill learning measures are more affected by average speed in some age groups. Consistent with previous studies of learning probabilistic regularities, statistical learning showed a gradual decline across the lifespan, and learning performance seemed to be independent of average speed, regardless of the age group. Overall, our results suggest that children are superior learners in various aspects of procedural learning, including both general skill and statistical learning. Our study also highlights the importance to test, and control for, the effect of average speed on other RT measures of cognitive functions, which can fundamentally affect the interpretation of group differences in developmental, aging and clinical psychology and neuroscience studies.

Effects of feedback delay and agency on feedback-locked beta and theta power during reinforcement learning.

Feedback-based learning initiated by dopamine (DA) cell firing is crucial for adaptive behavior. The nature and context of feedback can vary, however, affecting how feedback is processed. For example, the feedback-related negativity (FRN) in the ERP in humans, which has been linked to the DA system, is reduced for delayed feedback and for observational compared to active learning. Recent research suggested that oscillations in the theta and beta band over the medio-frontal cortex reflect distinct feedback processing mechanisms. In this study, we hypothesized that the power in both frequency bands is affected by feedback delay and agency. We thus investigated effects of feedback delay (1s vs. 7s) on induced theta and beta band power and the FRN in a probabilistic feedback learning task in two participant groups, one learning actively and one by observation. For theta and beta, a larger power difference between negative and positive feedback for immediate than delayed feedback was found, driven by positive feedback for beta and by negative feedback for theta, while no differential modulation by agency was seen for theta or beta power following positive and negative feedback. These results indicate that feedback-locked beta and theta both reflect neural processes that are specific for the integration of feedback and recently preceding events, possibly linked to cognitive control and memory. With respect to the FRN amplitude, we could replicate previous findings of both delay and agency modulations, suggesting that the neural processes underlying feedback-locked ERPs and theta and beta power modulations differ.