Reward Prediction Error Research Articles

Reward prediction error in learning-related behaviors.

Learning is a complex process, during which our opinions and decisions are easily changed due to unexpected information. But the neural mechanism underlying revision and correction during the learning process remains unclear. For decades, prediction error has been regarded as the core of changes to perception in learning, even driving the learning progress. In this article, we reviewed the concept of reward prediction error, and the encoding mechanism of dopaminergic neurons and the related neural circuities. We also discussed the relationship between reward prediction error and learning-related behaviors, including reversal learning. We then demonstrated the evidence of reward prediction error signals in several neurological diseases, including Parkinson's disease and addiction. These observations may help to better understand the regulatory mechanism of reward prediction error in learning-related behaviors.

Feeling and deciding: Subjective experiences rather than objective factors drive the decision to invest cognitive control

When presented with the choice to invest cognitive control in a task, several signals are monitored to reach a decision. Leading theoretical frameworks argued that the investment of cognitive control is determined by a cost-benefit computation. However, previous accounts remained silent on the potential role of subjective experience in this computation. We experience confidence when giving an answer, feel the excitement of an anticipated reward, and reflect on how much effort is required for successful task performance. Two questions are investigated in the present work: how objective task parameters give rise to subjective experience and whether these drive the decision to allocate cognitive control. To this end, we designed a task in which we manipulated three objective parameters in the same sequence of events (stimulus uncertainty, physical effort, and reward prediction error). We asked participants to report their subjective experiences associated with these manipulations: confidence, subjective physical effort, and reward satisfaction. At the end of each trial, participants indicated whether they wanted to repeat that trial on the next day. In response to the first question, we demonstrate that subjective ratings are reliable and selective. Subjective experiences closely mirrored their objective manipulations. In response to the second question, we demonstrate that subjective experiences provide a better fit for the decisions on future control investments. While objective task parameters are considered when deciding, they do not always produce the expected changes in subjective experience, and when dissociations occur, it is the subjective experience that better explains the decision to allocate cognitive control.

Non-invasive stimulation reveals ventromedial prefrontal cortex function in reward prediction and reward processing.

Studies suggest an involvement of the ventromedial prefrontal cortex (vmPFC) in reward prediction and processing, with reward-based learning relying on neural activity in response to unpredicted rewards or non-rewards (reward prediction error, RPE). Here, we investigated the causal role of the vmPFC in reward prediction, processing, and RPE signaling by transiently modulating vmPFC excitability using transcranial Direct Current Stimulation (tDCS). Participants received excitatory or inhibitory tDCS of the vmPFC before completing a gambling task, in which cues signaled varying reward probabilities and symbols provided feedback on monetary gain or loss. We collected self-reported and evaluative data on reward prediction and processing. In addition, cue-locked and feedback-locked neural activity via magnetoencephalography (MEG) and pupil diameter using eye-tracking were recorded. Regarding reward prediction (cue-locked analysis), vmPFC excitation (versus inhibition) resulted in increased prefrontal activation preceding loss predictions, increased pupil dilations, and tentatively more optimistic reward predictions. Regarding reward processing (feedback-locked analysis), vmPFC excitation (versus inhibition) resulted in increased pleasantness, increased vmPFC activation, especially for unpredicted gains (i.e., gain RPEs), decreased perseveration in choice behavior after negative feedback, and increased pupil dilations. Our results support the pivotal role of the vmPFC in reward prediction and processing. Furthermore, they suggest that transient vmPFC excitation via tDCS induces a positive bias into the reward system that leads to enhanced anticipation and appraisal of positive outcomes and improves reward-based learning, as indicated by greater behavioral flexibility after losses and unpredicted outcomes, which can be seen as an improved reaction to the received feedback.

Feasibility of dopamine as a vector-valued feedback signal in the basal ganglia

It is well established that midbrain dopaminergic neurons support reinforcement learning (RL) in the basal ganglia by transmitting a reward prediction error (RPE) to the striatum. In particular, different computational models and experiments have shown that a striatum-wide RPE signal can support RL over a small discrete set of actions (e.g., no/no-go, choose left/right). However, there is accumulating evidence that the basal ganglia functions not as a selector between predefined actions but rather as a dynamical system with graded, continuous outputs. To reconcile this view with RL, there is a need to explain how dopamine could support learning of continuous outputs, rather than discrete action values. Inspired by the recent observations that besides RPE, the firing rates of midbrain dopaminergic neurons correlate with motor and cognitive variables, we propose a model in which dopamine signal in the striatum carries a vector-valued error feedback signal (a loss gradient) instead of a homogeneous scalar error (a loss). We implement a local, "three-factor" corticostriatal plasticity rule involving the presynaptic firing rate, a postsynaptic factor, and the unique dopamine concentration perceived by each striatal neuron. With this learning rule, we show that such a vector-valued feedback signal results in an increased capacity to learn a multidimensional series of real-valued outputs. Crucially, we demonstrate that this plasticity rule does not require precise nigrostriatal synapses but remains compatible with experimental observations of random placement of varicosities and diffuse volume transmission of dopamine.

Neural Response to Expecting a Caloric Sweet Taste Stimulus Predicts Body Mass Index Longitudinally Among Young Adult Women With Anorexia Nervosa

BackgroundAnorexia nervosa (AN) is an often-chronic illness, and we lack biomarkers to predict long-term outcome. Recent neuroimaging studies using caloric taste stimuli suggest that paradigms that have tested conditioned neural responses to expectation or salient stimulus receipt may underpin behaviors. However, whether activation of those neural circuits can predict long-term outcome has not been studied. MethodsWe followed women treated for AN (n = 35, mean age [SD] = 23 [7] years) and tested whether functional imaging brain response during a taste conditioning paradigm could predict posttreatment body mass index (BMI). We anticipated greater neural activity relative to caloric stimulus expectation and that dopamine-related receipt conditions would predict lower posttreatment BMI, indicating fear-associated arousal. ResultsFollow-up occurred at mean (SD) = 1648 (1216) days after imaging. Stimulus expectation in orbitofrontal and striatal regions and BMI and BMI change at follow-up were negatively correlated, and these correlations remained significant for the right superior orbitofrontal cortex and BMI change after multiple comparison correction (r = −0.484, p = .003). This relationship remained significant after including time between brain scanning and follow-up in the model. Reward prediction error response did not predict long-term BMI. ConclusionsThe orbitofrontal cortex is involved in learning and conditioning, and these data implicate this region in learned caloric stimulus expectation and long-term prediction of weight outcomes in AN. Thus, conditioned elevated brain response to the anticipation of receiving a caloric stimulus may drive food avoidance, suggesting that breaking such associations is central for long-term recovery from AN.

Relationship between reward-related brain activity and opportunities to sit

The present study tested whether energy-minimizing behaviors evoke reward-related brain activity that promotes the repetition of these behaviors via reinforcement learning processes. Fifty-eight healthy young adults in a standing position performed a task where they could earn a reward either by sitting down or squatting while undergoing electroencephalographic (EEG) recording. Reward-prediction errors were quantified as the amplitude of the EEG-derived reward positivity. Results showed that reward positivity was larger on reward versus no reward trials, confirming the validity of our paradigm to measure evoked reward-related brain activity. However, results showed no evidence that sitting (versus standing and squatting) trials led to larger reward positivity. Moreover, we found no evidence suggesting that this effect was moderated by typical physical activity, physical activity on the day of the study, or energy expenditure during the experiment. However, at the behavioral level, results showed that the probability of choosing the stimulus more likely to lead to sitting than standing increased as the number of trials increased. In addition, results revealed that the probability of changing the selected stimulus was higher when the previous trial was a stand trial relative to a sit trial. In sum, neural results showed no evidence supporting the theory that opportunities to minimize energy expenditure are rewarding. However, behavioral findings suggested participants tend to choose the less effortful behavioral alternative and were therefore consistent with the theory of effort minimization (TEMPA).

The behavioral signature of stepwise learning strategy in male rats and its neural correlate in the basal forebrain

Studies of associative learning have commonly focused on how rewarding outcomes are predicted by either sensory stimuli or animals’ actions. However, in many learning scenarios, reward delivery requires the occurrence of both sensory stimuli and animals’ actions in a specific order, in the form of behavioral sequences. How such behavioral sequences are learned is much less understood. Here we provide behavioral and neurophysiological evidence to show that behavioral sequences are learned using a stepwise strategy. In male rats learning a new association, learning started from the behavioral event closest to the reward and sequentially incorporated earlier events. This led to the sequential refinement of reward-seeking behaviors, which was characterized by the stepwise elimination of ineffective and non-rewarded behavioral sequences. At the neuronal level, this stepwise learning process was mirrored by the sequential emergence of basal forebrain neuronal responses toward each event, which quantitatively conveyed a reward prediction error signal and promoted reward-seeking behaviors. Together, these behavioral and neural signatures revealed how behavioral sequences were learned in discrete steps and when each learning step took place.

Obsession, compulsion and learning in obsessive-compulsive disorder: A multilevel computational model

Obsessive-compulsive disorder (OCD) is characterized symptomatically by obsessive thoughts and compulsive behaviors to separate its behavioral features. Despite extensive research efforts into the pathogenesis and neural substrates in OCD, the interactions and causal associations between the two behaviors remain unclear. To theoretically fill this gap in mechanisms, we combine a cortico-basal ganglia-thalamic model of motor loops and a limbic system model based on classical reward prediction errors to model the complex dynamics between different loops in OCD. Based on the widely accepted hypothesis that dopamine bursts and reinforcement learning influence synaptic plasticity, we explore the mechanisms behind the progression of OCD related to stress triggers, overtraining, and striatal lesions, in terms of behavioral flexibility as a measurement. The obtained results indicate that reinforcement learning affects synaptic gain, which in turn influences loop stability and contributes to imbalance between loops. At the same time, it pulls the network into an over-stable state, similar to compulsive actions. Learning serves as the bridge between the limbic system (obsession) and the motor system (compulsion). Both systems work synergistically to facilitate the etiological transmission and progression of OCD. Our model provides plausible theoretical explanations for the behavioral features of OCD and may offer new approaches for its treatment.

Neural dissociation between reward and salience prediction errors through the lens of optimistic bias.

The question of how the brain represents reward prediction errors is central to reinforcement learning and adaptive, goal-directed behavior. Previous studies have revealed prediction error representations in multiple electrophysiological signatures, but it remains elusive whether these electrophysiological correlates underlying prediction errors are sensitive to valence (in a signed form) or to salience (in an unsigned form). One possible reason concerns the loose correspondence between objective probability and subjective prediction resulting from the optimistic bias, that is, the tendency to overestimate the likelihood of encountering positive future events. In the present electroencephalography (EEG) study, we approached this question by directly measuring participants' idiosyncratic, trial-to-trial prediction errors elicited by subjective and objective probabilities across two experiments. We adopted monetary gain and loss feedback in Experiment 1 and positive and negative feedback as communicated by the same zero-value feedback in Experiment 2. We provided electrophysiological evidence in time and time-frequency domains supporting both reward and salience prediction error signals. Moreover, we showed that these electrophysiological signatures were highly flexible and sensitive to an optimistic bias and various forms of salience. Our findings shed new light on multiple presentations of prediction error in the human brain, which differ in format and functional role.

Pramipexole Enhances Reward Learning by Preserving Value Estimates

BackgroundDopamine D2-like agonists show promise as treatments for depression. They are thought to act by enhancing reward learning; however, the mechanisms by which they achieve this are not clear. Reinforcement learning accounts describe 3 distinct candidate mechanisms: increased reward sensitivity, increased inverse decision–temperature, and decreased value decay. As these mechanisms produce equivalent effects on behavior, arbitrating between them requires measurement of how expectations and prediction errors are altered. We characterized the effects of 2 weeks of the D2-like agonist pramipexole on reward learning and used functional magnetic resonance imaging measures of expectation and prediction error to assess which of these 3 mechanistic processes were responsible for the behavioral effects. MethodsForty healthy volunteers (50% female) were randomized to 2 weeks of pramipexole (titrated to 1 mg/day) or placebo in a double-blind, between-subject design. Participants completed a probabilistic instrumental learning task before and after the pharmacological intervention, with functional magnetic resonance imaging data collected at the second visit. Asymptotic choice accuracy and a reinforcement learning model were used to assess reward learning. ResultsPramipexole increased choice accuracy in the reward condition with no effect on losses. Participants who received pramipexole had increased blood oxygen level–dependent response in the orbital frontal cortex during the expectation of win trials but decreased blood oxygen level–dependent response to reward prediction errors in the ventromedial prefrontal cortex. This pattern of results indicates that pramipexole enhances choice accuracy by reducing the decay of estimated values during reward learning. ConclusionsThe D2-like receptor agonist pramipexole enhances reward learning by preserving learned values. This is a plausible mechanism for pramipexole’s antidepressant effect.

Identify Elements of the VTA Projecting RPE Circuit Neurons inMouse Mus musculus Prefrontal Cortex

The Reward Prediction Error (RPE) model was presented decades ago but still has elements adding to the theory asthe mystery of neuron signaling pathways unravels. Following the projections of the dopaminergic neurons, the RPEcircuit originates from the ventral tegmental area (VTA), goes to the nucleus accumbens (NAc) reaches a higherorder processing center, which is the prefrontal cortex (PFC). Furthermore, downstream GABAergic neurons will beactivated according to the “error,” which indicates the value difference between the anticipated and actual rewards. Thisstudy aims to investigate the intermediate steps of the circuit, if any, and the role these intermediate neurons played instimulating the GABAergic neurons after the sensory inputs are integrated and processed at the PFC. Numerous studieshave investigated the function and mechanism of the VTA-NAc-PFC connections, whereas this experiment focuses onthe second half of the circuit. At least one, and possibly many, PFC residing, VTA projecting neurons with long axonscan be identified.

A Possible Reward Prediction Error Circuit Exists in Brain: AroundPrefrontal Cortex and Ventral Tegmental Area

This work aims to find a potential neuronal circuit from the Prefrontal cortex to the ventral tegmental area. In neurology,previous research focuses on how signal transfers from the ventral tegmental area to the Prefrontal cortex. Theoretically,the Prefrontal cortex is an effector and a processor. Therefore, a possible circuit, using the Prefrontal cortex as thedeparture, could be designed to find. This work is designed to employ some techniques, floxed genes, neuronal virustracing, and an electrical detector in the brain to prove the desired circuit and give some prediction results. Thoseresults will prove whether the neuronal circuit existed or not. If the electrical signals from the PFC region to VTA canbe detected in termination, such a desired circuit is possible. This work may help to complete the learning mechanism,reward prediction error system, physiologically.

Dopamine in the dorsal bed nucleus of stria terminalis signals Pavlovian sign-tracking and reward violations.

Midbrain and striatal dopamine signals have been extremely well characterized over the past several decades, yet novel dopamine signals and functions in reward learning and motivation continue to emerge. A similar characterization of real-time sub-second dopamine signals in areas outside of the striatum has been limited. Recent advances in fluorescent sensor technology and fiber photometry permit the measurement of dopamine binding correlates, which can divulge basic functions of dopamine signaling in non-striatal dopamine terminal regions, like the dorsal bed nucleus of the stria terminalis (dBNST). Here, we record GRABDA signals in the dBNST during a Pavlovian lever autoshaping task. We observe greater Pavlovian cue-evoked dBNST GRABDA signals in sign-tracking (ST) compared to goal-tracking/intermediate (GT/INT) rats and the magnitude of cue-evoked dBNST GRABDA signals decreases immediately following reinforcer-specific satiety. When we deliver unexpected rewards or omit expected rewards, we find that dBNST dopamine signals encode bidirectional reward prediction errors in GT/INT rats, but only positive prediction errors in ST rats. Since sign- and goal-tracking approach strategies are associated with distinct drug relapse vulnerabilities, we examined the effects of experimenter-administered fentanyl on dBNST dopamine associative encoding. Systemic fentanyl injections do not disrupt cue discrimination but generally potentiate dBNST dopamine signals. These results reveal multiple dBNST dopamine correlates of learning and motivation that depend on the Pavlovian approach strategy employed.

Involvement of cortical input to the rostromedial tegmental nucleus in aversion to foot shock.

The rostromedial tegmental nucleus (RMTg) encodes negative reward prediction error (RPE) and plays an important role in guiding behavioral responding to aversive stimuli. Previous research has focused on regulation of RMTg activity by the lateral habenula despite studies revealing RMTg afferents from other regions including the frontal cortex. The current study provides a detailed anatomical and functional analysis of cortical input to the RMTg of male rats. Retrograde tracing uncovered dense cortical input to the RMTg spanning the medial prefrontal cortex, the orbitofrontal cortex and anterior insular cortex. Afferents were most dense in the dorsomedial subregion of the PFC (dmPFC), an area that is also implicated in both RPE signaling and aversive responding. RMTg-projecting dmPFC neurons originate in layer V, are glutamatergic, and collateralize to select brain regions. In-situ mRNA hybridization revealed that neurons in this circuit are predominantly D1 receptor-expressing with a high degree of D2 receptor colocalization. Consistent with cFos induction in this neural circuit during exposure to foot shock and shock-predictive cues, optogenetic stimulation of dmPFC terminals in the RMTg drove avoidance. Lastly, acute slice electrophysiology and morphological studies revealed that exposure to repeated foot shock resulted in significant physiological and structural changes consistent with a loss of top-down modulation of RMTg-mediated signaling. Altogether, these data reveal the presence of a prominent cortico-subcortical projection involved in adaptive behavioral responding to aversive stimuli such as foot shock and provide a foundation for future work aimed at exploring alterations in circuit function in diseases characterized by deficits in cognitive control over reward and aversion.

Impulse control disorder in Parkinson's disease is associated with abnormal frontal value signalling.

Dopaminergic medication is well established to boost reward- versus punishment-based learning in Parkinson's disease. However, there is tremendous variability in dopaminergic medication effects across different individuals, with some patients exhibiting much greater cognitive sensitivity to medication than others. We aimed to unravel the mechanisms underlying this individual variability in a large heterogeneous sample of early-stage patients with Parkinson's disease as a function of comorbid neuropsychiatric symptomatology, in particular impulse control disorders and depression. One hundred and ninety-nine patients with Parkinson's disease (138 ON medication and 61 OFF medication) and 59 healthy controls were scanned with functional MRI while they performed an established probabilistic instrumental learning task. Reinforcement learning model-based analyses revealed medication group differences in learning from gains versus losses, but only in patients with impulse control disorders. Furthermore, expected-value related brain signalling in the ventromedial prefrontal cortex was increased in patients with impulse control disorders ON medication compared with those OFF medication, while striatal reward prediction error signalling remained unaltered. These data substantiate the hypothesis that dopamine's effects on reinforcement learning in Parkinson's disease vary with individual differences in comorbid impulse control disorder and suggest they reflect deficient computation of value in medial frontal cortex, rather than deficient reward prediction error signalling in striatum. See Michael Browning (https://doi.org/10.1093/brain/awad248) for a scientific commentary on this article.

Factual and counterfactual learning in major adolescent depressive disorder, evidence from an instrumental learning study.

The incidence of adolescent depressive disorder is globally skyrocketing in recent decades, albeit the causes and the decision deficits depression incurs has yet to be well-examined. With an instrumental learning task, the aim of the current study is to investigate the extent to which learning behavior deviates from that observed in healthy adolescent controls and track the underlying mechanistic channel for such a deviation. We recruited a group of adolescents with major depression and age-matched healthy control subjects to carry out the learning task with either gain or loss outcome and applied a reinforcement learning model that dissociates valence (positive v. negative) of reward prediction error and selection (chosen v. unchosen). The results demonstrated that adolescent depressive patients performed significantly less well than the control group. Learning rates suggested that the optimistic bias that overall characterizes healthy adolescent subjects was absent for the depressive adolescent patients. Moreover, depressed adolescents exhibited an increased pessimistic bias for the counterfactual outcome. Lastly, individual difference analysis suggested that these observed biases, which significantly deviated from that observed in normal controls, were linked with the severity of depressive symoptoms as measured by HAMD scores. By leveraging an incentivized instrumental learning task with computational modeling within a reinforcement learning framework, the current study reveals a mechanistic decision-making deficit in adolescent depressive disorder. These findings, which have implications for the identification of behavioral markers in depression, could support the clinical evaluation, including both diagnosis and prognosis of this disorder.

Prediction errors arising from switches between major and minor modes in music: An fMRI study

The major and minor modes in Western music have positive and negative connotations, respectively. The present fMRI study examined listeners’ neural responses to switches between major and minor modes. We manipulated the final chords of J. S. Bach’s keyboard pieces so that each major-mode passage ended with either the major (Major-Major) or minor (Major-Minor) tonic chord, and each minor-mode passage ended with either the minor (Minor-Minor) or major (Minor-Major) tonic chord. If the final major and minor chords have positive and negative reward values respectively, the Major-Minor and Minor-Major stimuli would cause negative and positive reward prediction errors (RPEs) respectively in a listener’s brain. We found that activity in a frontoparietal network was significantly higher for Major-Minor than for Major-Major. Based on previous research, these results support the idea that a major-to-minor switch causes negative RPE. The contrast of Minor-Major minus Minor-Minor yielded activation in the ventral insula and visual cortex, speaking against the idea that a minor-to-major switch causes positive RPE. We discuss our results in relation to executive functions and the emotional connotations of major versus minor modes.

Anticipation of Appetitive Operant Action Induces Sustained Dopamine Release in the Nucleus Accumbens.

The mesolimbic dopamine system is implicated in signaling reward-related information as well as in actions that generate rewarding outcomes. These implications are commonly investigated in either pavlovian or operant reinforcement paradigms, where only the latter requires instrumental action. To parse contributions of reward- and action-related information to dopamine signals, we directly compared the two paradigms: male rats underwent either pavlovian or operant conditioning while dopamine release was measured in the nucleus accumbens, a brain region central for processing this information. Task conditions were identical with the exception of the operant-lever response requirement. Rats in both groups released the same quantity of dopamine at the onset of the reward-predictive cue. However, only the operant-conditioning group showed a subsequent, sustained plateau in dopamine concentration throughout the entire 5 s cue presentation (preceding the required action). This dopamine ramp was unaffected by probabilistic reward delivery, occurred exclusively before operant actions, and was not related to task performance or task acquisition as it persisted throughout the 2 week daily behavioral training. Instead, the ramp flexibly increased in duration with longer cue presentation, seemingly modulating the initial cue-onset-triggered dopamine release, that is, the reward prediction error (RPE) signal, as both signal amplitude and sustainment diminished when reward timing was made more predictable. Thus, our findings suggest that RPE and action components of dopamine release can be differentiated temporally into phasic and ramping/sustained signals, respectively, where the latter depends on the former and presumably reflects the anticipation or incentivization of appetitive action, conceptually akin to motivation.SIGNIFICANCE STATEMENT It is unclear whether the components of dopamine signals that are related to reward-associated information and reward-driven approach behavior can be separated. Most studies investigating the dopamine system use either pavlovian or operant conditioning, which both involve the delivery of reward and necessitate appetitive approach behavior. Thus, used exclusively, neither paradigm can disentangle the contributions of these components to dopamine release. However, by combining both paradigms in the same study, we find that anticipation of a reward-driven operant action induces a modulation of reward-prediction-associated dopamine release, producing so-called dopamine ramps. Therefore, our findings provide new insight into dopamine ramps and suggest that dopamine signals integrate reward and appetitive action in a temporally distinguishable, yet dependent, manner.

Ventral striatum dopamine release encodes unique properties of visual stimuli in mice.

The mesolimbic dopamine system is an evolutionarily conserved set of brain circuits that plays a role in attention, appetitive behavior, and reward processing. In this circuitry, ascending dopaminergic projections from the ventral midbrain innervate targets throughout the limbic forebrain, such as the ventral striatum/nucleus accumbens (NAc). Dopaminergic signaling in the NAc has been widely studied for its role in behavioral reinforcement, reward prediction error encoding, and motivational salience. Less well characterized is the role of dopaminergic neurotransmission in the response to surprising or alerting sensory events. To address this, we used the genetically encoded dopamine sensor dLight1 and fiber photometry to explore the ability of striatal dopamine release to encode the properties of salient sensory stimuli in mice, such as threatening looming discs. Here, we report that lateral NAc (LNAc) dopamine release encodes the rate and magnitude of environmental luminance changes rather than visual stimulus threat level. This encoding is highly sensitive, as LNAc dopamine could be evoked by light intensities that were imperceptible to human experimenters. We also found that light-evoked dopamine responses are wavelength-dependent at low irradiances, independent of the circadian cycle, robust to previous exposure history, and involve multiple phototransduction pathways. Thus, we have further elaborated the mesolimbic dopamine system's ability to encode visual information in mice, which is likely relevant to a wide body of scientists employing light sources or optical methods in behavioral research involving rodents.

How a token-based game may elicit the reward prediction error and increase engagement of students in elementary school. A pilot study.

Student engagement is essential to academic success and student-wellbeing. In the past, fostering engagement though extrinsic rewards has often been found to be of limited effectiveness over the long term. However, extrinsic rewards are important for improving engagement with non-intrinsically rewarding activities. Thus, in the present study a mechanism that is meant to prolong the effects of extrinsic rewards was investigated: the reward prediction error. This error occurs when rewards are awarded contrary to the awardee's expectations. In a quasi-experiment, 39 elementary school students participated in a classroom-based game, which was supposed to motivate them to solve math exercises. It combined reinforcement with elements of luck, which were supposed to elicit the reward prediction error. After 2 weeks, the intervention group had completed significantly more math exercises compared to a pretest and, importantly, also more correctly solved exercises than a control group. This suggests that game-based reinforcement that elicits the reward prediction error might help to increase student engagement over the medium term. It furthermore highlights the importance of applying gamification elements not only digitally but also in analog settings.