Differences In Learning Rates Research Articles

The origins of time compression diseconomies

AbstractResearch SummaryThat capability development is subject to time compression diseconomies (TCD) is well‐known in the strategy literature. However, so far, there is limited attention paid to its origins, that is, why it exists, and demonstrating it empirically. In the context of fertility clinics in the United Kingdom, we show that faster experience accumulation is associated with lower success rates; that is, time compression in experience accumulation results in shallower learning curves. We also show that TCD is exacerbated for clinics that mainly treat complex cases and is mitigated for clinics that employ an integrator to coordinate across the different specialist functions involved in the treatment process. We propose differential learning rates as the mechanism that underlies TCD, and develop implications for firm capabilities and sources of competitive advantage.Managerial SummaryAs firms pursue new opportunities, it is intuitive to grow fast. Rapid growth has many advantages, including a quicker payback on investment as well as garner first mover advantages. However, there can also be a dark side to such high paced growth. By analyzing data on in‐vitro fertilization clinics in the United Kingdom, we show that clinics that grew the fastest were slower in improving their treatment success rates, while slower growing clinics improved more with increasing experience. This penalty of rapid growth is more severe for clinics that treat more complex cases, but it can be ameliorated by better coordination between specialist activities. Our research serves as a warning that faster is not always better.

The asymmetric learning rates of murine exploratory behavior in sparse reward environments

Goal-oriented behaviors of animals can be modeled by reinforcement learning algorithms. Such algorithms predict future outcomes of selected actions utilizing action values and updating those values in response to the positive and negative outcomes. In many models of animal behavior, the action values are updated symmetrically based on a common learning rate, that is, in the same way for both positive and negative outcomes. However, animals in environments with scarce rewards may have uneven learning rates. To investigate the asymmetry in learning rates in reward and non-reward, we analyzed the exploration behavior of mice in five-armed bandit tasks using a Q-learning model with differential learning rates for positive and negative outcomes. The positive learning rate was significantly higher in a scarce reward environment than in a rich reward environment, and conversely, the negative learning rate was significantly lower in the scarce environment. The positive to negative learning rate ratio was about 10 in the scarce environment and about 2 in the rich environment. This result suggests that when the reward probability was low, the mice tend to ignore failures and exploit the rare rewards. Computational modeling analysis revealed that the increased learning rates ratio could cause an overestimation of and perseveration on rare-rewarding events, increasing total reward acquisition in the scarce environment but disadvantaging impartial exploration.

Are efficient learners of verbal stimuli also efficient and precise learners of visuospatial stimuli?

ABSTRACT People differ in how quickly they learn information and how long they remember it, and these two variables are correlated such that people who learn more quickly tend to retain more of the newly learned information. Zerr and colleagues [2018. Learning efficiency: Identifying individual differences in learning rate and retention in healthy adults. Psychological Science, 29(9), 1436–1450] termed the relation between learning rate and retention as learning efficiency, with more efficient learners having both a faster acquisition rate and better memory performance after a delay. Zerr et al. also demonstrated in separate experiments that how efficiently someone learns is stable across a range of days and years with the same kind of stimuli. The current experiments (combined N = 231) replicate the finding that quicker learning coincides with better retention and demonstrate that the correlation extends to multiple types of materials. We also address the generalisability of learning efficiency: A person’s efficiency with learning Lithuanian-English (verbal-verbal) pairs predicts their efficiency with Chinese-English (visuospatial-verbal) and (to a lesser extent) object-location (visuospatial-visuospatial) paired associates. Finally, we examine whether quicker learners also remember material more precisely by using a continuous measure of recall accuracy with object-location pairs.

What Can Performance in the IEDS Task Tell Us About Attention Shifting in Clinical Groups?

The Intra-Extra-dimensional set shift task (IEDS) is a widely used test of learning and attention, believed to be sensitive to aspects of executive function. The task proceeds through a number of stages, and it is generally claimed that patterns of errors across stages can be used to discriminate between reduced attention switching and more general reductions in rates of learning. A number of papers have used the IEDS task to argue for specific attention shifting difficulties in Autism Spectrum Disorder (ASD) and Schizophrenia, however, it remains unclear how well the IEDS really differentiates between reduced attention shifting and other causes of impaired performance. To address this issue, we introduce a simple computational model of performance in the IEDS task, designed to separate the competing effects of attention shifting and general learning rate. We fit the model to data from ASD and comparison individuals matched on age and IQ, as well as to data from four previous studies which used the IEDS task. Model fits do not show consistent evidence for reductions in attention shifting rates in ASD and Schizophrenia. Instead, we find performance is better explained by differences in learning rate, particularly from punishment, which we show correlates with IQ. We, therefore, argue that the IEDS task is not a good measure of attention shifting in clinical groups. LAY SUMMARY: The Intra-Extra-Dimensional Set shift task (IEDS) is often given to autistic individuals, who tend to make more errors relative to comparison groups. This higher error rate is taken to mean that autistic individuals struggle with attention control. Our computational model of the IEDS shows that the performance of ASD and some other clinical groups can be explained instead by differences in learning rate, rather than differences in attention control.

Self-compassion Buffers Impaired Decision-Making by Potential Problem Gamblers in a Casino Setting.

Self-compassionate individuals treat themselves kindly when undergoing stress. The present study examined self-compassion's relationship to risky decision-making on a gambling task by 240 problem and non-problem gamblers who were tested in a casino setting. Multi-level modeling analyses showed that participants expressed differential rates of learning to avoid risks on the gambling task, depending on their status as potential problem/non-problem gamblers and their level of self-compassion. Among potential problem gamblers, participants higher in self-compassion showed significant gains in performance over decision-making trials, which approximated those of non-problem gamblers and adults without impairments in the population. In contrast, potential problem gamblers lower in self-compassion showed chance levels of performance, which approximated those of adults with impairments. In some circumstances, self-compassion can disinhibit individuals from taking greater risks. For potential problem gamblers in a casino setting, however, the benefits of self-compassion as a means to reinforce self-controlappear to outweigh the risks.

Infants born full term and preterm increase the height of anti-gravity leg movements during a kick-activated mobile task using a scaffolded task environment.

Prior research supports that infants born very preterm (PT), compared with full term (FT), have early differences in rate of learning and motor control that may hinder their ability to learn challenging motor tasks. Four-month-old infants born FT (n=18) and PT (n=18) participated in an infant kick-activated mobile task that was scaffolded to motivate progressively higher kicks. We found the FT group learned the association between their leg movements and mobile activation on the second day, but the PT group learned the association on the third day. Both groups of infants increased the height of their kicks on the day they learned the task, compared with their spontaneous kicking height. These findings suggest that infants born PT have the ability to learn challenging motor tasks, such as kicking high, when participating in a task environment that uses scaffolding.

The impact of cognitive aging on route learning rate and the acquisition of landmark knowledge

Aging is accompanied by changes in general cognitive functioning which may impact the learning rate of older adults; however, this is often not controlled for in cognitive aging studies. We investigated the contribution of differences in learning rates to age-related differences in landmark knowledge acquired from route learning. In Experiment 1 we used a standard learning procedure in which participants received a fixed amount of exposure to a route. Consistent with previous research, we found age-related deficits in associative cue and landmark sequence knowledge. Experiment 2 controlled for differences in learning rates by using a flexible exposure learning procedure. Specifically, participants were trained to a performance criterion during route learning before being tested on the content of their route knowledge. While older adults took longer to learn the route than younger adults, the age-related differences in associative cue knowledge were abolished. The deficit in landmark sequence knowledge, however, remained. Experiment 3 replicated these results and introduced a test situation in which a deficit in landmark sequence knowledge yielded an increased likelihood of disorientation in older adults. The findings of this study suggest that age-related deficits in landmark associative cue knowledge are attenuated by controlling for learning rates. In contrast, landmark sequence knowledge deficits persist and are best explained by changes in the learning strategy of older adults to acquire task essential associative cue knowledge at the expense of supplementary sequence knowledge.

Transfer of Odor Perception From the Retronasal to the Orthonasal Pathway.

Although orthonasal odorants are often associated with the external environment, retronasal odorants are accompanied by consummatory behaviors and indicate an internal state of an animal. Our study aimed to examine whether the same odorants may generate a consistent perceptual experience when 2 olfactory routes potentiate variations in concentration in the nasal cavity and orosensory activation. A customized lick spout with vacuum removing odorants around the animal's nares was used to render a pure retronasal exposure experience. We found that pre-exposing rats to odorants retronasally with positive or negative reinforcers (sweet or bitter) lead to a significant learning rate difference between high- and low-vapor-pressure odorants. This effect was not observed for novel odorants, suggesting that odorants may generate similar perceptual quality in a volatility-dependent manner.

FetNet: a recurrent convolutional network for occlusion identification in fetoscopic videos

PurposeFetoscopic laser photocoagulation is a minimally invasive surgery for the treatment of twin-to-twin transfusion syndrome (TTTS). By using a lens/fibre-optic scope, inserted into the amniotic cavity, the abnormal placental vascular anastomoses are identified and ablated to regulate blood flow to both fetuses. Limited field-of-view, occlusions due to fetus presence and low visibility make it difficult to identify all vascular anastomoses. Automatic computer-assisted techniques may provide better understanding of the anatomical structure during surgery for risk-free laser photocoagulation and may facilitate in improving mosaics from fetoscopic videos.MethodsWe propose FetNet, a combined convolutional neural network (CNN) and long short-term memory (LSTM) recurrent neural network architecture for the spatio-temporal identification of fetoscopic events. We adapt an existing CNN architecture for spatial feature extraction and integrated it with the LSTM network for end-to-end spatio-temporal inference. We introduce differential learning rates during the model training to effectively utilising the pre-trained CNN weights. This may support computer-assisted interventions (CAI) during fetoscopic laser photocoagulation.ResultsWe perform quantitative evaluation of our method using 7 in vivo fetoscopic videos captured from different human TTTS cases. The total duration of these videos was 5551 s (138,780 frames). To test the robustness of the proposed approach, we perform 7-fold cross-validation where each video is treated as a hold-out or test set and training is performed using the remaining videos.ConclusionFetNet achieved superior performance compared to the existing CNN-based methods and provided improved inference because of the spatio-temporal information modelling. Online testing of FetNet, using a Tesla V100-DGXS-32GB GPU, achieved a frame rate of 114 fps. These results show that our method could potentially provide a real-time solution for CAI and automating occlusion and photocoagulation identification during fetoscopic procedures.

The Role of the Striatum in Learning to Orthogonalize Action and Valence: A Combined PET and 7T MRI Aging Study.

Pavlovian biases influence instrumental learning by coupling reward seeking with action invigoration and punishment avoidance with action suppression. Using a probabilistic go/no-go task designed to orthogonalize action (go/no-go) and valence (reward/punishment), recent studies have shown that the interaction between the two is dependent on the striatum and its key neuromodulator dopamine. Using this task, we sought to identify how structural and neuromodulatory age-related differences in the striatum may influence Pavlovian biases and instrumental learning in 25 young and 31 older adults. Computational modeling revealed a significant age-related reduction in reward and punishment sensitivity and marked (albeit not significant) reduction in learning rate and lapse rate (irreducible noise). Voxel-based morphometry analysis using 7Tesla MRI images showed that individual differences in learning rate in older adults were related to the volume of the caudate nucleus. In contrast, dopamine synthesis capacity in the dorsal striatum, assessed using [18F]-DOPA positron emission tomography in 22 of these older adults, was not associated with learning performance and did not moderate the relationship between caudate volume and learning rate. This multiparametric approach suggests that age-related differences in striatal volume may influence learning proficiency in old age.

The effect of instructional set size on learning efficiency

ABSTRACT Research has found that factors such as content, prompts, and individual motivational differences affect instructional effectiveness. While instructional effectiveness is acritical outcome variable, the limited availability of instructional time results in instructional efficiency being acritical consideration. This study evaluated whether varying instructional set sizes (i.e. the number of unknown items taught within a specified period of time) produced differential learning. Instruction was provided on arbitrary number–word relationships. Response to learning sets was studied using a within-participants experimental design. This study differs from prior research by examining set size for the commonly used constant time delay procedure. The results suggested that, for the majority of students, learning five novel items at a time was less efficient than one of the larger sets of unknown items. Additionally, participants demonstrated individual differences in learning rates. The implications of these findings for instructional presentation and intervention efficiency are discussed.

Abstract 2474: Automated somatic variant classifier to reduce false positives identified by tumor normal variant callers

Abstract Accurate identification of somatic mutations is key to targeted cancer therapies. However, due to the complexity of next-generation sequencing, there are many variables that give rise to sequencing and alignment artifacts. Most sequencing workflows using tumor normal paired samples rely on the intersection or union of mutations by more than one somatic variant caller such as Mutect2, Strelka, VarScan2, Muse, and SomaticSniper. Yet, they still contain a number of false positive and false negative calls. Therefore, there is a need for a manual review of variants using Integrative Genomics Viewer (IGV) to incorporate information that is difficult to take into account in automated workflows. To alleviate the burden of manual inspection, we present an automated classification method to pre classify detected somatic variants into true or false classes for further manual review. Methods: We created a false positive classifier that utilizes automated IGV visualization of somatic variants. Our somatic workflow involves tumor normal DNA sequencing, BWAMEM alignment, deduplication of reads via Picard, somatic calling using two somatic callers, followed by the union of the detected variants, SnpEff annotation, and extracting nonsilent variants with variant allele fractions greater than 5 percent. We used the data for 1,413 nonsilent variants from a set of 10 tumor normal pairs of various cancer types. The variants were scored by cancer analysts using IGV. Initial false positive rate was 28 percent. We fine tuned Residual Deep Neural Network which utilizes weights from the network with the same architecture trained on ImageNet. Discriminative fine tuning was performed in two stages: first unfreezing the last fully connected layer only, and then unfreezing all the layers and training with differential learning rates. Results and Conclusion: Our method achieves 93 percent overall accuracy. The classifier greatly reduces the false positive rate for the set of variant calls from 28 percent down to 4 percent at the expense of calling 3 percent of true variants false. Previously, deep neural networks have been shown to yield the state of the art performance in multiple domains including image processing, speech, and text processing. In this work, we describe a way to fine tune Residual Neural Network trained on a large image data set to perform false positive variant detection in tumor-normal variant alignment snapshots from IGV. Our general approach that significantly reduces false positive rate of putative variants can be extended to any subset of somatic callers. Citation Format: Alena S. Harley, Corine K. Lau, Eve Shinbrot. Automated somatic variant classifier to reduce false positives identified by tumor normal variant callers [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2019; 2019 Mar 29-Apr 3; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2019;79(13 Suppl):Abstract nr 2474.

Dopaminergic drug treatment remediates exaggerated cingulate prediction error responses in obsessive-compulsive disorder

RationalePatients with obsessive-compulsive disorder (OCD) have been found to show exaggerated error responses and prediction error learning signals in a variety of EEG and fMRI tasks, with data converging on the anterior cingulate cortex as a key locus of dysfunction. Considerable evidence has linked prediction error processing to dopaminergic function.ObjectiveIn this study, we investigate potential dopaminergic dysfunction during reward processing in the context of OCD.MethodsWe studied OCD patients (n = 18) and controls (n = 18) whilst they learned probabilistic associations between abstract stimuli and monetary rewards in the fMRI scanner involving administration (on separate visits) of a dopamine receptor agonist, pramipexole 0.5 mg; a dopamine receptor antagonist, amisulpride 400 mg; and placebo. We fitted a Q-learning computational model to fMRI prediction error responses; group differences were examined in anterior cingulate and nucleus accumbens regions of interest.ResultsThere were no significant group, drug, or interaction effects in the number of correct choices; computational modeling suggested a marginally significant difference in learning rates between groups (p = 0.089, partial ƞ2 = 0.1). In the imaging results, there was a significant interaction of group by drug (p = 0.013, partial ƞ2 = 0.13). OCD patients showed abnormally strong cingulate signaling of prediction errors during omission of an expected reward, with unexpected reduction by both pramipexole and amisulpride (p = 0.014, partial ƞ2 = 0.26, 1-β error probability = 0.94). Exaggerated cingulate prediction error signaling to omitted reward in placebo was related to trait subjective difficulty in self-regulating behavior in OCD.ConclusionsOur data support cingulate dysfunction during reward processing in OCD, and bidirectional remediation by dopaminergic modulation, suggesting that exaggerated cingulate error signals in OCD may be of dopaminergic origin. The results help to illuminate the mechanisms through which dopamine receptor antagonists achieve therapeutic benefit in OCD. Further research is needed to disentangle the different functions of dopamine receptor agonists and antagonists during bidirectional modulation of cingulate activation.

Identifying the Elasticity of Experience and Its Effect on Market Structure

Experience is a factor of production. Practice makes perfect. But when prior production enters the firm's current production function, standard dynamic models of firm production like the Olley and Pakes (1996) proxy model suffer from a simultaneity problem. Furthermore, flexible inputs are chosen dynamically and firms may even choose negative markups temporarily to increase their stock of experience. So identification approaches that are based on assuming zero or positive markups will not work (like Gandhi, Navarro, and Rivers 2019). I develop an identification approach that works for this problem. My approach only requires panel data on firm input and output choice so it can be broadly applied. I use this appoach to study the evolution of learning rates (the rate at which firms learn from experience over time) and how differences in learning rates help explain differences in the firm size distribution across industries.

Episodic memory and learning rates in amyotrophic lateral sclerosis without dementia

In amyotrophic lateral sclerosis (ALS), memory deficits may be primary or secondary to executive dysfunction. We assessed episodic memory and executive function of nondemented ALS patients, comparing episodic memory profiles and learning rates of ALS patients with those of mild cognitive impairment (MCI) subjects and cognitively healthy controls (HC). In a multidisciplinary tertiary centre for motor neuron disease, 72 nondemented ALS patients, 57 amnestic MCI (aMCI), 89 single non amnestic MCI with compromised executive functions (dysexecutive MCI), and 190 HC were enrolled. They were screened using the Frontal Assessment Battery and Mini Mental State Examination. Episodic memory performances and learning rates were tested using the Rey Auditory Verbal Learning Test (RAVLT). Episodic memory dysfunction (immediate recall) was found in 14 ALS patients (19.4%). The ALS group had lower performance than HC on immediate recall, without differences in learning rate, and better performance than aMCI subjects on all RAVLT measures. Compared to dysexecutive MCI subjects, ALS patients had only better verbal learning abilities. ALS patients with executive dysfunction had a lower score on immediate and delayed recalls, verbal learning, and primacy effect than ALS patients without executive dysfunction. The immediate recall among couples of diagnostic groups differed in a statistically significant way except for the ALS/dysexecutive MCI groups. In ALS patients, episodic memory performances and learning rates appeared to be better than in aMCI subjects and similar to those with dysexecutive MCI, suggesting also a secondary functional damage due to executive impairment.

Selective attention in rat visual category learning.

A prominent theory of category learning, COVIS, posits that new categories are learned with either a declarative or procedural system, depending on the task. The declarative system uses the prefrontal cortex (PFC) to learn rule-based (RB) category tasks in which there is one relevant sensory dimension that can be used to establish a rule for solving the task, whereas the procedural system uses corticostriatal circuits for information integration (II) tasks in which there are multiple relevant dimensions, precluding use of explicit rules. Previous studies have found faster learning of RB versus II tasks in humans and monkeys but not in pigeons. The absence of a learning rate difference in pigeons has been attributed to their lacking a PFC. A major gap in this comparative analysis, however, is the lack of data from a nonprimate mammalian species, such as rats, that have a PFC but a less differentiated PFC than primates. Here, we investigated RB and II category learning in rats. Similar to pigeons, RB and II tasks were learned at the same rate. After reaching a learning criterion, wider distributions of stimuli were presented to examine generalization. A second experiment found equivalent RB and II learning with wider category distributions. Computational modeling revealed that rats extract and selectively attend to category-relevant information but do not consistently use rules to solve the RB task. These findings suggest rats are on a continuum of PFC function between birds and primates, with selective attention but limited ability to utilize rules relative to primates.

Cortical neural modulation by previous trial outcome differentiates fast- from slow-learning rats on a visuomotor directional choice task.

To better understand the neural cortical underpinnings that explain behavioral differences in learning rate, we recorded single-unit activity in primary motor (M1) and secondary motor (M2) areas while rats learned to perform a directional (left or right) operant visuomotor association task. Analysis of neural activity during the early portion of the cue period showed that neural modulation in the motor cortex was most strongly associated with two task factors: the previous trial outcome (success or error) and the current trial's directional choice (left or right). Furthermore, the fast learners, defined as those who had steeper learning curves and required fewer learning sessions to reach criterion performance, encoded the previous trial outcome factor more strongly than the directional choice factor. Conversely, the slow learners encoded directional choice more strongly than previous trial outcome. These differences in task factor encoding were observed in both the percentage of neurons and the neural modulation depth. These results suggest that fast learning is accompanied by a stronger component of previous trial outcome in the modulation representation present in motor cortex, which therefore may be a contributing factor to behavioral differences in learning rate. NEW & NOTEWORTHY We chronically recorded neural activity as rats learned a visuomotor directional choice task from a naive state. Learning rates varied. Single-unit neural modulation of two motor areas revealed that the fast learners encoded previous trial outcome more strongly than directional choice, whereas the reverse was true for slow learners. This finding provides novel evidence that rat learning rate is strongly correlated with the strength of neural modulation by previous trial outcome in motor cortex.

The statistical structures of reinforcement learning with asymmetric value updates

Reinforcement learning (RL) models have been broadly used in modeling the choice behavior of humans and other animals. In standard RL models, the action values are assumed to be updated according to the reward prediction error (RPE), i.e., the difference between the obtained reward and the expected reward. Numerous studies have noted that the magnitude of the update is biased depending on the sign of the RPE. The bias is represented in RL models by differential learning rates for positive and negative RPEs. However, which aspect of behavioral data that the estimated differential learning rates reflect is not well understood. In this study, we investigate how the differential learning rates influence the statistical properties of choice behavior (i.e., the relation between past experiences and the current choice) based on theoretical considerations and numerical simulations. We clarify that when the learning rates differ, the impact of a past outcome depends on the subsequent outcomes, in contrast to standard RL models with symmetric value updates. Based on the results, we propose a model-neutral statistical test to validate the hypothesis that value updates are asymmetric. The asymmetry in the value updates induces the autocorrelation of choice (i.e., the tendency to repeat the same choice or to switch the choice irrespective of past rewards). Conversely, if an RL model without an intrinsic autocorrelation factor is fitted to data that possess an intrinsic autocorrelation, a statistical bias to overestimate the difference in learning rates arises. We demonstrate that this bias can cause a statistical artifact in RL-model fitting leading to a “pseudo-positivity bias” and a “pseudo-confirmation bias.”

Visuomotor learning and unlearning in children and adolescents with tourette syndrome

Tourette syndrome (TS) is a childhood-onset neurological condition characterised by an evolving repertoire of chronic motor tics and one or more phonic tics. Tics, like habits, are inflexible and repetitive behaviours that are acquired over a period of time. It has been proposed that tics arise in TS as a result of increased habit learning: which may bias the child to acquire automatic behaviours (i.e., tics) more readily than is normal and make it harder to unlearn maladaptive habits once they have been acquired. Using a well-established visuomotor adaptation task, we investigated motor learning in a group of children and adolescents with a clinical diagnosis of TS relative to a group of age and gender matched typically developing individuals. In particular, we quantified differences in the strength and quality of motor learning and unlearning in TS, and the consolidation of motor learning over a 24 h washout period. We demonstrated that there was a marginally significant decrease in learning rate in the individuals with TS relative to age and gender matched typically developing controls. However, this effect was not associated with tic severity and could be entirely accounted for by the severity of co-occurring ADHD symptoms. Thus, once ADHD symptoms had been accounted for, there were no between group differences in learning rate or the degree of learning observed. By contrast, and more importantly, we found that following learning the rate of forgetting (unlearning) was significantly negatively associated with motor tic severity, such that individuals with more severe tics took longer to unlearn previously learnt motor patterns of behaviour. This finding is consistent with the proposal that TS is associated with alterations in the striatal habit learning system and with the view that TS may make it harder to unlearn maladaptive motor habits once they have been acquired.

Noradrenergic and Cholinergic Modulation of Belief Updating.

To make optimal predictions in a dynamic environment, the impact of new observations on existing beliefs-that is, the learning rate-should be guided by ongoing estimates of change and uncertainty. Theoretical work has proposed specific computational roles for various neuromodulatory systems in the control of learning rate, but empirical evidence is still sparse. The aim of the current research was to examine the role of the noradrenergic and cholinergic systems in learning rate regulation. First, we replicated our recent findings that the centroparietal P3 component of the EEG-an index of phasic catecholamine release in the cortex-predicts trial-to-trial variability in learning rate and mediates the effects of surprise and belief uncertainty on learning rate (Study 1, n = 17). Second, we found that pharmacological suppression of either norepinephrine or acetylcholine activity produced baseline-dependent effects on learning rate following nonobvious changes in an outcome-generating process (Study 1). Third, we identified two genes, coding for α2A receptor sensitivity (ADRA2A) and norepinephrine reuptake (NET), as promising targets for future research on the genetic basis of individual differences in learning rate (Study 2, n = 137). Our findings suggest a role for the noradrenergic and cholinergic systems in belief updating and underline the importance of studying interactions between different neuromodulatory systems.