Probability Of Reinforcement Research Articles

The partial reinforcement extinction effect: The proportion of trials reinforced during conditioning predicts the number of trials to extinction.

Four experiments compared the extinction of responding to a continuously reinforced (CRf) conditioned stimulus (conditional stimulus [CS]) consistently reinforced on every trial, with extinction of responding to a partially reinforced (PRf) CS that had been inconsistently reinforced. To equate the acquisition of responding between the two CSs, the average duration of the CRf CS was extended so that it scheduled the same overall rate of reinforcement per unit time as the PRf CS. Experiment 1 used a within-subjects design to compare the rates of extinction for a 10-s PRf CS reinforced on 33% of trials versus a 30-s CRf CS. Experiment 2 made the same comparison but using a between-subjects design. Experiment 3 compared extinction in a group trained with a 10-s PRf CS reinforced on 20% of trials and a group trained with a 50-s CRf CS. Experiment 4 compared the rates of extinction following two partial reinforcement schedules: a 10-s PRf CS reinforced on 33% of trial versus a 20-s CRf CS reinforced on 66% of trials. In each experiment, responding took longer to extinguish for the CS that scheduled a lower per-trial probability of reinforcement. Modeling of individual extinction curves using Weibull functions indicated that the latency to initiate extinction was directly related to the per-trial probability of reinforcement learned during acquisition. For example, compared with training with a CRf CS, rats reinforced on 33% of trials took approximately 3 times as many trials to initiate extinction, and rats reinforced on 20% of trials took 5 times as many trials to initiate extinction. These results provide support for trial-based accounts of extinction (e.g., Capaldi & Deutsch, 1967), whereby rats learn about the expected number of trials per reinforcer, and extinction depends on the number of expected reinforcers that have been omitted rather than on the number of extinction trials per se. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The Role of Frontostriatal Systems in Instructed Reinforcement Learning: Evidence From Genetic and Experimentally-Induced Variation.

Instructions have a powerful effect on learning and decision-making, biasing choice even in the face of disconfirming feedback. Detrimental biasing effects have been reported in a number of studies in which instruction was given prior to trial-and-error learning. Previous work has attributed individual differences in instructional bias to variations in prefrontal and striatal dopaminergic genes, suggesting a role for prefrontally-mediated cognitive control processes in biasing learning. The current study replicates and extends these findings. Human subjects performed a probabilistic reinforcement learning task after receiving inaccurate instructions about the quality of one of the options. In order to establish a causal relationship between prefrontal cortical mechanisms and instructional bias, we applied transcranial direct current stimulation over dorsolateral prefrontal cortex (anodal, cathodal, or sham) while subjects performed the task. We additionally genotyped subjects for the COMT Val158Met genetic polymorphism, which influences the breakdown of prefrontal dopamine, and for the DAT1/SLC6A3 variable number tandem repeat, which affects expression of striatal dopamine transporter. We replicated the finding that the COMT Met allele is associated with increased instructional bias and further demonstrated that variation in DAT1 has similar effects to variation in COMT, with 9-repeat carriers demonstrating increased bias relative to 10-repeat homozygotes. Consistent with increased top-down regulation of reinforcement learning, anodal subjects demonstrated greater bias relative to sham, though this effect was present only early in training. In contrast, there was no effect of cathodal stimulation. Finally, we fit computational models to subjects' data to better characterize the mechanisms underlying instruction bias. A novel choice bias model, in which instructions influence decision-making rather than learning, was found to best account for subjects' behavior. Overall, these data provide further evidence for the role of frontostriatal interactions in biasing instructed reinforcement learning, which adds to the growing literature documenting both costs and benefits of cognitive control.

Probabilistic Learning by Positive and Negative Reinforcement in Generalized Anxiety Disorder.

This experiment examined learning tendencies in generalized anxiety disorder (GAD) using reinforcement feedback for probabilistic outcomes. One hundred sixty-six GAD and 105 non-GAD participants were randomized to a computerized probabilistic learning task that used either negative or positive reinforcement. Participants chose between stimuli with specific probabilities of reinforcement to learn which of each pair had the highest probability. Reinforced choices either removed an angry face (negative reinforcement) or made a happy face appear (positive reinforcement). Results showed that those with GAD learned the correct probabilistic choices at a slower rate over time and to a lesser degree than control participants regardless of reinforcement type. Estimations of the likelihood of receiving a good outcome posttask were also more inaccurate for those with GAD, especially when true likelihoods were high. Furthermore, compared with control participants, those with GAD reported lower perceived reinforcement sensitivity, higher behavioral inhibition sensitivity, and higher undesirable feelings toward probabilistic learning.

Contrast between what is expected and what occurs increases pigeon's suboptimal choice.

When pigeons are given a choice between 50% signaled reinforcement and 100% reinforcement they typically do not choose optimally, sometimes even preferring 50% reinforcement. Smith and Zentall (J Exp Psychol Anim Behav Process 42:212-220, 2016) proposed that choice depends primarily on the predictive value of the signal for reinforcement associated with each alternative (both 100% reinforcement) and not the frequency of the signal for reinforcement (50% vs. 100%). With extended training, however, Case and Zentall (Behav Process, 2018) found that pigeons actually show a reliable preference for the 50% reinforcement alternative. They suggested that contrast between the expected outcome at the time of choice (50% reinforcement) and the value of the signal for reinforcement (100% reinforcement) is the mechanism responsible for the preference for the suboptimal alternative (for the optimal alternative there should be no contrast). In the present research, we tested the contrast hypothesis by increasing the probability of reinforcement for choice of the suboptimal alternative to 75%, thereby reducing the contrast between expected and obtained reinforcement and found a reduced preference for the suboptimal alternative. That is, increasing the probability of reinforcement for choice of the suboptimal alternative decreased suboptimal choice.Thus, preference for the suboptimal alternative appears to result from two mechanisms: (1) the value of the signal for reinforcement that follows choice of the alternative and (2) positive contrast between the expected and obtained probability of reinforcement. We compared this interpretation with other hypotheses.

Neural modulation of social reinforcement learning by intranasal oxytocin in male adults with high-functioning autism spectrum disorder: a randomized trial.

Reduced social motivation is a hallmark of individuals with autism spectrum disorders (ASDs). Although the exact neural mechanisms are unclear, oxytocin has been shown to enhance motivation and attention to social stimuli, suggesting a potential to augment social reinforcement learning as the central mechanism of behavioral interventions in ASD. We tested how reinforcement learning in social contexts and associated reward prediction error (RPE) signals in the nucleus accumbens (NAcc) were modulated by intranasal oxytocin. Male adults with a childhood diagnosis of ASD (n = 15) and healthy controls (n = 24; aged 18-26 years) performed a probabilistic reinforcement learning task during functional magnetic resonance imaging in a single-center (research center in Germany), randomized double-blind, placebo-controlled cross-over trial. The interventions were intranasal oxytocin (Syntocinon®, Novartis; 10 puffs = 20 international units (IUs) per treatment) and placebo spray. Using computational modeling of behavioral data, trial-by-trial RPE signals were assessed and related to brain activation in NAcc during reinforcing feedback in social and non-social contexts. The order of oxytocin/placebo was randomized for 60 participants. Twenty-one participants were excluded from analyses, leaving 39 for the final analysis. Behaviorally, individuals with ASD showed enhanced learning under oxytocin when the learning target as well as feedback was social as compared to non-social (social vs. non-social target: 87.09% vs. 71.29%, 95% confidence interval (CI): 7.28-24.33, p = .003; social vs. non-social feedback: 81.00% vs. 71.29%, 95% CI: 2.81-16.61, p = .027). Correspondingly, oxytocin enhanced the correlation of the RPE signal with NAcc activation during social (vs. non-social) feedback in ASD (3.48 vs. -1.12, respectively, 95% CI: 2.98-6.22, p = .000), whereas in controls, this effect was found in the placebo condition (2.90 vs. -1.14, respectively, 95% CI: 1.07-7.01, p = .010). In ASD, a similar pattern emerged when the learning target was social (3.00 vs. -0.64, respectively, 95% CI: -0.13 to 7.41, p = .057), whereas controls showed a reduced correlation for social learning targets under oxytocin (-0.70 vs. 2.72, respectively, 95% CI: -5.86 to 0.98, p = .008). The current data suggest that intranasal oxytocin has the potential to enhance social reinforcement learning in ASD. Future studies are warranted that investigate whether oxytocin can potentiate social learning when combined with behavioral therapies, resulting in greater treatment benefits than traditional behavior-only approaches.

Probabilistic reinforcement learning abnormalities and their correlates in adolescent bipolar disorders.

Theoretical models of bipolar disorders (BD) posit core deficits in reward system function. However, specifying which among the multiple reward system's neurobehavioral processes are abnormal in BD is necessary to develop appropriately targeted interventions. Research on probabilistic-reinforcement learning deficits in BD is limited, particularly during adolescence, a period of significant neurodevelopmental changes in the reward system. The present study investigated probabilistic-reinforcement learning, using a probabilistic selection task (PST), and its correlates, using self-reported reward/threat sensitivities and cognitive tasks, in 104 adolescents with and without BD. Compared with healthy peers, adolescents with BD were less likely to persist with their choices based on prior positive feedback (i.e., lower win-stay rates) in the PST's acquisition phase. Across groups, a greater win-stay rate appeared to be a more efficient learning strategy-associated with fewer acquisition trials and better testing phase performance. Win-stay rates were also related to verbal learning indices, but not self-reported reward/threat sensitivities. Finally, lower win-stay rates had significant incremental validity in predicting a BD diagnosis, after accounting for effects of current symptoms, reward sensitivities, verbal learning, and IQ. The present findings support multiple dysfunctional processes of the reward system in adolescent BD that require additional examinations. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Daily-life stress differentially impacts ventral striatal dopaminergic modulation of reward processing in first-degree relatives of individuals with psychosis

Emerging evidence shows that stress can impair the ability to learn from and pursue rewards, which in turn has been linked to motivational impairments characteristic of the psychotic disorder. Ventral striatal dopaminergic neurotransmission has been found to modulate reward processing, and appears to be disrupted by exposure to stress. We investigated the hypothesis that stress experienced in the everyday life has a blunting effect on reward-induced dopamine release in the ventral striatum of 16 individuals at a familial risk for psychosis compared to 16 matched control subjects. Six days of ecological momentary assessments quantified the amount of daily-life stress prior to [18F]fallypride PET imaging while performing a probabilistic reinforcement learning task. Relative to the controls, individuals at a familial risk for psychosis who encountered more daily-life stress showed significantly diminished extent of reward-induced dopamine release in the right ventral striatum, as well as poorer performance on the reward task. These findings provide the first neuromolecular evidence for stress-related deregulation of reward processing in familial predisposition to psychosis. The implication of daily-life stress in compromised modulation of reward function may facilitate the design of targeted neuropharmacological and ecological interventions.

Adquisición y Dinámica de Preferencias en Programas Múltiples de Reforzamiento

Ocho palomas fueron entrenadas en programas múltiples de reforzamiento Razón Variable-Razón Variable (mult RV-RV) con cambios rápidos e imprevistos en las distribuciones de refuerzo en ambos componentes del programa múltiple. El objetivo principal fue evaluar cómo se ajustan las tasas de respuestas a cambios abruptos y no señalados en las condiciones de reforzamiento en distintos períodos y en particular determinar si la dinámica del ajuste del comportamiento es dirigida por la razón o por la diferencia en las probabilidades en las tasas de reforzamiento obtenido en dos componentes de un programa múltiple. Los principales hallazgos fueron que cuando las diferencias entre dos programas de reforzamiento (pobre y rico) son constantes, el desarrollo de la preferencia por una de las respuestas del programa múltiple fue más rápido cuando la razón de las probabilidades de reforzamiento fue mayor (5 a 1), lo cual es congruente con los resultados obtenidos en programas de reforzamiento concurrentes. Sin embargo, cuando la razón se mantuvo constante, la tasa de adquisición no fue más rápida cuando la diferencia entre la probabilidad de reforzamiento fue mayor, hallazgo distinto al reportado en experimentos con programas concurrentes. Los resultados resaltan la importancia de la discriminabilidad en probabilidades de reforzamiento entre la fase de entrenamiento y de transición.

Long-term deficits in risky decision-making after traumatic brain injury on a rat analog of the Iowa gambling task

Traumatic brain injury (TBI) affects 2.8 million people annually in the United States, with significant populations suffering from ongoing cognitive dysfunction. Impairments in decision-making can have major implications for patients and their caregivers, often enduring for years to decades, yet are rarely explored in experimental TBI. In the current study, the Rodent Gambling Task (RGT), an Iowa Gambling Task analog, was used to assess risk-based decision-making and motor impulsivity after TBI. During testing, rats chose between options associated with different probabilities of reinforcement (sucrose) or punishment (timeout). To determine effects of TBI on learned behaviors versus the learning process, rats were trained either before, or after, a bilateral frontal controlled cortical impact TBI, and then assessed for 12 weeks. To evaluate the degree to which monoamine systems, such as dopamine, were affected by TBI, rats were given an amphetamine challenge, and behavior recorded. Injury immediately and chronically decreased optimal decision-making, and biased rats towards both riskier, and safer (but suboptimal) choices, regardless of prior learning history. TBI also increased motor impulsivity across time, reflecting ongoing neural changes. Despite these similarities in trained and acquisition rats, those that learned the task after injury demonstrated reduced effects of amphetamine on optimal decision-making, suggesting a lesser role of monoamines in post-injury learning. Amphetamine also dose-dependently reduced motor impulsivity in injured rats. This study opens up the investigation of psychiatric-like dysfunction in animal models of TBI and tasks such as the RGT will be useful in identifying therapeutics for the chronic post-injury period.

Motivational deficits in schizophrenia relate to abnormalities in cortical learning rate signals.

Individuals from across the psychosis spectrum display impairments in reinforcement learning. In some individuals, these deficits may result from aberrations in reward prediction error (RPE) signaling, conveyed by dopaminergic projections to the ventral striatum (VS). However, there is mounting evidence that VS RPE signals are relatively intact in medicated people with schizophrenia (PSZ). We hypothesized that, in PSZ, reinforcement learning deficits often are not related to RPE signaling per se but rather their impact on learning and behavior (i.e., learning rate modulation), due to dysfunction in anterior cingulate and dorsomedial prefrontal cortex (dmPFC). Twenty-six PSZ and 23 healthy volunteers completed a probabilistic reinforcement learning paradigm with occasional, sudden, shifts in contingencies. Using computational modeling, we found evidence of an impairment in trial-wise learning rate modulation (α) in PSZ before and after a reinforcement contingency shift, expressed most in PSZ with more severe motivational deficits. In a subsample of 22 PSZ and 22 healthy volunteers, we found little evidence for between-group differences in VS RPE and dmPFC learning rate signals, as measured with fMRI. However, a follow-up psychophysiological interaction analysis revealed decreased dmPFC-VS connectivity concurrent with learning rate modulation, most prominently in individuals with the most severe motivational deficits. These findings point to an impairment in learning rate modulation in PSZ, leading to a reduced ability to adjust task behavior in response to unexpected outcomes. At the level of the brain, learning rate modulation deficits may be associated with decreased involvement of the dmPFC within a greater RL network.

Dolphins maintain high echolocation vigilance for eight hours without primary (food) reinforcement.

Studies have demonstrated that dolphins can maintain continuous auditory or echolocation vigilance for up to 5 to 15 days when provided with continuous primary reinforcement (i.e., food reward after each correct detection). The goals of this study were to examine whether dolphins could perform an 8-h echolocation vigilance task featuring variable reinforcement schedules, where correct responses were intermittently rewarded, and variable acoustic secondary reinforcement (feedback) patterns. Three dolphins were trained to echolocate simulated targets and press a response paddle upon detecting echoes. Three conditioned reinforcement conditions were utilized: no (acoustic) feedback, acoustic feedback, and structured acoustic feedback. The probability of primary reinforcement following a correct response began at 50% for all dolphins but was sequentially reduced to 25%, 12%, 6%, and 0% each time performance criteria were met. Conditions including acoustic feedback resulted in two dolphins successfully performing the echolocation vigilance task under the 0% primary reinforcement schedule (8 h before receiving primary reinforcement). None of the animals reached 0% reinforcement probability in the no feedback condition. The results demonstrate that dolphins can perform experimental echolocation tasks for extended time periods without primary reinforcement and suggest that secondary reinforcement may be important to maintain this behavior.

Reinforcement uncertainty enhances preference for choice in humans.

Under concurrent-chains schedules of reinforcement, participants often prefer situations that allow selection among alternatives (free choice) to situations that do not (forced choice). The present experiment examined the effects of reinforcement probability on choice preferences. Preferences for free versus forced choice were measured under a condition in which participants' choices were always reinforced (reinforcement probability of 1.0) and a condition in which outcomes were uncertain (reinforcement probability of 0.5). Forty-four college students participated and preferences were examined under a concurrent-chains schedule of reinforcement. Participants preferred free choice under uncertain reinforcement, but a bias toward free choice was not observed when reinforcement was certain. These results align with previous findings of preference for free choice under conditions of uncertainty, but suggest that preference may be dependent upon probabilistic reinforcement contingencies in the terminal links of the concurrent-chains arrangement. Thus, reinforcement probability is an important variable to consider when conducting similar studies on the value of choice.

Selective Effects of the Loss of NMDA or mGluR5 Receptors in the Reward System on Adaptive Decision-Making.

Selecting the most advantageous actions in a changing environment is a central feature of adaptive behavior. The midbrain dopamine (DA) neurons along with the major targets of their projections, including dopaminoceptive neurons in the frontal cortex and basal ganglia, play a key role in this process. Here, we investigate the consequences of a selective genetic disruption of NMDA receptor and metabotropic glutamate receptor 5 (mGluR5) in the DA system on adaptive choice behavior in mice. We tested the effects of the mutation on performance in the probabilistic reinforcement learning and probability-discounting tasks. In case of the probabilistic choice, both the loss of NMDA receptors in dopaminergic neurons or the loss mGluR5 receptors in D1 receptor-expressing dopaminoceptive neurons reduced the probability of selecting the more rewarded alternative and lowered the likelihood of returning to the previously rewarded alternative (win-stay). When observed behavior was fitted to reinforcement learning models, we found that these two mutations were associated with a reduced effect of the expected outcome on choice (i.e., more random choices). None of the mutations affected probability discounting, which indicates that all animals had a normal ability to assess probability. However, in both behavioral tasks animals with targeted loss of NMDA receptors in dopaminergic neurons or mGluR5 receptors in D1 neurons were significantly slower to perform choices. In conclusion, these results show that glutamate receptor-dependent signaling in the DA system is essential for the speed and accuracy of choices, but at the same time probably is not critical for correct estimation of probable outcomes.

A new variable interval schedule with constant hazard rate and finite time range.

We propose a new variable interval (VI) schedule that achieves constant probability of reinforcement in time while using a bounded range of intervals. By sampling each trial duration from a uniform distribution ranging from 0 to 2 T seconds, and then applying a reinforcement rule that depends linearly on trial duration, the schedule alternates reinforced and unreinforced trials, each less than 2 T seconds, while preserving a constant hazard function.

Dopamine Release in Antidepressant-Naive Major Depressive Disorder: A Multimodal [11C]-(+)-PHNO Positron Emission Tomography and Functional Magnetic Resonance Imaging Study

BackgroundMesolimbic dopamine system dysfunction is believed to contribute to major depressive disorder (MDD), but molecular neuroimaging of striatal dopamine neurotransmission has yielded mixed results, possibly owing to limited sensitivity of antagonist radioligands used with positron emission tomography to assess dopamine release capacity. This study used an agonist radioligand with agonist challenge to assess dopamine release capacity and D2/D3 receptor availability in MDD. MethodsTwenty-six treatment-naive adults with MDD and 26 healthy comparison participants underwent functional magnetic resonance imaging during a probabilistic reinforcement task, and positron emission tomography with the D3-preferring ligand [11C]-(+)-PHNO, before and after oral dextroamphetamine. MDD participants then received pramipexole treatment for 6 weeks. ResultsMDD participants had trend-level greater dopamine release capacity in the ventral striatum, as measured by percent change in baseline binding potential relative to nondisplaceable compartment (ΔBPND) (−34% vs. −30%; p = .072, d = 0.58) but no difference in D2/D3 receptor availability (BPND). Striatal and extrastriatal BPND and percent change in baseline BPND were not significantly associated with blood oxygen level–dependent response to reward prediction error in the ventral striatum, severity of depression and anhedonia, or antidepressant response to pramipexole (response rate = 72.7%). Conclusions[11C]-(+)-PHNO demonstrated high sensitivity to displacement by amphetamine-induced dopamine release, but dopamine release capacity and D2/D3 availability were not associated with ventral striatal activation to reward prediction error or clinical features, in this study powered to detect large effects. While a preponderance of indirect evidence implicates dopaminergic dysfunction in MDD, these findings suggest that presynaptic dopamine dysregulation may not be a feature of MDD or a prerequisite for treatment response to dopamine agonists.

Rats’ preferences in the suboptimal choice procedure: Evaluating the impact of reinforcement probability and conditioned inhibitors

Previous research has shown that pigeons and other birds display a strong and consistent preference for an alternative of reinforcement that presents stimuli that allow to discriminate whether a reinforcer will be delivered or not, even when its probability of reinforcement is lower than that of another alternative without those stimuli. In contrast, most of the studies performed with rats report that they show the opposite preference, choosing the alternative with higher probability of reinforcement. To explain these opposite preferences, it has been proposed that rats and pigeons have a differential sensitivity to the conditioned inhibition that emerges from the stimulus that predicts non-reinforcement: While it does not have an impact in pigeons, it strongly influences rats´ preferences. Alternatively, it was recently proposed that there is not a fundamental difference in the behavior of rats and pigeons, but that the procedure employed to evaluate each of these species has generated the difference; in particular, it was proposed that both species prefer the discriminative alternative when the discriminative stimuli have incentive salience. Two recent studies provide support for each of these hypotheses, so that the available evidence does not allow to distinguish between them. In the present report, we present three studies that systematically explore the influence of the procedural differences between the studies with discrepant results. The obtained results provide support for the following ideas: a) there is a fundamental difference between pigeons and rats in their choice behavior in the “suboptimal choice procedure”, b) considering the incentive salience of the discriminative stimuli does not resolve it, and c) rats’ optimality is a consistent phenomenon, which resists manipulations in reinforcement probabilities and the absence of conditioned inhibitors in the discriminative alternative.

Selective effects of 5-HT2C receptor modulation on performance of a novel valence-probe visual discrimination task and probabilistic reversal learning in mice

RationaleDysregulation of the serotonin (5-HT) system is a pathophysiological component in major depressive disorder (MDD), a condition closely associated with abnormal emotional responsivity to positive and negative feedback. However, the precise mechanism through which 5-HT tone biases feedback responsivity remains unclear. 5-HT2C receptors (5-HT2CRs) are closely linked with aspects of depressive symptomatology, including abnormalities in reinforcement processes and response to stress. Thus, we aimed to determine the impact of 5-HT2CR function on response to feedback in biased reinforcement learning.MethodsWe used two touchscreen assays designed to assess the impact of positive and negative feedback on probabilistic reinforcement in mice, including a novel valence-probe visual discrimination (VPVD) and a probabilistic reversal learning procedure (PRL). Systemic administration of a 5-HT2CR agonist and antagonist resulted in selective changes in the balance of feedback sensitivity bias on these tasks.ResultsSpecifically, on VPVD, SB 242084, the 5-HT2CR antagonist, impaired acquisition of a discrimination dependent on appropriate integration of positive and negative feedback. On PRL, SB 242084 at 1 mg/kg resulted in changes in behaviour consistent with reduced sensitivity to positive feedback. In contrast, WAY 163909, the 5-HT2CR agonist, resulted in changes associated with increased sensitivity to positive feedback and decreased sensitivity to negative feedback.ConclusionsThese results suggest that 5-HT2CRs tightly regulate feedback sensitivity bias in mice with consequent effects on learning and cognitive flexibility and specify a framework for the influence of 5-HT2CRs on sensitivity to reinforcement.

Age affects reinforcement learning through dopamine-based learning imbalance and high decision noise-not through Parkinsonian mechanisms.

Probabilistic reinforcement learning declines in healthy cognitive aging. While some findings suggest impairments are especially conspicuous in learning from rewards, resembling deficits in Parkinson's disease, others also show impairments in learning from punishments. To reconcile these findings, we tested 252 adults from 3 age groups on a probabilistic reinforcement learning task, analyzed trial-by-trial performance with a Q-reinforcement learning model, and correlated both fitted model parameters and behavior to polymorphisms in dopamine-related genes. Analyses revealed that learning from both positive and negative feedback declines with age but through different mechanisms: when learning from negative feedback, older adults were slower due to noisy decision-making; when learning from positive feedback, they tended to settle for a nonoptimal solution due to an imbalance in learning from positive and negative prediction errors. The imbalance was associated with polymorphisms in the DARPP-32 gene and appeared to arise from mechanisms different from those previously attributed to Parkinson's disease. Moreover, this imbalance predicted previous findings on aging using the Probabilistic Selection Task, which were misattributed to Parkinsonian mechanisms.

The paradoxical effect of low reward probabilities in suboptimal choice.

When offered a choice between 2 alternatives, animals sometimes prefer the option yielding less food. For instance, pigeons and starlings prefer an option that on 20% of the trials presents a stimulus always followed by food, and on the remaining 80% of the trials presents a stimulus never followed by food (the Informative Option), over an option that provides food on 50% of the trials regardless of the stimulus presented (the Noninformative Option). To explain this suboptimal behavior, it has been hypothesized that animals ignore (or do not engage with) the stimulus that is never followed by food in the Informative Option. To assess when pigeons attend to the stimulus usually not followed by food, we increased the probability of reinforcement, p, in the presence of that stimulus. Across 2 experiments, we found that the value of the Informative Option decreased with p. To account for the results, we added to the Reinforcement Rate Model (and also to the Hyperbolic Discounting Model) an engagement function, f(p), that specified the likelihood the animal attends to a stimulus followed by reward with probability p, and then derived the model predictions for 2 forms of f(p), a linear function, and an all-or-none threshold function. Both models predicted the observed findings with a linear engagement function: The higher the probability of reinforcement after a stimulus, the higher the probability of engaging the stimulus, and, surprisingly, the less the value of the option comprising the stimulus. (PsycINFO Database Record

Age and sex differences in behavioral flexibility, sensitivity to reward value, and risky decision-making.

Compared with adults, adolescent behavior is often characterized by reduced behavioral flexibility, increased sensitivity to reward, and increased likelihood to take risks. These traits, which have been hypothesized to confer heightened vulnerability to psychopathologies such as substance use disorders (SUDs), have been the focus of studies in laboratory animal models that seek to understand their neural underpinnings. However, rodent studies to date have typically used only males and have adopted standard methodological practices (e.g., weight loss inducing food restriction) that are likely to have a disparate impact on adolescents compared with adults. Here, we used adolescent and adult Sprague-Dawley rats of both sexes to study instrumental behavior tasks that assess behavioral flexibility (strategy shifting and reversal learning; Experiment 1), sensitivity to reward value (outcome devaluation; Experiment 2), and risky decision making (probability discounting; Experiment 3). In Experiment 1, we found that adolescents were faster to acquire reversal learning than adults but there were no differences in strategy shifting. In Experiments 2 and 3, adolescents and adults were equally sensitive to changes in reward value and exhibited similar reductions in preference for a large reward when reinforcement probability was decreased. However, adolescents responded more efficiently and earned reinforcers at a higher rate than their same-sex, adult counterparts. Together, these findings provide only limited support for the existence of an "adolescent-typical" phenotype in Sprague-Dawley rats and instead suggest that age differences in the expression of these behaviors may depend on conditions such as pubertal status and motivational state. (PsycINFO Database Record