Reward Magnitude Discrimination Research Articles

Loss of Sensitivity to Rewards by Dopamine Neurons May Underlie Age-Related Increased Probability Discounting.

Normative aging is known to affect how decisions are made in risky situations. Although important individual variability exists, on average, aging is accompanied by greater risk aversion. Here the behavioral and neural mechanisms of greater risk aversion were examined in young and old rats trained on an instrumental probability discounting task. Consistent with the literature, old rats showed greater discounting of reward value when the probability of obtaining rewards dropped below 100%. Behaviorally, reward magnitude discrimination was the same between young and old rats, and yet these same rats exhibited reduced sensitivity to positive, but not negative, choice outcomes. The latter behavioral result was congruent with additional findings that the aged ventral tegmental neurons (including dopamine cells) were less responsive to rewards when compared to the same cell types recorded from young animals. In sum, it appears that reduced responses of dopamine neurons to rewards contribute to aging-related changes in risky decisions.

Selective Functional Interaction Between the Lateral Habenula and Hippocampus During Different Tests of Response Flexibility

The lateral habenula (LHb) has been shown to play critical roles in a variety of appetitive tasks (e.g., spatial memory and object recognition) that require animals to flexibly respond to changing task conditions. These types of tasks are known to be dependent on hippocampus (HPC) and/or medial prefrontal cortex (mPFC), suggesting that the LHb contributes to the limbic memory circuit. Here we provide new evidence that the LHb and HPC play distinct but complimentary roles in tasks that require flexible responding to changing task conditions. Experiment 1 tested whether the LHb is needed for the performance of a HPC-dependent maze-based spatial delayed alternation task. The importance of interactions between the LHb and HPC to accomplish the same spatial delayed alternation task was tested in Experiment 2 where the LHb and HPC were disconnected both ipsilaterally and contralaterally. Experiment 3 tested LHb’s involvement in a standard behavioral economic task that requires flexible responding (maze-based delayed discounting), a task previously shown to rely on HPC. Results of Experiment 1, revealed that LHb inactivation impairs spatial delayed alternation during asymptotic performance but not during initial learning. Importantly, working memory did not appear to be affected as performance remained above chance levels both during initial learning and asymptotic testing. Experiment 2 showed that ipsilateral and contralateral disconnection of the LHb and HPC led to impaired performance on the spatial delayed alternation task. Impairments were not observed after unilateral inactivation of only one structure. Results of Experiment 3 were similar to our previous report of the effects of HPC inactivation: LHb inactivation impaired delayed discounting. All effects could not be accounted for by changes in reward magnitude discrimination, reward location per se, or sex of the animal. These findings, combined with other recent publications confirms and extends our working hypothesis that the LHb enables adaptive and flexible responding, particularly when established rules must be flexibly applied on a trial by trial basis. Since there are no known direct anatomical connections between LHb and HPC, future research is needed to understand how these structures communicate to enable flexible and rapid responding.

Nucleus accumbens core lesions induce sub-optimal choice and reduce sensitivity to magnitude and delay in impulsive choice tasks

The nucleus accumbens core (NAc) has long been recognized as an important contributor to the computation of reward value that is critical for impulsive choice behavior. Impulsive choice refers to choosing a smaller-sooner (SS) over a larger-later (LL) reward when the LL is more optimal in terms of the rate of reward delivery. Two experiments examined the role of the NAc in impulsive choice and its component processes of delay and magnitude processing. Experiment 1 delivered an impulsive choice task with manipulations of LL reward magnitude, followed by a reward magnitude discrimination task. Experiment 2 tested impulsive choice under manipulations of LL delay, followed by temporal bisection and progressive interval tasks. NAc lesions, in comparison to sham control lesions, produced suboptimal preferences that resulted in lower reward earning rates, and led to reduced sensitivity to magnitude and delay within the impulsive choice task. The secondary tasks revealed intact reward magnitude and delay discrimination abilities, but the lesion rats persisted in responding more as the progressive interval increased during the session. The results suggest that the NAc is most critical for demonstrating good sensitivity to magnitude and delay, and adjusting behavior accordingly. Ultimately, the NAc lesions induced suboptimal choice behavior rather than simply promoting impulsive choice, suggesting that an intact NAc is necessary for optimal decision making.

Acute and long-term effects of adolescent methylphenidate on decision-making and dopamine receptor mRNA expression in the orbitofrontal cortex

Though commonly used as a treatment for ADHD, the psychostimulant methylphenidate (MPH) is also misused and abused in adolescence in both clinical and general populations. Although MPH acts via pathways activated by other drugs of abuse, the short- and long-term effects of MPH on reward processing in learning and decision-making are not clearly understood. We examined the effect of adolescent MPH treatment on a battery of reward-directed behaviors both in adolescence during its administration and in adulthood after its discontinuation. We further measured whether MPH had lasting effects on dopamine receptor mRNA expression in orbitofrontal cortex (OFC) that may correspond with behavior. Long-Evans rats were injected with MPH (0, 1, 2.5, or 5mg/kg IP) twice daily from middle to late adolescence (PD38-57). During adolescence, the high dose of MPH reduced preference for large rewards in a Reward Magnitude Discrimination task, but did not affect preference for smaller-sooner rewards in a Delay Discounting task. In adulthood, after discontinuation of MPH, animals previously treated with the moderate dose of MPH showed improved acquisition, but not reversal, in a Reversal Learning task. MPH exposure did not increase preference for large-risky rewards in a Risk task in adulthood. We then quantified mRNA expression of D1, D2, and D3 receptors in the OFC using qPCR. MPH increased mRNA expression of dopamine D3 receptor subtype, but not D1 or D2. Overall, these results indicate that MPH has both immediate and lasting effects on reward-dependent learning and decisions, as well as dopaminergic function in rodents.

Lateral habenula integration of proactive and retroactive information mediates behavioral flexibility

The lateral habenula (LHb) is known to play an important role in signaling aversive or adverse events that have happened or are predicted by cues under Pavlovian conditions. In rodents, it is also required for behavioral flexibility when changes in reward outcomes signal that strategies should be changed. It is not known whether the LHb also controls appetitive behaviors when an animal is able to utilize external cues proactively to guide upcoming decisions. In order to test this, male Long–Evans rats were trained to switch between two arms of a figure eight maze based on the tone presented prior to the choice. Importantly, the tones were switched every three to six trials so rats were able establish a response pattern before being required to switch. This caused rats to rely on both proactive (tones) and retroactive information (reward feedback) to guide behavior. Inactivation of the LHb with the GABA agonists baclofen and muscimol impaired overall performance by increasing both errors when the tones are switched (switch errors) as well as on subsequent trials (perseverative errors) indicating that both proactive and retroactive information are utilized by the LHb to guide behavioral flexibility. Once a correct choice was made in a given block, LHb inactivated rats did not make more errors than controls. A control study revealed that the LHb is not required for tone or reward magnitude discrimination per se. These results demonstrate for the first time that the LHb contributes to behavioral flexibility through utilizing both proactive and retroactive information when performing appetitive tasks.

Basolateral amygdala lesions abolish mutual reward preferences in rats

In a recent study, we demonstrated that rats prefer mutual rewards in a Prosocial Choice Task. Here, employing the same task, we show that the integrity of basolateral amygdala was necessary for the expression of mutual reward preferences. Actor rats received bilateral excitotoxic (n=12) or sham lesions (n=10) targeting the basolateral amygdala and were subsequently tested in a Prosocial Choice Task where they could decide between rewarding (“Both Reward”) or not rewarding a partner rat (“Own Reward”), either choice yielding identical reward to the actors themselves. To manipulate the social context and control for secondary reinforcement sources, actor rats were paired with either a partner rat (partner condition) or with an inanimate rat toy (toy condition). Sham-operated animals revealed a significant preference for the Both-Reward-option in the partner condition, but not in the toy condition. Amygdala-lesioned animals exhibited significantly lower Both-Reward preferences than the sham group in the partner but not in the toy condition, suggesting that basolateral amygdala was required for the expression of mutual reward preferences. Critically, in a reward magnitude discrimination task in the same experimental setup, both sham-operated and amygdala-lesioned animals preferred large over small rewards, suggesting that amygdala lesion effects were restricted to decision making in social contexts, leaving self-oriented behavior unaffected.

Mechanisms of impulsive choice: III. The role of reward processes

Two experiments examined the relationship between reward processing and impulsive choice. In Experiment 1, rats chose between a smaller-sooner (SS) reward (1 pellet, 10s) and a larger-later (LL) reward (1, 2, and 4 pellets, 30s). The rats then experienced concurrent variable-interval 30-s schedules with variations in reward magnitude to evaluate reward magnitude discrimination. LL choice behavior positively correlated with reward magnitude discrimination. In Experiment 2, rats chose between an SS reward (1 pellet, 10s) and an LL reward (2 and 4 pellets, 30s). The rats then received either a reward intervention which consisted of concurrent fixed-ratio schedules associated with different magnitudes to improve their reward magnitude discrimination, or a control task. All rats then experienced a post-intervention impulsive choice task followed by a reward magnitude discrimination task to assess intervention efficacy. The rats that received the intervention exhibited increases in post-intervention LL choice behavior, and made more responses for larger-reward magnitudes in the reward magnitude discrimination task, suggesting that the intervention heightened sensitivities to reward magnitude. The results suggest that reward magnitude discrimination plays a key role in individual differences in impulsive choice, and could be a potential target for further intervention developments.

Contributions of the nucleus accumbens and its subregions to different aspects of risk-based decision making

The nucleus accumbens (NAc) has been implicated in mediating different forms of decision making in humans and animals. In the present study, we observed that inactivation of the rat NAc, via infusion of GABA agonists, reduced preference for a large/risky option and increased response latencies on a probabilistic discounting task. Discrete inactivations of the NAc shell and core revealed further differences between these regions in mediating choice and response latencies, respectively. The effect on choice was attributable to reduced win-stay performance (i.e., choosing risky after a being rewarded for a risky choice on a preceding trial). Moreover, NAc inactivation altered choice only when the large/risky option had greater long-term value, in terms of the amount of food that could be obtained over multiple trials relative to the small/certain option. Inactivation of the NAc or the shell subregion also slightly reduced preference for larger rewards on a reward magnitude discrimination. Thus, the NAc seems to play a small role in biasing choice toward larger rewards, but its contribution to behavior is amplified when delivery of these rewards is uncertain, helping to direct response selection toward more favorable outcomes.

Effect of instrumental response pretraining in classical-to-instrumental transfer of a differential reward magnitude discrimination

Amount of preliminary bar-press training (25 vs 125 discrete fixed-ratio trials) and amount of noncontingent pre training (0 vs 500 presentations of cues and associated reward magnitudes with the lever retracted) were manipulated factorially prior to contingent discrimination between five- and one-pellet cues. The 500-presentation groups demonstrated differential responding in start speed earlier in the contingent phase than the O-presentation groups, confirming previous findings of classical-to-instrumental transfer of the discrimination. The major effect of the lever pre training was a higher small-reward start speed in the 125-trial groups than in the 25-trial groups. Ratio speed data differed markedly from start speed data in showing large differences in favor of one-pellet ratio speeds early in training and very little evidence of five- over one-pellet discrimination late in training.

Three-level differential reward magnitude discrimination in rats

In two experiments conducted in a discrete-trial fixed-ratio leverpress situation, rats clearly discriminated among three stimuli associated with large reward (5 or 11 pellets), small reward (1 pellet), and nonreward. The course of acquisition and extinction was mapped for measures of start speed, ratio speed, and foodcup entrance behavior. Throughout acquisition and in the early portion of extinction there was an inverse relationship between leverpress speed and foodcup entrance responses. Resistance to extinction varied directly with amount of reward in within-S comparisons.

Extinction of differential reward magnitude discrimination in a discrete bar-pressing situation

Two groups of rats distinguished by the number of bar-presses required to complete a trial (FR 1 or FR 5) were given acquisition, extinction, and reacquisition in an instrumental discrimination involving differential reward magnitude (five pellets vs one pellet). The FR 5 group displayed better discrimination and extinguished faster than the FR 1 group. A “reversed magnitude-extinction effect” was found in that the usual between-groups finding of an inverse relation between resistance to extinction and acquisition reward magnitude failed to appear, ie., five-pellet start speeds of both groups remained above one-pellet speeds throughout extinction and there was a nonsignificant tendency for relative extinction rates of start speeds to be faster under one-pellet than five-pellet cues.

Competing responses in a differential magnitude of reward discrimination

The incidence of alley retracing was recorded in a differential reward magnitude discrimination. Five groups of rats received 500 mg reward to one stimulus (S1) and either 500, 250, 90, 37, or 0 mg reward to the other (S2). Following training, extinction trials were run to S1 only. The retracing patterns observed were discussed in the context of frustration theory.