Reward Size Research Articles

Midbrain Dopamine Neurons Signal Belief in Choice Accuracy during a Perceptual Decision

Central to the organization of behavior is the ability to predict the values of outcomes to guide choices. The accuracy of such predictions is honed by a teaching signal that indicates how incorrect a prediction was ("reward prediction error," RPE). In several reinforcement learning contexts, such as Pavlovian conditioning and decisions guided by reward history, this RPE signal is provided by midbrain dopamine neurons. In many situations, however, the stimuli predictive of outcomes are perceptually ambiguous. Perceptual uncertainty is known to influence choices, but it has been unclear whether or how dopamine neurons factor it into their teaching signal. To cope with uncertainty, we extended a reinforcement learning model with a belief state about the perceptually ambiguous stimulus; this model generates an estimate of the probability of choice correctness, termed decision confidence. We show that dopamine responses in monkeys performing a perceptually ambiguous decision task comply with the model's predictions. Consequently, dopamine responses did not simply reflect a stimulus' average expected reward value but were predictive of the trial-to-trial fluctuations in perceptual accuracy. These confidence-dependent dopamine responses emerged prior to monkeys' choice initiation, raising the possibility that dopamine impacts impending decisions, in addition to encoding a post-decision teaching signal. Finally, by manipulating reward size, we found that dopamine neurons reflect both the upcoming reward size and the confidence in achieving it. Together, our results show that dopamine responses convey teaching signals that are also appropriate for perceptual decisions.

Brain Stimulation Reward Supports More Consistent and Accurate Rodent Decision-Making than Food Reward.

Animal models of decision-making rely on an animal’s motivation to decide and its ability to detect differences among various alternatives. Food reinforcement, although commonly used, is associated with problematic confounds, especially satiety. Here, we examined the use of brain stimulation reward (BSR) as an alternative reinforcer in rodent models of decision-making and compared it with the effectiveness of sugar pellets. The discriminability of various BSR frequencies was compared to differing numbers of sugar pellets in separate free-choice tasks. We found that BSR was more discriminable and motivated greater task engagement and more consistent preference for the larger reward. We then investigated whether rats prefer BSR of varying frequencies over sugar pellets. We found that animals showed either a clear preference for sugar reward or no preference between reward modalities, depending on the frequency of the BSR alternative and the size of the sugar reward. Overall, these results suggest that BSR is an effective reinforcer in rodent decision-making tasks, removing food-related confounds and resulting in more accurate, consistent, and reliable metrics of choice.

In monkeys making value-based decisions, amygdala neurons are sensitive to cue value as distinct from cue salience.

Neurons in the lateral intraparietal (LIP) area of macaque monkey parietal cortex respond to cues predicting rewards and penalties of variable size in a manner that depends on the motivational salience of the predicted outcome (strong for both large reward and large penalty) rather than on its value (positive for large reward and negative for large penalty). This finding suggests that LIP mediates the capture of attention by salient events and does not encode value in the service of value-based decision making. It leaves open the question whether neurons elsewhere in the brain encode value in the identical task. To resolve this issue, we recorded neuronal activity in the amygdala in the context of the task employed in the LIP study. We found that responses to reward-predicting cues were similar between areas, with the majority of reward-sensitive neurons responding more strongly to cues that predicted large reward than to those that predicted small reward. Responses to penalty-predicting cues were, however, markedly different. In the amygdala, unlike LIP, few neurons were sensitive to penalty size, few penalty-sensitive neurons favored large over small penalty, and the dependence of firing rate on penalty size was negatively correlated with its dependence on reward size. These results indicate that amygdala neurons encoded cue value under circumstances in which LIP neurons exhibited sensitivity to motivational salience. However, the representation of negative value, as reflected in sensitivity to penalty size, was weaker than the representation of positive value, as reflected in sensitivity to reward size.NEW & NOTEWORTHY This is the first study to characterize amygdala neuronal responses to cues predicting rewards and penalties of variable size in monkeys making value-based choices. Manipulating reward and penalty size allowed distinguishing activity dependent on motivational salience from activity dependent on value. This approach revealed in a previous study that neurons of the lateral intraparietal (LIP) area encode motivational salience. Here, it reveals that amygdala neurons encode value. The results establish a sharp functional distinction between the two areas.

Delay discounting in mild cognitive impairment

ABSTRACTIntroduction: Patients with mild cognitive impairment (MCI) may make suboptimal decisions particularly in complex situations, and this could be due to temporal discounting, the tendency to prefer immediate rewards over delayed but larger rewards. The present study proposes to evaluate intertemporal preferences in MCI patients as compared to healthy controls. Method: Fifty-five patients with MCI and 57 healthy controls underwent neuropsychological evaluation and a delay discounting questionnaire, which evaluates three parameters: hyperbolic discounting (k), the percentage of choices for delayed and later rewards (%LL), and response consistency (Acc). Results: No significant differences were found in the delay discounting questionnaire between MCI patients and controls for the three reward sizes considered, small, medium, and large, using both k and %LL parameters. There were also no differences in the response consistency, Acc, between the two groups. Conclusions: Patients with MCI perform similarly to healthy controls in a delay discounting task. Memory deficits do not notably affect intertemporal preferences.

Prior Cocaine Experience Impairs Normal Phasic Dopamine Signals of Reward Value in Accumbens Shell.

Dopamine signals have repeatedly been linked to associative learning and motivational processes. However, there is considerably less agreement on a role for dopamine in reward processing, and therefore whether neuroplastic changes in dopamine function following chronic exposure to drugs of abuse such as cocaine may impair appropriate valuation of rewarding stimuli. To quantify this, we voltammetrically measured real-time dopamine release in the nucleus accumbens (NAc) core or shell while rats received unsignaled deliveries of either a small (1 pellet) or large (2 pellets) reward. In drug-naive controls, core dopamine signals did not discriminate between reward size at any point, while in the shell dopamine encoded magnitude differences only in a slower postpeak period. Despite this lack of discrimination between rewards by the peak DA response, controls easily discriminated between reward options in a subsequent choice task. In contrast, phasic dopamine reward signals were strongly altered by cocaine experience; core dopamine decreased peak response but increased discrimination between reward magnitudes while shell lost phasic responses to reward receipt altogether. Notably, animals with cocaine-associated alterations in dopamine signals for reward magnitude failed to subsequently discriminate between reward options. These findings suggest that cocaine self-administration alters the ability for dopamine signals to appropriately assign value to rewards and thus may in part contribute to later deficits in behaviors that depend on appropriate outcome valuation.

The converse to Bergmann's rule in bumblebees, a phylogenetic approach.

Two patterns commonly emerge when animal body size is analyzed as a function of latitudinal distribution. First, body size increases with latitude, a temperature effect known as Bergmann's rule, and second, the converse to Bergmann's rule, a pattern in which body size decreases with latitude. However, other geographic patterns can emerge when the mechanisms that generate Bergmann's and the converse to Bergmann's clines operate together. Here, we use phylogenetic comparative analysis in order to control for phylogenetic inertia, and we show that bumblebees exhibit the converse to Bergmann's rule. Bumblebee taxa are distributed worldwide in temperate and tropical regions. The largest species are found in places with high water availability during the driest time of the year. Nonetheless, large body size is constrained by extreme temperatures. Bumblebees’ body size could be related to a higher extent to the size of food rewards to be harvested than to the energetic advantages of thermoregulation. Moreover, we found that the body size of eusocial and cuckoo species responded in the same way to environmental variables, suggesting that they have not diverged due to different selective pressures.

Acute ethanol effects on neural encoding of reward size and delay in the nucleus accumbens.

Acute ethanol administration can cause impulsivity, resulting in increased preference for immediately available rewards over delayed but more valuable alternatives. The manner in which reward size and delay are represented in neural firing is not fully understood, and very little is known about ethanol effects on this encoding. To address this issue, we used in vivo electrophysiology to characterize neural firing in the core of the nucleus accumbens (NAcc) in rats responding for rewards that varied in size or delay after vehicle or ethanol administration. The NAcc is a central element in the circuit that governs decision-making and importantly, promotes choice of delayed rewards. We found that NAcc firing in response to reward-predictive cues encoded anticipated reward value after vehicle administration, but ethanol administration disrupted this encoding, resulting in a loss of discrimination between immediate and delayed rewards in cue-evoked neural responses. In addition, NAcc firing occurring at the time of the operant response (lever pressing) was inversely correlated with behavioral response latency, such that increased firing rates were associated with decreased latencies to lever press. Ethanol administration selectively attenuated this lever press-evoked firing when delayed but not immediate rewards were expected. These effects on neural firing were accompanied by increased behavioral latencies to respond for delayed rewards. Our results suggest that ethanol effects on NAcc cue- and lever press-evoked encoding may contribute to ethanol-induced impulsivity.

In-car usage-based insurance feedback strategies. A comparative driving simulator study

Usage-Based Insurances (UBI) enable policyholders to actively reduce the impact of vehicle insurance costs by adopting a safer and more eco-friendly driving style. UBI is especially relevant for younger drivers, who are a high-risk population. The effectiveness of UBI should be enhanced by providing in-car feedback optimised for individual drivers. Thirty young novice drivers were therefore invited to complete six experimental drives with an in-car interface that provided real-time information on rewards gained, their driving behaviour and the speed limit. Reward size was either displayed directly in euro, indirectly as a relatively large amount of credits, or as a percentage of the maximum available bonus. Also, interfaces were investigated that provided partial information to reduce the potential for driver distraction. Compared to a control no-UBI condition, behaviour improved similarly across interfaces, suggesting that interface personalisation after an initial familiarisation period could be feasible without compromising feedback effectiveness.Practitioner Summary: User experiences and effects on driving behaviour of six in-car interfaces were compared. The interface provided information on driving behaviour and rewards in a UBI setting. Results suggest that some personalisation of interfaces may be an option after an initial familiarisation period as driving behaviour improved similarly across interfaces.

The subjective value of probabilistic outcomes: Impact of reward magnitude on choice with uncertain rewards in rats

Interest is rising for animal modelling of Gambling disorder (GD), which is rapidly emerging as a mental health concern. In the present study, we assessed gambling proneness in male Wistar-Han rats using the “Probabilistic Delivery Task” (PDT). This operant protocol is based on choice between either certain, small amounts of food (SS) or larger amounts of food (LLL) delivered (or not) depending on a given (and progressively decreasing) probability. Here, we manipulated the ratio between large and small reward size to assess the impact of different magnitudes on rats’ performance. Specifically, we drew a comparison between threefold (2 vs 6 pellets) and fivefold (1 vs 5 pellets) sizes. As a consequence, the “indifferent point” (IP, at which either choice is mathematically equivalent in terms of total foraging) was at 33% and 20% probability of delivery, respectively. Animals tested with the sharper contrast (i.e. fivefold ratio) exhibited sustained preference for LLL far beyond the IP, despite high uncertainty and low payoff, which rendered LLL a sub-optimal option. By contrast, animals facing a slighter contrast (i.e. threefold ratio) were increasingly disturbed by progressive rarefaction of rewards, as expressed by enhanced inadequate nose-poking: this was in accordance with their prompt shift in preference to SS, already shown around the IP. In conclusion, a five-folded LLL-to-SS ratio was not only more attractive, but also less frustrating than a three-folded one. Thus, a profile of gambling proneness in the PDT is more effectively induced by marked contrast between alternative options.

A Theory of Whistleblower Rewards

AbstractTo enforce the law, the government must learn about violations of the law. One way of obtaining such information is by employing police officers and investigators. An alternative way is by rewarding whistleblowers. In this paper I consider two basic questions relating to whistleblower rewards. First, what is the optimal size of whistleblower rewards? Second, how should we choose between employing police officers and rewarding whistleblowers? I develop a model that highlights two features of the whistleblowing context: whistleblowers bear a personal cost, and a reward may encourage false reports. I find that there is a nonmonotonic relationship between the personal cost to whistleblowers and the optimal reward, and between the risk of a false report and the optimal reward. Furthermore, offering a whistleblower reward dominates the employment of police officers and investigators when the risk of a false report is small.

Chemogenetic disconnection of monkey orbitofrontal and rhinal cortex reversibly disrupts reward value.

To study how the interaction between orbitofrontal (OFC) and rhinal (Rh) cortices influences the judgment of reward size, we reversibly disconnected these regions using the hM4Di-DREADD (Designer Receptor Exclusively Activated by Designer Drug). Repeated inactivation reduced sensitivity to differences in reward size in two monkeys. Results suggest that retrieval of relative stimulus values from memory appears to depend on interaction between Rh and OFC.

Commentary: Noradrenaline and Dopamine Neurons in the Reward/Effort Trade-off: A Direct Electrophysiological Comparison in Behaving Monkeys.

Commentary: Noradrenaline and Dopamine Neurons in the Reward/Effort Trade-off: A Direct Electrophysiological Comparison in Behaving Monkeys.

Hierarchical Bayesian estimation and hypothesis testing for delay discounting tasks.

A state-of-the-art data analysis procedure is presented to conduct hierarchical Bayesian inference and hypothesis testing on delay discounting data. The delay discounting task is a key experimental paradigm used across a wide range of disciplines from economics, cognitive science, and neuroscience, all of which seek to understand how humans or animals trade off the immediacy verses the magnitude of a reward. Bayesian estimation allows rich inferences to be drawn, along with measures of confidence, based upon limited and noisy behavioural data. Hierarchical modelling allows more precise inferences to be made, thus using sometimes expensive or difficult to obtain data in the most efficient way. The proposed probabilistic generative model describes how participants compare the present subjective value of reward choices on a trial-to-trial basis, estimates participant- and group-level parameters. We infer discount rate as a function of reward size, allowing the magnitude effect to be measured. Demonstrations are provided to show how this analysis approach can aid hypothesis testing. The analysis is demonstrated on data from the popular 27-item monetary choice questionnaire (Kirby, Psychonomic Bulletin & Review, 16(3), 457-462 2009), but will accept data from a range of protocols, including adaptive procedures. The software is made freely available to researchers.

The Effects of Flower, Floral Display, and Reward Sizes on Bumblebee Foraging Behavior When Pollen Is the Reward and Plants Are Dichogamous

Premise of research. The size of floral display, flower, and reward can influence pollinator foraging behavior. However, the importance of correlations between floral traits and rewards in affecting pollinator preferences is unclear. Moreover, in monoecious plants and plants with strong within-flower dichogamy, some flowers do not offer a pollen reward, and pollinator attraction may depend on the number of rewarding flowers rather than floral display size. We examined these questions in the Rocky Mountain columbine, where bumblebees can forage only for pollen produced in male-phase flowers.Methodology. We used dual-choice assays and experimental arrays to determine whether a correlation between pollen reward and flower size was necessary before bees could distinguish between flowers with different pollen rewards or sizes. The impact of floral display size on bee foraging was first examined using inflorescences where all flowers provided a reward, either small or large. We then varied either floral display size or number of rewarding flowers while keeping the other variable constant.Pivotal results. Bumblebees preferred flowers with a larger pollen reward or larger flowers only when the size of the pollen reward was correlated with flower size. Bees selected larger floral displays when all flowers presented a similar amount of pollen but preferred inflorescences with more pollen-rewarding flowers when only a proportion of the flowers offered a pollen reward.Conclusions. Bees could identify whether a flower offered pollen and could quantify the number of pollen-rewarding flowers. Correlations between floral traits and pollen reward affected pollinator preferences and may facilitate selection of floral traits by pollinators. Pollinators preferred inflorescences with more pollen-producing flowers rather than larger floral displays, which suggests that selection of floral display size by pollinators may differ between plants with different mating systems.

Reward Size Informs Repeat-Switch Decisions and Strongly Modulates the Activity of Neurons in Parietal Cortex.

Behavior is guided by previous experience. Good, positive outcomes drive a repetition of a previous behavior or choice, whereas poor or bad outcomes lead to an avoidance. How these basic drives are implemented by the brain has been of primary interest to psychology and neuroscience. We engaged animals in a choice task in which the size of a reward outcome strongly governed the animals' subsequent decision whether to repeat or switch the previous choice. We recorded the discharge activity of neurons implicated in reward-based choice in 2 regions of parietal cortex. We found that the tendency to retain previous choice following a large (small) reward was paralleled by a marked decrease (increase) in the activity of parietal neurons. This neural effect is independent of, and of sign opposite to, value-based modulations reported in parietal cortex previously. This effect shares the same basic properties with signals previously reported in the limbic system that detect the size of the recently obtained reward to mediate proper repeat-switch decisions. We conclude that the size of the obtained reward is a decision variable that guides the decision between retaining a choice or switching, and neurons in parietal cortex strongly respond to this novel decision variable.

Commentary: "Neural signatures of intransitive preferences".

Commentary: "Neural signatures of intransitive preferences".

Trading up: chimpanzees (Pan troglodytes) show self-control through their exchange behavior.

Self-control is defined as the ability or capacity to obtain an objectively more valuable outcome rather than an objectively less valuable outcome though tolerating a longer delay or a greater effort requirement (or both) in obtaining that more valuable outcome. A number of tests have been devised to assess self-control in non-human animals, including exchange tasks. In this study, three chimpanzees (Pan troglodytes) participated in a delay of gratification task that required food exchange as the behavioral response that reflected self-control. The chimpanzees were offered opportunities to inhibit eating and instead exchange a currently possessed food item for a different (and sometimes better) item, often needing to exchange several food items before obtaining the highest valued reward. We manipulated reward type, reward size, reward visibility, delay to exchange, and location of the highest valued reward in the sequence of exchange events to compare performance within the same individuals. The chimpanzees successfully traded until obtaining the best item in most cases, although there were individual differences among participants in some variations of the test. These results support the idea that self-control is robust in chimpanzees even in contexts in which they perhaps anticipate future rewards and sustain delay of gratification until they can obtain the ultimately most valuable item.

Interactions between target location and reward size modulate the rate of microsaccades in monkeys.

We have studied how rewards modulate the occurrence of microsaccades by manipulating the size of an expected reward and the location of the cue that sets the expectations for future reward. We found an interaction between the size of the reward and the location of the cue. When monkeys fixated on a cue that signaled the size of future reward, the frequency of microsaccades was higher if the monkey expected a large vs. a small reward. When the cue was presented at a site in the visual field that was remote from the position of fixation, reward size had the opposite effect: the frequency of microsaccades was lower when the monkey was expecting a large reward. The strength of pursuit initiation also was affected by reward size and by the presence of microsaccades just before the onset of target motion. The gain of pursuit initiation increased with reward size and decreased when microsaccades occurred just before or after the onset of target motion. The effect of the reward size on pursuit initiation was much larger than any indirect effects reward might cause through modulation of the rate of microsaccades. We found only a weak relationship between microsaccade direction and the location of the exogenous cue relative to fixation position, even in experiments where the location of the cue indicated the direction of target motion. Our results indicate that the expectation of reward is a powerful modulator of the occurrence of microsaccades, perhaps through attentional mechanisms.

The impacts of elicitation mechanism and reward size on estimated rates of time preference

We run experiments with real monetary rewards ranging from $10 to $500 to estimate rates of time preference and test for hyperbolic discounting. Individuals become more patient with increasing reward sizes, which is consistent with a magnitude effect. This magnitude effect is robust across specifications including a nonparametric analysis and structural maximum likelihood estimation. Subjects are divided between two different elicitation mechanisms (one a matching task and one a choice task) that should both theoretically provide an incentive for participants to reveal their true time preferences. We find some evidence of differences between the rates from the matching and choice tasks but these differences disappear when appropriately modeling the behavioral noise. We uncover little to no evidence of present-biased time preferences.

Review of Medicare, Medicaid, and Commercial Quality of Care Measures: Considerations for Assessing Accountable Care Organizations.

Accountable care organizations (ACOs) have proliferated under the Affordable Care Act (ACA). If ACOs are to improve health care quality and lower costs, quality measures will be increasingly important in determining if provider consolidations associated with the development of ACOs are achieving their intended purpose. This article assesses quality measurement across public and private sectors. We reviewed available quality measures for a subset of programs in six organizations and assessed the number and domain of measures (structure, process, outcomes, and patient experience). Two-thirds of all quality measures were categorized as process measures. Outcome measures made up nearly 20 percent of measures. Patient experience and structure measures made up approximately 8 percent and 7 percent, respectively. We propose further improvements to quality measurement initiatives. For example, programs that reward providers should consider reward size and distribution within the organization. Quality improvement initiatives should consider what encourages provider buy-in and participation and the effects on populations with disproportionate health care needs. As the focus of quality initiatives may change from year to year, measures should be periodically revisited to ensure continued improvement and sustainability. Finally, we suggest quality measures that regulators could use prior to ACO formation or in the year or two following formation.