Reward Magnitude Research Articles

Reward Contrast Effects on Impulsive Choice and Timing in Rats.

Despite considerable interest in impulsive choice as a predictor of a variety of maladaptive behaviors, the mechanisms that drive choice behavior are still poorly understood. The present study sought to examine the influence of one understudied variable, reward magnitude contrast, on choice and timing behavior as changes in magnitude commonly occur within choice procedures. In addition, assessments of indirect effects on choice behavior through magnitude-timing interactions were assessed by measuring timing within the choice task. Rats were exposed to choice procedures composed of different pairs of magnitudes of rewards for either the smaller-sooner (SS) or larger-later (LL) option. In Phase 2, the magnitude of reward either increased or decreased by 1 pellet in different groups (LL increase = 1v1→1v2; SS decrease = 2v2 → 1v2; SS increase = 1v2 → 2v2), followed by a return to baseline in Phase 3. Choice behavior was affected by the initial magnitudes experienced in the task, demonstrating a strong anchor effect. The nature of the change in magnitude affected choice behavior as well. Timing behavior was also affected by the reward contrast manipulation albeit to a lesser degree and the timing and choice effects were correlated. The results suggest that models of choice behavior should incorporate reinforcement history, reward contrast elements, and magnitude-timing interactions, but that direct effects of reward contrast on choice should be given more weight than the indirect reward-timing interactions. A better understanding of the factors that contribute to choice behavior could supply key insights into this important individual differences variable.

The effects of reward magnitude on reward processing: An averaged and single trial event-related potential study

From a neurobiological and motivational perspective, the feedback-related negativity (FRN) and reward positivity (RewP) event-related potential (ERP) components should increase with reward magnitude (reward associated with valence (success/failure) feedback). To test this hypothesis, we recorded participants’ electroencephalograms while presenting them with potential monetary rewards ($0.00–$4.96) pre-trial for each trial of a reaction time task and presenting them with valence feedback post-trial. Averaged ERPs time-locked to valence feedback were extracted, and results revealed a valence by magnitude interaction for neural activity in the FRN/RewP time window. This interaction was driven by magnitude affecting RewP, but not FRN, amplitude. Moreover, single trial ERP analyses revealed a reliable correlation between magnitude and RewP, but not FRN, amplitude. Finally, P3b and late positive potential (LPP) amplitudes were affected by magnitude. Results partly support the neurobiological (dopamine) account of the FRN/RewP and suggest motivation affects feedback processing, as indicated by multiple ERP components.

How do magnitude and frequency of monetary reward guide visual search?

How does reward guide spatial attention during visual search? In the present study, we examine whether and how two types of reward information-magnitude and frequency-guide search behavior. Observers were asked to find a target among distractors in a search display to earn points. We manipulated multiple levels of value across the search display quadrants in two ways: For reward magnitude, targets appeared equally often in each quadrant, and the value of each quadrant was determined by the average points earned per target; for reward frequency, we varied how often the target appeared in each quadrant but held the average points earned per target constant across the quadrants. In Experiment 1, we found that observers were highly sensitive to the reward frequency information, and prioritized their search accordingly, whereas we did not find much prioritization based on magnitude information. In Experiment 2, we found that magnitude information for a nonspatial feature (color) could bias search performance, showing that the relative insensitivity to magnitude information during visual search is not generalized across all types of information. In Experiment 3, we replicated the negligible use of spatial magnitude information even when we used limited-exposure displays to incentivize the expression of learning. In Experiment 4, we found participants used the spatial magnitude information during a modified choice task-but again not during search. Taken together, these findings suggest that the visual search apparatus does not equally exploit all potential sources of spatial value information; instead, it favors spatial reward frequency information over spatial reward magnitude information.

The Effect of Forest Management Strategy on Carbon Storage and Revenue in Western Washington: A Probabilistic Simulation of Tradeoffs

The objectives of this study are to understand tradeoffs between forest carbon and timber values, and evaluate the impact of uncertainty in improved forest management (IFM) carbon offset projects to improve forest management decisions. The study uses probabilistic simulation of uncertainty in financial risk for three management scenarios (clearcutting in 45- and 65-year rotations and no harvest) under three carbon price schemes (historic voluntary market prices, cap and trade, and carbon prices set to equal net present value (NPV) from timber-oriented management). Uncertainty is modeled for value and amount of carbon credits and wood products, the accuracy of forest growth model forecasts, and four other variables relevant to American Carbon Registry methodology. Calculations use forest inventory data from a 1,740 ha forest in western Washington State, using the Forest Vegetation Simulator (FVS) growth model. Sensitivity analysis shows that FVS model uncertainty contributes more than 70% to overall NPV variance, followed in importance by variability in inventory sample (3-14%), and short-term prices for timber products (8%), while variability in carbon credit price has little influence (1.1%). At regional average land-holding costs, a no-harvest management scenario would become revenue-positive at a carbon credit break-point price of $14.17/Mg carbon dioxide equivalent (CO2 e). IFM carbon projects are associated with a greater chance of both large payouts and large losses to landowners. These results inform policymakers and forest owners of the carbon credit price necessary for IFM approaches to equal or better the business-as-usual strategy, while highlighting the magnitude of financial risk and reward through probabilistic simulation.

The scarcity heuristic impacts reward processing within the medial-frontal cortex.

Objects that are rare are often perceived to be inherently more valuable than objects that are abundant - a bias brought about in part by the scarcity heuristic. In the present study, we sought to test whether perception of rarity impacted reward evaluation within the human medial-frontal cortex. Here, participants played a gambling game in which they flipped rare and abundant 'cards' on a computer screen to win financial rewards while electroencephalographic data were recorded. Unbeknownst to participants, reward outcome and frequency was random and equivalent for both rare and abundant cards; thus, only a perception of scarcity was true. Analysis of the electroencephalographic data indicated that the P300 component of the event-related brain potential differed in amplitude for wins and losses following the selection of rare cards, but not following the selection of abundant cards. Importantly, then, we found that the perception of card rarity impacted reward processing even though reward feedback was independent of and subsequent to card selection. Our data indicate a top-down influence of the scarcity heuristic on reward evaluation, and specifically the processing of reward magnitude, within the human medial-frontal cortex.

Transcranial random-noise stimulation of visual cortex potentiates value-driven attentional capture.

Reward feedback following visual search causes the visual characteristics of targets to become salient and attention-drawing, but little is known about the mechanisms underlying this value-driven capture effect. Here, we use transcranial random noise stimulation (tRNS) to demonstrate that such reward potentiation involves induced plasticity in visual cortex. Human participants completed a feature-search reward-learning task involving the selection of a red or green colored target presented among distractors of various color. Each correct trial garnered reward and the magnitude of reward was determined by the color of the target. Three groups completed this task: two groups received tRNS over either occipital or frontal cortex, and the third group received sham stimulation as a control. In a subsequent test phase of the experiment participants searched for a unique shape presented among colored distractors. During the test phase, no tRNS was applied and no reward was available. However, in some trials a single distractor had color matching that associated with reward during training. Search for the target was impacted by the presence of such reward-associated distractors in the occipital stimulation group, demonstrating that plasticity in visual cortex contributes to value-driven attentional capture.

The subjective value of probabilistic outcomes: Impact of reward magnitude on choice with uncertain rewards in rats

Interest is rising for animal modelling of Gambling disorder (GD), which is rapidly emerging as a mental health concern. In the present study, we assessed gambling proneness in male Wistar-Han rats using the “Probabilistic Delivery Task” (PDT). This operant protocol is based on choice between either certain, small amounts of food (SS) or larger amounts of food (LLL) delivered (or not) depending on a given (and progressively decreasing) probability. Here, we manipulated the ratio between large and small reward size to assess the impact of different magnitudes on rats’ performance. Specifically, we drew a comparison between threefold (2 vs 6 pellets) and fivefold (1 vs 5 pellets) sizes. As a consequence, the “indifferent point” (IP, at which either choice is mathematically equivalent in terms of total foraging) was at 33% and 20% probability of delivery, respectively. Animals tested with the sharper contrast (i.e. fivefold ratio) exhibited sustained preference for LLL far beyond the IP, despite high uncertainty and low payoff, which rendered LLL a sub-optimal option. By contrast, animals facing a slighter contrast (i.e. threefold ratio) were increasingly disturbed by progressive rarefaction of rewards, as expressed by enhanced inadequate nose-poking: this was in accordance with their prompt shift in preference to SS, already shown around the IP. In conclusion, a five-folded LLL-to-SS ratio was not only more attractive, but also less frustrating than a three-folded one. Thus, a profile of gambling proneness in the PDT is more effectively induced by marked contrast between alternative options.

The Nature and Extent of Post-Reward Crowding-Out: The Effort-Balancingg Account

Although incentives can be a powerful motivator of behavior, research on intrinsic motivation has suggested that rewards can crowd-out task interest, reducing engagement when rewards end. This research has resulted in widespread skepticism among practitioners and academics alike about using incentives in interventions. However, recent field studies examining the long-term effects of temporary incentives have not found such effects. We propose a new Effort-Balancing account, which suggests that post-reward crowding-out often represents the need for a ‘break’ after investing effort, rather than a change in interpretation of the task or beliefs about own preferences. As a result, post-reward crowding-out is relatively momentary and consistent with longer-term neutral or positive spillover effects of temporary incentives. We test a series of novel predictions, including that momentary crowding-out will be reduced when efforts are more balanced by rewards (e.g., higher reward magnitude or a less effortful activity) or when people are given a break. The implications of momentary crowding-out for long-term behavior and design of incentive programs are discussed.

Lateral habenula integration of proactive and retroactive information mediates behavioral flexibility

The lateral habenula (LHb) is known to play an important role in signaling aversive or adverse events that have happened or are predicted by cues under Pavlovian conditions. In rodents, it is also required for behavioral flexibility when changes in reward outcomes signal that strategies should be changed. It is not known whether the LHb also controls appetitive behaviors when an animal is able to utilize external cues proactively to guide upcoming decisions. In order to test this, male Long–Evans rats were trained to switch between two arms of a figure eight maze based on the tone presented prior to the choice. Importantly, the tones were switched every three to six trials so rats were able establish a response pattern before being required to switch. This caused rats to rely on both proactive (tones) and retroactive information (reward feedback) to guide behavior. Inactivation of the LHb with the GABA agonists baclofen and muscimol impaired overall performance by increasing both errors when the tones are switched (switch errors) as well as on subsequent trials (perseverative errors) indicating that both proactive and retroactive information are utilized by the LHb to guide behavioral flexibility. Once a correct choice was made in a given block, LHb inactivated rats did not make more errors than controls. A control study revealed that the LHb is not required for tone or reward magnitude discrimination per se. These results demonstrate for the first time that the LHb contributes to behavioral flexibility through utilizing both proactive and retroactive information when performing appetitive tasks.

How sequential changes in reward magnitude modulate cognitive flexibility: Evidence from voluntary task switching.

There is much evidence that the prospect of reward modulates cognitive control in terms of more stable behavior. Increases in expected reward magnitude, however, have been suggested to increase flexible behavior as evidenced by reduced switch costs. In a series of experiments, the authors provide evidence that this increased cognitive flexibility following increases in reward magnitude also promotes deliberate task switching. A modified task switching paradigm with forced- and free-choice trials and varying reward prospects was used. In Experiments 1-3 the prospect of a reward increase as compared to unchanged high reward increased voluntary switching rate (VSR). Experiment 4 showed that the prospect of a reward decrease did not alter VSR as compared to unchanged low reward. Experiment 5 used a standard voluntary task switching procedure and confirmed VSR effects found in Experiments 1-4. These findings are strong evidence for a mechanism that biases the cognitive system either toward stability or flexibility depending on changing reward expectation. Results are discussed within the framework of the adaptive gain theory.

Inefficient effort allocation and negative symptoms in individuals with schizophrenia

Negative symptoms like avolition and anhedonia are thought to involve difficulties with reward processing and motivation. The current study aimed to replicate and extend prior findings that individuals with schizophrenia display reduced willingness to expend effort for rewards and that such reduced effort is associated with negative symptoms, poor functioning, and cognitive impairment. The present study compared the effortful decision making of individuals with schizophrenia (n=48) and healthy controls (n=27) on the Effort Expenditure for Rewards Task (EEfRT). Individuals with schizophrenia chose a smaller proportion of hard tasks than healthy controls across all probability and reward levels with the exception of trials with a 12% probability and low or medium reward magnitude wherein both groups chose similarly few hard tasks. Contrary to expectations, in individuals with schizophrenia, greater negative symptoms were associated with making more effortful choices. Effortful decision making was unrelated to positive symptoms, depression, cognition, and functioning in individuals with schizophrenia. Our results are consistent with prior findings that revealed a pattern of inefficient decision making in individuals with schizophrenia relative to healthy controls. However the results did not support the hypothesized association of negative symptoms and reduced effort in schizophrenia and highlight prior inconsistencies in this literature. Future research is needed to understand what factors may be related to diminished effortful decision making in schizophrenia and the clinical significance of such performance deficits.

The better, the bigger: The effect of graded positive performance feedback on the reward positivity

In this study on skill acquisition in a computerized throwing task, we examined the effect of graded correct-related performance feedback on the reward positivity of the event-related brain potential (ERP). Theories of reinforcement learning predict effects of reward magnitude and expectancy on the reward prediction error. The later is supposed to be reflected in reward positivity, a fronto-central ERP component. A sample of 68 participants learned to throw at a beamer-projected target disk while performance accuracy, displayed as the place of impact of the projectile on the target, served as graded feedback. Effects of performance accuracy in successful trials, hit frequency, and preceding trial performance on reward positivity were analyzed simultaneously on a trial-by-trial basis by means of linear mixed models. In accord with previous findings, reward positivity increased with feedback about more accurate performance. This relationship was not linear, but cubic, with larger impact of feedback towards the end of the accuracy distribution. In line with being a measure of expectancy, the reward positivity decreased with increasing hit frequency and was larger after unsuccessful trials. The effect of hit frequency was more pronounced following successful trials. These results indicate a fast trial-by-trial adaptation of expectation. The results confirm predictions of reinforcement learning theory and extend previous findings on reward magnitude to the area of complex, goal directed skill acquisition.

Impulsivity and risky decision making among taxi drivers in Hong Kong: An event-related potential study

Taxi drivers play an important role in providing safe and professional public transport services. However, they tend to be more involved than other professional driver groups in accidents caused by deliberate recklessness. This study used an event-related potential (ERP) experiment to examine risk-taking behavior arising from impulsivity by comparing the underlying neural processes of taxi drivers with and without traffic offence records in Hong Kong. A sample of 15 traffic offenders and 15 nonoffenders, matched by sociodemographic characteristics, was recruited. The results show that the offender group demonstrated significantly less negative-going (less negative) feedback-related negativity but more positive-going (more positive) feedback-related P300 when than with their nonoffending counterparts. These findings show that taxi drivers with traffic offence records were less sensitive to the consequences of behavior and more attuned to the magnitude of potential reward. In addition, behavioral data revealed that they were more willing to make risky decisions. All these characteristics pertain to impulsive personality traits. Based on these findings, we can conclude that the offenders in this sample were more impulsive than their nonoffending counterparts.

Modulation of impulsivity and reward sensitivity in intertemporal choice by striatal and midbrain dopamine synthesis in healthy adults.

Converging evidence links individual differences in mesolimbic and mesocortical dopamine (DA) to variation in the tendency to choose immediate rewards ("Now") over larger, delayed rewards ("Later"), or "Now bias." However, to date, no study of healthy young adults has evaluated the relationship between Now bias and DA with positron emission tomography (PET). Sixteen healthy adults (ages 24-34 yr; 50% women) completed a delay-discounting task that quantified aspects of intertemporal reward choice, including Now bias and reward magnitude sensitivity. Participants also underwent PET scanning with 6-[(18)F]fluoro-l-m-tyrosine (FMT), a radiotracer that measures DA synthesis capacity. Lower putamen FMT signal predicted elevated Now bias, a more rapidly declining discount rate with increasing delay time, and reduced willingness to accept low-interest-rate delayed rewards. In contrast, lower FMT signal in the midbrain predicted greater sensitivity to increasing magnitude of the Later reward. These data demonstrate that intertemporal reward choice in healthy humans varies with region-specific measures of DA processing, with regionally distinct associations with sensitivity to delay and to reward magnitude.

Physicians and Insider Trading.

Although insider trading is illegal, recent high-profile cases have involved physicians and scientists who are part of corporate governance or who have access to information about clinical trials of investigational products. Insider trading occurs when a person in possession of information that might affect the share price of a company's stock uses that information to buy or sell securities--or supplies that information to others who buy or sell--when the person is expected to keep such information confidential. The input that physicians and scientists provide to business leaders can serve legitimate social functions, but insider trading threatens to undermine any positive outcomes of these relationships. We review insider-trading rules and consider approaches to securities fraud in the health care field. Given the magnitude of the potential financial rewards, the ease of concealing illegal conduct, and the absence of identifiable victims, the temptation for physicians and scientists to engage in insider trading will always be present. Minimizing the occurrence of insider trading will require robust education, strictly enforced contractual provisions, and selective prohibitions against high-risk conduct, such as participation in expert consulting networks and online physician forums, by those individuals with access to valuable inside information.

Dysfunctional feedback processing in adolescent males with conduct disorder

Abnormalities in neural feedback-processing systems may play a role in the development of dysfunctional behavior in individuals diagnosed with conduct disorder (CD). The present study investigated the relation between CD adolescents and feedback processing by measuring event-related potentials (ERPs) in a single outcome gambling task, which included reward valence (loss and gain) and reward magnitude (10 and 50cents) as outcomes. N2 and P3 components have been established as effective indicators in studies of behavioral disinhibition, reward processing, and decision-making. Eighteen adolescent males (age: 13–17years) diagnosed with CD and 19 healthy age-matched male controls were recruited. Compared to healthy controls, CD individuals exhibited reduced N2 amplitudes in response to loss condition. There was also a significant decreased P3 amplitude in all conditions. The amplitudes of P3 were negatively correlated with impulsivity scores across both groups, and the amplitudes of N2 were positively correlated with impulsivity scores across both groups. Our findings suggest that adolescents with CD may be impaired in neural sensitivity feedback and the processing of environmental cues compared to healthy controls. Moreover, N2 and P3 may be reliable indices to detect different sensitivity in reward and punishment feedback processing.

The Roles of Dopamine and Hypocretin in Reward: A Electroencephalographic Study.

The proper functioning of the mesolimbic reward system is largely dependent on the neurotransmitter dopamine. Recent evidence suggests that the hypocretin system has significant projections to this reward system. We examined the distinct effects of reduced dopamine or reduced hypocretin levels on reward activity in patients with Parkinson’s disease, dopamine deficient, as well as patients with narcolepsy-cataplexy, hypocretin depleted, and healthy controls. Participants performed a simple game-like task while high-density electroencephalography was recorded. Topography and timing of event-related potentials for both reward cue, and reward feedback was examined across the entire dataset. While response to reward cue was similar in all groups, two distinct time points were found to distinguish patients and controls for reward feedback. Around 160ms both patient groups had reduced ERP amplitude compared to controls. Later at 250ms, both patient groups also showed a clear event-related potential (ERP), which was absent in controls. The initial differences show that both patient groups show a similar, blunted response to reward delivery. The second potential corresponds to the classic feedback-related negativity (FRN) potential which relies on dopamine activity and reflects reward prediction-error signaling. In particular the mismatch between predicted reward and reward subsequently received was significantly higher in PD compared to NC, independent of reward magnitude and valence. The intermediate FRN response in NC highlights the contribution of hypocretin in reward processing, yet also shows that this is not as detrimental to the reward system as in Parkinson’s. Furthermore, the inability to generate accurate predictions in NC may explain why hypocretin deficiency mediates cataplexy triggered by both positive and negative emotions.

Basolateral amygdala lesions abolish mutual reward preferences in rats

In a recent study, we demonstrated that rats prefer mutual rewards in a Prosocial Choice Task. Here, employing the same task, we show that the integrity of basolateral amygdala was necessary for the expression of mutual reward preferences. Actor rats received bilateral excitotoxic (n=12) or sham lesions (n=10) targeting the basolateral amygdala and were subsequently tested in a Prosocial Choice Task where they could decide between rewarding (“Both Reward”) or not rewarding a partner rat (“Own Reward”), either choice yielding identical reward to the actors themselves. To manipulate the social context and control for secondary reinforcement sources, actor rats were paired with either a partner rat (partner condition) or with an inanimate rat toy (toy condition). Sham-operated animals revealed a significant preference for the Both-Reward-option in the partner condition, but not in the toy condition. Amygdala-lesioned animals exhibited significantly lower Both-Reward preferences than the sham group in the partner but not in the toy condition, suggesting that basolateral amygdala was required for the expression of mutual reward preferences. Critically, in a reward magnitude discrimination task in the same experimental setup, both sham-operated and amygdala-lesioned animals preferred large over small rewards, suggesting that amygdala lesion effects were restricted to decision making in social contexts, leaving self-oriented behavior unaffected.

Age differences in delay discounting in Chinese adults

Delay discounting is the decrease in subjective value of a reward as the interval of receiving it is increased. Previous studies have found inconsistent results on the relationship between age and delay discounting, and few studies have been conducted using Chinese participants. The current study aimed to clarify this relationship using a relatively large sample of Chinese adults with a wide age range (viz., 18 to 86years old). A total of 1288 individuals completed the Monetary Choice Questionnaire. Results showed that the rate of delay discounting increased with age across adulthood, with younger participants (18–30years) discounting less than both middle-aged participants (31–60years) and older participants (over 60years); and middle-aged participants discounting less than older participants. Furthermore, when the reward magnitude was large, participants were more likely to wait for delayed rewards. The increase in delay discounting rate from middle-aged adults to older adults might be explained by the life-cycle theory. The increase in delay discounting rate from young adults to middle-aged adults may reflect that young adults expect much time and a variety of future positive life events in the rest of their lives.

Dissociable contribution of nucleus accumbens and dorsolateral striatum to the acquisition of risk choice behavior in the rat

While a growing body of research has suggested that the mesocorticolimbic dopamine systems play a key role in decision making under risk, how the nucleus accumbens (NAC) is involved in the acquisition of risk choice behavior remains unclear. This study used a T-maze task to assess risk-based decision making in which the rat was required to assess the risk by choosing to enter either a small and certain reward arm or a large but uncertain reward arm of the maze. The latter option, when chosen, resulted in provision of 2, 4, or 8 sweeten pellets with a probability (p) of 0.5, 0.25, or 0.125, respectively. Thus the latter arm provided three different conditions of reward ratio, compared to the choice of former arm, which always provided 1 pellet with p=1. This risk choice task was then run with the expected value being equality between the binary choice options. The experimental rats first received an excitoneurotoxic lesion affecting either the NAC or the dorsolateral striatum (DLS) and this was followed by post-lesion behavioral examination. The sham lesion control rats acquired a stable risk choice with regard to each reward ratio over a 10-day test. The pattern of choice behavior appeared in risk-seeking when p=0.5 to obtain 2 pellets, and was risk-averse when larger reward resulted in lower p. The NAC lesion significantly disrupted the acquisition of the aforementioned risk choice behavior and apparently shifted the choice into a risk-averse style for all three reward ratios. No such effect was observed in the rats with DLS lesions. Neither the gross motor action nor the discrimination of different reward magnitudes was impaired by the lesions affecting either the NAC or DLS as assessed by an additional experiment. These findings suggest that firstly there is heterogeneity between NAC and DLS with respect to risk-based decision making, and that secondly the NAC is involved and critical to the acquisition of behavioral choice under risk, specially when the expected value of the reward under the two choice options is equal.