Discounting Of Future Rewards Research Articles

Dopamine transients follow a striatal gradient of reward time horizons

Animals make predictions to guide their behavior and update those predictions through experience. Transient increases in dopamine (DA) are thought to be critical signals for updating predictions. However, it is unclear how this mechanism handles a wide range of behavioral timescales—from seconds or less (for example, if singing a song) to potentially hours or more (for example, if hunting for food). Here we report that DA transients in distinct rat striatal subregions convey prediction errors based on distinct time horizons. DA dynamics systematically accelerated from ventral to dorsomedial to dorsolateral striatum, in the tempo of spontaneous fluctuations, the temporal integration of prior rewards and the discounting of future rewards. This spectrum of timescales for evaluative computations can help achieve efficient learning and adaptive motivation for a broad range of behaviors.

Smoking, hazardous drinking and individual differences in cost and gain discounting.

Aims:Past research has linked substance use to individual differences in discounting of future rewards. Since behaviours such as smoking and excessive drinking appear to involve a devaluation of future negative consequences, discounting of costs may also be relevant in the understanding of such behaviour. The primary aims were to investigate the association between cost discounting and the behaviours smoking and hazardous drinking.Methods:In four studies, individuals recruited from the crowdsourcing marketplace Amazon Mechanical Turk responded to measures of discounting. Results were combined by meta-analysis of the standardised mean differences (d) between self-reported smokers and non-smokers, and between participants with AUDIT scores of 10+ (hazardous drinking) and those with scores below 10.Results:In comparison with non-smokers, smokers’ relative valuations of future gains were lower, d = –0.32, 95% CI [–0.47, –0.18]. There was no association between smoking and cost discounting, d = –0.02, 95% CI [–0.17, 0.13]. Participants with AUDIT scores of 10+ valued future gains somewhat lower than participants with scores below 10, d = –0.17, 95% CI [–0.35, 0.01]. There was no association between hazardous drinking and cost discounting, d = –0.02, 95% CI [–0.21, 0.16]. According to Bayes Factors (BF), the data supported an association between gain discounting and smoking (BF > 100). It was insensitive in the analysis of gain discounting and hazardous drinking (BF = 0.6), but it strongly supported null-effects in both analyses of cost discounting (BFs = 0.1).Conclusion:The results suggest that the robust link between gain discounting and smoking status does not reflect a general devaluation of future outcomes among smokers.

Latent Profile Analysis of Heavy Episodic Drinking in Emerging Adults: A Reinforcer Pathology Approach.

Heavy episodic drinking (HED) is a major public health problem among emerging adults (individuals 18 to 25), but with considerable heterogeneity in concurrent substance use and psychopathology. The current study used latent profile analysis (LPA) to detect discrete subgroups of HED based on alcohol, other drug severity, and concurrent psychopathology. A reinforcer pathology approach was used to understand motivational differences among the latent subgroups. Participants were 2 samples of emerging adults reporting regular HED, 1 Canadian (n=730) and 1 American (n=602). Indicators for the LPA were validated dimensional self-report assessments of alcohol severity, cannabis severity, other drug severity, nicotine dependence, depression, anxiety, posttraumatic stress disorder, and attention-deficit/hyperactivity disorder. Reinforcer pathology indicators were measures of alcohol demand, proportionate substance-related reinforcement, and discounting of future rewards. The LPA yielded parallel 3-class solutions in both samples. The largest subgroup was characterized by comparatively low substance severity and psychopathology (Low overall severity). The second largest subgroup was characterized by comparatively high alcohol and other drug severity (excluding tobacco) and high levels of psychopathology (Heavy alcohol & high psychiatric severity). The third subgroup exhibited high alcohol, smoking and intermediate levels of other substance use and psychopathology (Heavy alcohol, smoking, & intermediate psychiatric severity). The Heavy alcohol & high psychiatric severity and Heavy alcohol, smoking, & intermediate psychiatric severity subgroups exhibited significantly higher alcohol demand, greater proportionate substance-related reinforcement, and steeper delay discounting. Parallel latent subgroups of emerging adults engaging in HED were present in both samples, and the high-risk subgroups were significantly differentiated by the reinforcer pathology indicators. These latent profiles may ultimately inform heterogeneity in the longitudinal course of HED in emerging adults.

Rationally inattentive intertemporal choice

Discounting of future rewards is traditionally interpreted as evidence for an intrinsic preference in favor of sooner rewards. However, temporal discounting can also arise from internal uncertainty in value representations of future events, if one assumes that noisy mental simulations of the future are rationally combined with prior beliefs. Here, we further develop this idea by considering how simulation noise may be adaptively modulated by task demands, based on principles of rational inattention. We show how the optimal allocation of mental effort can give rise to the magnitude effect in intertemporal choice. In a re-analysis of two prior data sets, and in another experiment, we reveal several behavioral signatures of this theoretical account, tying choice stochasticity to the magnitude effect. We conclude that some aspects of temporal discounting may result from a cognitively plausible adaptive response to the costs of information processing.

The Role of Future Discounting in Subsistence Decisions: The Case of Hohokam Agave Bajada Cultivation

ABSTRACTThis article demonstrates the relevance of future discounting behavior to studies of prehistoric subsistence strategies. The case study of Agave spp. bajada cultivation among the Classic Period, a.d. 1150–1450, Hohokam of southern Arizona illustrates how slight discounting of future rewards can significantly alter agriculture investment strategies. Despite a higher kcal/hour return than maize agriculture, agave cultivation lessened or ceased in several communities during a time in which intensification would be predicted due to inferred subsistence stress. We surmise the decade-long maturation of agave made it unattractive relative to more immediate return resources in any context that presented risks to land tenure. The cessation of agave cultivation can be understood on two levels of explanation. The long-term investments required by agave and tendency to discount future rewards are the ultimate causes. Political and social instability during the tumultuous Classic period, causing increased insecurity in land tenure, are the proximal causes.

Gender-Specific Differences in the Relationship between Autobiographical Memory and Intertemporal Choice in Older Adults.

As the population of older adults grows, their economic choices will have increasing impact on society. Research on the effects of aging on intertemporal decisions shows inconsistent, often opposing results, indicating that yet unexplored factors might play an essential role in guiding one's choices. Recent studies suggest that episodic future thinking, which is based on the same neural network involved in episodic memory functions, leads to reductions in discounting of future rewards. As episodic memory functioning declines with normal aging, but to greatly variable degrees, individual differences in delay discounting might be due to individual differences in the vitality of this memory system in older adults. We investigated this hypothesis, using a sample of healthy older adults who completed an intertemporal choice task as well as two episodic memory tasks. We found no clear evidence for a relationship between episodic memory performance and delay discounting in older adults. However, when additionally considering gender differences, we found an interaction effect of gender and autobiographical memory on delay discounting: while men with higher memory scores showed less delay discounting, women with higher memory scores tended to discount the future more. We speculate that this gender effect might stem from the gender-specific use of different modal representation formats (i.e. temporal or visual) during assessment of intertemporal choice options.

Supplemental Material for Discounting of Future Rewards in Behavioural Variant Frontotemporal Dementia and Alzheimer’s Disease

Supplemental Material for Discounting of Future Rewards in Behavioural Variant Frontotemporal Dementia and Alzheimer’s Disease

Activity in dlPFC and its effective connectivity to vmPFC are associated with temporal discounting

There is widespread interest in identifying computational and neurobiological mechanisms that influence the ability to choose long-term benefits over more proximal and readily available rewards in domains such as dietary and economic choice. We present the results of a human fMRI study that examines how neural activity relates to observed individual differences in the discounting of future rewards during an intertemporal monetary choice task. We found that a region of left dorsolateral prefrontal cortex (dlPFC) BA-46 was more active in trials where subjects chose delayed rewards, after controlling for the subjective value of those rewards. We also found that the connectivity from dlPFC BA-46 to a region of ventromedial prefrontal cortex (vmPFC) widely associated with the computation of stimulus values, increased at the time of choice, and especially during trials in which subjects chose delayed rewards. Finally, we found that estimates of effective connectivity between these two regions played a critical role in predicting out-of-sample, between-subject differences in discount rates. Together with previous findings in dietary choice, these results suggest that a common set of computational and neurobiological mechanisms facilitate choices in favor of long-term reward in both settings.

Age-related changes in insula cortical thickness and impulsivity: Significance for emotional development and decision-making

Insula function has been associated with emotional regulation, adjusting to changing outcomes under risk, reward and loss anticipation, discounting of future rewards, and self-rated impulsivity. The role of the insula in these processes may be fundamentally related to prospective thinking, a trait that increases with age. There is evidence that insular cortical thickness shows age related decreases that parallel age related increases in future orientation and planning. We tested the hypothesis that nonplanning decreases with age and that insula thickness is related to both age and nonplanning impulsivity. Fifty-nine male and female participants, ranging in age from 10 to 22 years old, underwent structural magnetic resonance imaging (MRI) procedures and were assessed using the Barratt Impulsiveness Scale (BIS). We observed that anterior insula thickness and nonplanning impulsivity show an inverse relationship with age and that there is a significant positive linear relationship between anterior insula thickness and nonplanning.

Decisions for Others Become Less Impulsive the Further Away They Are on the Family Tree

BackgroundPeople tend to prefer a smaller immediate reward to a larger but delayed reward. Although this discounting of future rewards is often associated with impulsivity, it is not necessarily irrational. Instead it has been suggested that it reflects the decision maker’s greater interest in the ‘me now’ than the ‘me in 10 years’, such that the concern for our future self is about the same as for someone else who is close to us.Methodology/Principal FindingsTo investigate this we used a delay-discounting task to compare discount functions for choices that people would make for themselves against decisions that they think that other people should make, e.g. to accept $500 now or $1000 next week. The psychological distance of the hypothetical beneficiaries was manipulated in terms of the genetic coefficient of relatedness ranging from zero (e.g. a stranger, or unrelated close friend), .125 (e.g. a cousin), .25 (e.g. a nephew or niece), to .5 (parent or sibling).Conclusions/SignificanceThe observed discount functions were steeper (i.e. more impulsive) for choices in which the decision-maker was the beneficiary than for all other beneficiaries. Impulsiveness of decisions declined systematically with the distance of the beneficiary from the decision-maker. The data are discussed with reference to the implusivity and interpersonal empathy gaps in decision-making.

Self-Control, Discounting and Reward: Why Picoeconomics is Economics

We review efforts by economists to capture the qualitative predictions of picoeconomics – that is, accounts of willpower based on hyperbolic discounting of future rewards – using so-called quasi-hyperbolic or β-δ functions. We find that these models do not predict the full suite of phenomena associated with reward bundling, the re-framing of discrete sequences of choices into series. External, specifically social, structures often promote such bundling; thus willpower is not a strictly internal process based on management of visceral influences. We then present an experiment that demonstrates manipulation of dispositions to bundle in a group of nicotine addicts. Non-smokers in the experiment are not similarly influenced, which may reflect the fact that people who avoid addiction have generally learned to bundle without specific cueing or suggestion.

ALTERING THE MAGNITUDE OF DELAY DISCOUNTING BY PATHOLOGICAL GAMBLERS

The present study explored the delay discounting of future and past monetary rewards by pathological gamblers. Using a multiple baseline design, following repeated exposure to choices between smaller immediate and larger delayed consequences, participants completed a relational responding task that attempted to alter the psychological functions of irrelevant stimuli and to affect subsequent delay discounting. Results support previous literature on the discounting of delayed consequences by pathological gamblers, illustrate that the discounting of past rewards occurs in a similar fashion to the well-documented literature on the discounting of future rewards, and that magnitude of discounting can be altered.

Alcohol consumption and discounting

This study addresses whether discounting of future rewards may have an impact on alcohol consumption and propensity to alcohol-related harms, by applying survey data among young people whose drinking career hardly may have affected their time preferences. Analyses from a school survey among 17,000, over 13–17 year olds in Norway showed that discount rates were positively associated with drinking frequency and intoxication frequency when age, gender, impulsivity and disposable income were controlled for. Moreover, the results suggested that having made choices under the influence of alcohol leading to a negative outcome (deliberate self-harm, drunk-driving, vandalism or use of narcotics) was significantly more prevalent among those with high-discount rates compared to others, also when controlling for alcohol consumption, age, gender and impulsivity. The results are discussed in relation to the potential role of discounting in pathways to addictive behaviour.

Discounting of Past and Future Rewards of Texas Hold’em Gamblers

The present study investigated the delay discounting of future and past monetary rewards by Texas Hold’em Poker gamblers. While present at a poker tournament, eight participants completed three delay discounting tasks which presented choices between smaller immediate and larger delayed consequences. All participants discounted the larger amounts of money in a hyperbolic fashion, and the degree to which discounting occurred appeared to be a function of the magnitude of regular wagering (less than or greater than 10 dollars per game) made during poker play. Results support previous research on the discounting of delayed consequences by gamblers and illustrate that the discounting of past rewards occurs in a similar fashion to the well-documented literature on the discounting of future rewards.

The job-search problem with competition: an evolutionarily stable dynamic strategy

We consider a game-theoretical solution for an optimal stopping problem which we describe in terms of a job-search problem with an infinite population of candidates and an infinite population of posts of varying value. Each candidate finds posts from the post population at unit rate. If a post found is still vacant, a candidate can either accept it or reject it. The reward to a candidate is the value of the post if one is accepted and zero if he never accepts a post. There are no costs for searching, no discounting of future rewards and no recall of previously found posts. The only pressure on a candidate to accept a post comes from the changing rate at which he finds vacant posts (and the values associated with them) as a result of the actions of the other candidates. It is not possible to define optimality for a single candidate without reference to the policies followed by the other candidates. We say a policy π is an evolutionarily stable strategy (ESS) if it has the following property: if all candidates used π and an individual candidate was given the option of changing his policy, then it would not be to his advantage to do so. We first find the optimal value function and optimal policy for the case of a single candidate operating in an environment where the distribution of posts on offer and the chance of finding one both vary with time in a known way. We then show that for the infinite population there is a unique ESS given by a control-limit policy π c, where the control-limit function c is the solution of a given differential equation with a given initial condition. This function c also gives the expected future reward function for any single candidate when all candidates use π c.

The job-search problem with competition: an evolutionarily stable dynamic strategy

We consider a game-theoretical solution for an optimal stopping problem which we describe in terms of a job-search problem with an infinite population of candidates and an infinite population of posts of varying value. Each candidate finds posts from the post population at unit rate. If a post found is still vacant, a candidate can either accept it or reject it. The reward to a candidate is the value of the post if one is accepted and zero if he never accepts a post. There are no costs for searching, no discounting of future rewards and no recall of previously found posts. The only pressure on a candidate to accept a post comes from the changing rate at which he finds vacant posts (and the values associated with them) as a result of the actions of the other candidates. It is not possible to define optimality for a single candidate without reference to the policies followed by the other candidates. We say a policy π is an evolutionarily stable strategy (ESS) if it has the following property: if all candidates used π and an individual candidate was given the option of changing his policy, then it would not be to his advantage to do so. We first find the optimal value function and optimal policy for the case of a single candidate operating in an environment where the distribution of posts on offer and the chance of finding one both vary with time in a known way. We then show that for the infinite population there is a unique ESS given by a control-limit policy π c, where the control-limit function c is the solution of a given differential equation with a given initial condition. This function c also gives the expected future reward function for any single candidate when all candidates use π c.