Temporal Reward Discounting Research Articles

OLDER AGE IS ASSOCIATED WITH TEMPORAL DISCOUNTING OF TIME USE BUT NOT MONETARY REWARDS

Abstract Age differences in temporal discounting have long puzzled researchers. Although older adults tend to prioritize the present over the future due to more limited time horizons compared to younger adults there is no evidence for an age association with temporal discounting of monetary rewards. Socioemotional selectivity theory posits that as time horizons become more limited goals related to emotional meaning are prioritized over future-oriented goals because they are realized in the present, leading older adults to place more value on experiences than younger adults. A small body of evidence showing age-related preferences for small, immediate, rewards in emotionally meaningful domains is consistent with SST. However, prior studies used hypothetical tasks with vague trade-offs, limiting their interpretability and generalizability. In the present study, we developed a novel paradigm to examine age differences in temporal discounting of rewards related to emotional experience in a controlled environment. 120 participants, aged 22-96 years, came to the lab and made a series of choices indicating if they would prefer to replace 5 minutes of boring tasks with an emotionally meaningful variant of the task today, or a larger amount of time in their next visit in six months. Participants then made similar choices about the timing of receiving comparable monetary rewards. We hypothesized that in contrast to monetary rewards, older adults are less likely than younger adults to wait for rewards related to emotional experiences because time becomes increasingly valued as it grows scarce. Findings support the hypothesis and highlight the role of perceived time-horizons.

Erotic cue exposure increases physiological arousal, biases choices toward immediate rewards, and attenuates model-based reinforcement learning.

Computational psychiatry focuses on identifying core cognitive processes that appear altered across distinct psychiatric disorders. Temporal discounting of future rewards and model-based control during reinforcement learning have proven as two promising candidates. Despite its trait-like stability, temporal discounting may be at least partly under contextual control. Highly arousing cues were shown to increase discounting, although evidence to date remains somewhat mixed. Whether model-based reinforcement learning is similarly affected by arousing cues remains unclear. Here, we tested cue-reactivity effects (erotic pictures) on subsequent temporal discounting and model-based reinforcement learning in a within-subjects design in n = 39 healthy heterosexual male participants. Self-reported and physiological arousal (cardiac activity and pupil dilation) were assessed before and during cue exposure. Arousal was increased during exposure of erotic versus neutral cues both on the subjective and autonomic level. Erotic cue exposure increased discounting as reflected by more impatient choices. Hierarchical drift diffusion modeling (DDM) linked increased discounting to a shift in the starting point bias of evidence accumulation toward immediate options. Model-based control during reinforcement learning was reduced following erotic cues according to model-agnostic analysis. Notably, DDM linked this effect to attenuated forgetting rates of unchosen options, leaving the model-based control parameter unchanged. Our findings replicate previous work on cue-reactivity effects in temporal discounting and for the first time show similar effects in model-based reinforcement learning in a heterosexual male sample. This highlights how environmental cues can impact core human decision processes and reveal that comprehensive modeling approaches can yield novel insights in reward-based decision processes.

Poor sleep quality is significantly associated with effort but not temporal discounting of monetary rewards.

Poor sleep quality is significantly associated with effort but not temporal discounting of monetary rewards.

Decision Making across Adulthood during Physical Distancing

ABSTRACT Covid-19-related social-distancing measures have dramatically limited physical social contact between individuals and increased monetary and health concerns for individuals of all ages. We wondered how these new societal conditions would impact the choices individuals make about monetary, health, and social rewards, and if these unprecedented conditions would have a differential impact on older individuals. We conducted two online studies to examine temporal discounting of monetary, health, and social rewards; stated preferences for monetary, health, and social rewards; and physical distancing behaviors. Both studies recruited equal numbers of White/Caucasian, Black/African American, and Hispanic/Latinx participants. We found that older adults were more likely to prefer smaller, sooner social and health-related rewards in decision-making tasks. These data further support the assertion that older adults have increased motivation for social and health rewards compared to younger individuals and that these age differences in motivation are important to consider when examining decision-making across the adult life span.

Effects of tDCS on reward responsiveness and valuation in Parkinson's patients with impulse control disorders.

Parkinson's disease (PD) patients with impulse control disorders (ICD) frequently report hypersensitivity to rewards. However, a few studies have explored the effectiveness of modulation techniques on symptoms experienced by these patients. In this study, we assessed the effect of anodal tDCS over the DLPFC on reward responsiveness and valuation in PD patients with ICD. 43 participants (15 PD patients with ICD, 13 PD without ICD, and 15 healthy matched controls) were asked to perform a reward-craving test employing both explicit (self-ratings of liking and wanting) and implicit (heart rate and skin conductance response) measures, as well as two temporal discounting tasks with food and money rewards. Each participant performed the experimental tasks during active anodal tDCS of the left dorsolateral prefrontal cortex (DLPFC), anodal tDCS of the primary motor cortex (M1), and sham tDCS. Results showed increased wanting and a steeper temporal discounting of rewards in PD with ICD compared to the other groups. Moreover, we found that PD without ICD exhibit reduced liking for rewards. tDCS results capable to modulate the altered intensity of PD patients' liking, but not wanting and temporal discounting of rewards in PD patients with ICD. These findings confirm that alterations in reward responsiveness and valuation are characteristics of impulse control disorders in patients with PD but suggest that anodal tDCS over the left DLPFC is not capable to influence these processes. At the same time, they provide new insight into affective experience of rewards in PD.

Do ADHD Symptoms, Executive Function, and Study Strategies Predict Temporal Reward Discounting in College Students with Varying Levels of ADHD Symptoms? A Pilot Study.

The purpose of this study was to examine the relationship between temporal reward discounting and attention deficit hyperactivity disorder (ADHD) symptoms in college students. Additionally, we examined whether temporal reward discounting was associated with executive functioning in daily life and with learning and study strategies in this group. Thirty-nine college students (19 with ADHD and 20 controls) participated after meeting criteria for ADHD or non-ADHD based on standardized assessment. Strong preferences for small immediate rewards were specifically associated with the ADHD symptom domain hyperactivity–impulsivity. Additionally, these preferences were associated with daily life executive function problems and with weak learning and study strategies. This suggests that steep temporal discounting may be a key mechanism playing a role in the daily life challenges that college students with ADHD symptoms face. If these findings are replicated in larger samples, then intervention strategies may profitably be developed to counteract this strong preference for small immediate rewards in college students with ADHD symptoms.

Intact value-based decision-making during intertemporal choice in women with remitted anorexia nervosa? An fMRI study

Extreme restrictive food choice in anorexia nervosa is thought to reflect excessive self-control and/or abnormal reward sensitivity. Studies using intertemporal choice paradigms have suggested an increased capacity to delay reward in anorexia nervosa, and this may explain an unusual ability to resist immediate temptation and override hunger in the long-term pursuit of thinness. It remains unclear, however, whether altered delay discounting in anorexia nervosa constitutes a state effect of acute illness or a trait marker observable after recovery. We repeated the analysis from our previous fMRI investigation of intertemporal choice in acutely underweight patients with anorexia nervosa in a sample of weight-recovered women with anorexia nervosa (n = 36) and age-matched healthy controls (n = 36) who participated in the same study protocol. Follow-up analyses explored functional connectivity separately in both the weight-recovered/healthy controls sample and the acute/healthy controls sample. In contrast to our previous findings in acutely underweight patients with anorexia nervosa, we found no differences between weight-recovered patients with anorexia nervosa and healthy controls at either behavioural or neural levels. New analysis of data from the acute/healthy controls sample sample revealed increased coupling between dorsal anterior cingulate cortex and posterior brain regions as a function of decision difficulty, supporting the hypothesis of altered neural efficiency in the underweight state. This was a cross-sectional study, and the results may be task-specific. Although our results underlined previous demonstrations of divergent temporal reward discounting in acutely underweight patients with anorexia nervosa, we found no evidence of alteration in patients with weight-recovered anorexia nervosa. Together, these findings suggest that impaired valuebased decision-making may not constitute a defining trait variable or “scar” of the disorder.

Vigor of reaching, walking, and gazing movements: on the consistency of interindividual differences.

Movement vigor is an important feature of motor control that is thought to originate from cortico-basal ganglia circuits and processes shared with decision-making, such as temporal reward discounting. Accordingly, vigor may be related to one's relationship with time, which may, in turn, reflect a general trait-like feature of individuality. While significant interindividual differences of vigor have been typically reported for isolated motor tasks, little is known about the consistency of such differences across tasks and movement effectors. Here, we assessed interindividual consistency of vigor across reaching (both dominant and nondominant arm), walking, and gazing movements of various distances within the same group of 20 participants. Given distinct neural pathways and biomechanical specificities of each movement modality, a significant consistency would corroborate the trait-like aspect of vigor. Vigor scores for dominant and nondominant arm movements were found to be highly correlated across individuals. Vigor scores of reaching and walking were also significantly correlated across individuals, indicating that people who reach faster than others also tend to walk faster. At last, vigor scores of saccades were uncorrelated with those of reaching and walking, reaffirming that the vigor of stimulus-elicited eye saccades is distinct. These findings highlight the trait-like aspect of vigor for reaching movements with either arms and, to a lesser extent, walking.NEW & NOTEWORTHY Robust interindividual differences of movement vigor have been reported for arm reaching and saccades. Beyond biomechanics, personality trait-like characteristics have been proposed to account for those differences. Here, we examined for the first time the consistency of interindividual differences of vigor during dominant/nondominant arm reaching, walking, and gazing to assess the trait-like aspect of vigor. We found a significant consistency of vigor within our group of individuals for all tested tasks/effectors except saccades.

Deep brain stimulation of the subthalamic nucleus and the temporal discounting of primary and secondary rewards

Although deep brain stimulation of the subthalamic nucleus is an effective surgical treatment for Parkinson's disease, it may expose patients to non-motor side effects such as increased impulsivity and changes in decision-making behavior. Even if several studies have shown that stimulation of the subthalamic nucleus increases the incentive salience of food rewards in both humans and animals, temporal discounting for food rewards has never been investigated in patients who underwent STN-DBS. In this study, we measured inter-temporal choice after STN-DBS, using both primary and secondary rewards. In particular, PD patients who underwent STN-DBS (in ON medication/ON stimulation), PD patients without STN-DBS (in ON medication) and healthy matched controls (C) performed three temporal discounting tasks with food (primary reward), money and discount vouchers (secondary rewards). Participants performed also neuropsychological tests assessing memory and executive functions. Our results show that STN-DBS patients and PD without DBS behave as healthy controls. Even PD patients who after DBS experienced weight gain and/or eating alterations did not show an increased temporal discounting for food rewards. Interestingly, patients taking a higher dosage of dopaminergic medications, fewer years from DBS surgery and, unexpectedly, with better episodic memory were also those who discounted rewards more. In conclusion, this study shows that STN-DBS does not affect temporal discounting of primary and secondary rewards. Furthermore, by revealing interesting correlations between clinical measures and temporal discounting, it also shed light on the clinical outcomes that follow STN-DBS in patients with PD.

Neural mechanisms of individual differences in temporal discounting of monetary and primary rewards in adolescents

Adolescents are generally characterized as impulsive. However, impulsivity is a multi-dimensional construct that involves multiple component processes. Which of these components contribute to adolescent impulsivity is currently unclear. This study focused on the neural mechanisms underlying individual differences in distinct components of temporal discounting (TD), i.e., the preference for smaller immediate rewards over larger delayed rewards. Participants were 58 adolescents (12–16 years-old) who performed an fMRI TD task with both monetary and snack rewards. Using mixed-effects modeling, we determined participants’ average impatience, and further decomposed TD choices into: 1) amount sensitivity (unique contribution of the magnitude of the immediate reward); and 2) delay sensitivity (unique contribution of delay duration). Adolescents’ average impatience was positively correlated with frontoparietal and ventral striatal activity during delayed reward choices, and with ventromedial prefrontal cortex activity during immediate reward choices. Adolescents’ amount sensitivity was positively associated with ventral striatal and dorsal anterior cingulate cortex activity during immediate reward choices. Delay sensitivity was positively correlated with inferior parietal cortex activity during delayed reward choices. As expected, snacks were discounted more steeply than money, and TD of both reward types was associated with overlapping activation in the inferior parietal cortex. Exploring whether testosterone or estradiol were associated with TD and its neural correlates revealed no significant associations. These findings indicate that distinct components contribute uniquely to TD choice and that individual differences in amount sensitivity are uniquely associated with activation of reward valuation areas, while individual differences in delay sensitivity are uniquely associated with activation of cognitive control areas.

Altered Neural Efficiency of Decision Making During Temporal Reward Discounting in Anorexia Nervosa

The ability of individuals with anorexia nervosa (AN) to resist hunger and restrict caloric intake is often believed to reflect an unusual amount of self-control. However, the underlying neural substrate is poorly understood, especially in adolescent patients. Functional magnetic resonance imaging was used during an intertemporal choice task to probe the hemodynamic correlates of a common measurement of self-control-delayed (monetary) reward discounting-in a sample of acutely ill, predominately adolescent female patients with AN (n= 31) and age-matched healthy controls (n= 31). Delayed discounting rates did not differ between the groups, but decision making was consistently faster in the AN group. Although no group differences in the neural correlates of reward valuation were evident, activation associated with decision making was decreased in the AN group, most notably in the lateral prefrontal and posterior parietal regions implicated in executive control. Follow-up analysis of difficult decisions showed decreased activation in the AN group in a region of the dorsal anterior cingulate cortex. Decreased activation in frontoparietal regions involved in decision making, but faster and more consistent choice behavior, suggests that the altered efficiency of neural resource allocation might underlie an increased level of self-control in AN. This pattern of neural activation and behavior might reflect an ingrained "habit" to sustain high-level proactive (anticipatory) cognitive control in AN, which in turn might compromise reactive control mechanisms needed to adapt to changing cognitive demands, such as when difficult decisions must be made.

The Role of Prospection in Steep Temporal Reward Discounting in Gambling Addiction.

Addiction and pathological gambling (PG) have been consistently associated with high impulsivity and a steep devaluation of delayed rewards, a process that is known as temporal discounting (TD). Recent studies indicated that enhanced episodic future thinking (EFT) results in less impulsive TD in healthy controls (HCs). In a separate line of research, it has been suggested that non-linearities in time perception might contribute to reward devaluation during inter-temporal choice. Therefore, in addition to deficits in valuation processes and executive control, impairments in EFT and non-linearities in time perception have been hypothesized to contribute to steep TD in addiction. In this study, we explore such a potential association of impairments in EFT and time perception with steep TD in PG. We investigated 20 PGs and 20 matched HCs. TD was assessed via a standard computerized binary choice task. EFT was measured using a variation of the Autobiographical Memory Interview by Levine et al. (1). Time perception was assessed with a novel task, utilizing a non-linear rating procedure via circle-size adjustments. Groups did not differ in baseline EFT. In both groups, a power law accounted time perception best, and the degree of non-linearity in time perception correlated with discounting across groups. A multiple regression analysis across all predictors and covariates revealed that only group status (PG/HC) and depression were significantly associated with discounting behavior such that PG increased TD and depression attenuated TD. Our findings speak against the idea that steep TD in PG is due to a skewed perception of time or impairments in EFT, at least under the present task conditions. The lack of overall group differences in EFT does not rule out the possibility of more complex interactions of EFT and decision-making. These interactions might be diminished in pathological gambling or addiction more generally, when other task configurations are used.

Temporal delay discounting in acutely ill and weight-recovered patients with anorexia nervosa.

Patients with anorexia nervosa (AN) are characterized by a very low body weight but readily give up immediate rewards (food) for long-term goals (slim figure), which might indicate an unusual level of self-control. This everyday clinical observation may be quantifiable in the framework of the anticipation-discounting dilemma. Using a cross-sectional design, this study compared the capacity to delay reward in 34 patients suffering from acute AN (acAN), 33 weight-recovered AN patients (recAN) and 54 healthy controls. We also used a longitudinal study to reassess 21 acAN patients after short-term weight restoration. A validated intertemporal choice task and a hyperbolic model were used to estimate temporal discounting rates. Confirming the validity of the task used, decreased delay discounting was associated with age and low self-reported impulsivity. However, no group differences in key measures of temporal discounting of monetary rewards were found. Increased cognitive control, which has been suggested as a key characteristic of AN, does not seem to extend the capacity to wait for delayed monetary rewards. Differences between our study and the only previous study reporting decreased delay discounting in adult AN patients may be explained by the different age range and chronicity of acute patients, but the fact that weight recovery was not associated with changes in discount rates suggests that discounting behavior is not a trait marker in AN. Future studies using paradigms with disorder-specific stimuli may help to clarify the role of delay discounting in AN.

Homeostatic reinforcement learning for integrating reward collection and physiological stability.

Efficient regulation of internal homeostasis and defending it against perturbations requires adaptive behavioral strategies. However, the computational principles mediating the interaction between homeostatic and associative learning processes remain undefined. Here we use a definition of primary rewards, as outcomes fulfilling physiological needs, to build a normative theory showing how learning motivated behaviors may be modulated by internal states. Within this framework, we mathematically prove that seeking rewards is equivalent to the fundamental objective of physiological stability, defining the notion of physiological rationality of behavior. We further suggest a formal basis for temporal discounting of rewards by showing that discounting motivates animals to follow the shortest path in the space of physiological variables toward the desired setpoint. We also explain how animals learn to act predictively to preclude prospective homeostatic challenges, and several other behavioral patterns. Finally, we suggest a computational role for interaction between hypothalamus and the brain reward system.

Predictive validity of delay discounting behavior in adolescence: a longitudinal twin study.

A standard assumption in the delay discounting literature is that individuals who exhibit steeper discounting of hypothetical rewards also experience greater difficulty deferring gratification to real-world rewards. There is ample cross-sectional evidence that delay discounting paradigms reflect a variety of maladaptive psychosocial outcomes, including substance use pathology. We sought to determine whether a computerized assessment of hypothetical delay discounting (HDD) taps into behavioral impulsivity in a community sample of adolescent twins (N = 675). Using a longitudinal design, we hypothesized that greater HDD at age 14-15 predicts real-world impulsive choices and risk for substance use disorders in late adolescence. We also examined the genetic and environmental structure of HDD performance. Individual differences in HDD behavior showed moderate heritability, and were prospectively associated with real-world temporal discounting at age 17-18. Contrary to expectations, HDD was not consistently related to substance use or trait impulsivity. Although a significant association between HDD behavior and past substance use emerged in males, this effect was mediated by cognitive ability. In both sexes, HDD failed to predict a comprehensive index of substance use problems and behavioral disinhibition in late adolescence. In sum, we present some of the first evidence that HDD performance is heritable and predictive of real-world temporal discounting of rewards. Nevertheless, HDD might not serve as a valid marker of substance use disorder risk in younger adolescents, particularly females.

Temporal reward discounting in children, adolescents, and emerging adults during an experiential task.

The goal of this study was to examine age effects on the ability/willingness to wait for large rewards in a real temporal reward discounting task from childhood to adulthood. Therefore, a real temporal discounting (TD) task was administered to children aged 6–12 (n = 39), adolescents aged 13–17 (n = 28), and young adults aged 18–19 (n = 55). Findings indicated that the cross-sectional development of TD followed a quadratic pattern across age groups, with adolescents choosing more often than children and adults to wait for the large delayed reward, resulting in reward-maximization. Various interpretations of this finding were offered, including a focus on reward maximization despite an immature ability to exert self-control, and flexible self-control which was high during this task as a result of strong motivation to maximize financial gains.

Medial prefrontal and anterior cingulate cortical thickness predicts shared individual differences in self-generated thought and temporal discounting

When deprived of compelling perceptual input, the mind is often occupied with thoughts unrelated to the immediate environment. Previous behavioral research has shown that this self-generated task-unrelated thought (TUT), especially under non-demanding conditions, relates to cognitive capacities such as creativity, planning, and reduced temporal discounting. Despite the frequency and importance of this type of cognition, little is known about its structural brain basis. Using MRI-based cortical thickness measures in 37 participants, we were able to show that individuals with a higher tendency to engage in TUT under low-demanding conditions (but not under high-demanding conditions) show an increased thickness of medial prefrontal cortex (mPFC) and anterior/midcingulate cortex. Thickness of these regions also related to less temporal discounting (TD) of monetary rewards in an economic task, indicative of more patient decision-making. The findings of a shared structural substrate in mPFC and anterior/midcingulate cortex underlying both TUT and TD suggest an important role of these brain regions in supporting the self-generation of information that is unrelated to the immediate environment and which may be adaptive in nature.

The neural correlates of temporal reward discounting.

Temporal reward discounting (TD) refers to the decrease in subjective value of a reward when the delay to that reward increases. In recent years, a growing number of studies on the neural correlates of temporal reward discounting have been conducted. This article focuses on functional magnetic resonance imaging (fMRI) studies on TD in humans. First, we describe the different types of tasks (also from behavioral studies) and the dependent variables. Subsequently, we discuss the evidence for three neurobiological models of TD: the dual-systems model, the single-system model and the self-control model. Further, studies in which nontraditional tasks (e.g., with nonmonetary rewards) were used to study TD are reviewed. Finally, we discuss the neural correlates of individual differences in discounting, and its development across the lifespan. We conclude that the evidence for each of the three neurobiological models of TD is mixed, in that all models receive (partial) support, and several studies provide support for multiple models. Because of large differences between studies in task design and analytical approach, it is difficult to draw a firm conclusion regarding which model provides the best explanation of the neural correlates of temporal discounting. We propose that some components of these models can complement each other, and future studies should test the predictions offered by different models against each other. Several future research directions are suggested, including studying the connectivity between brain regions in relation to discounting, and directly comparing the neural mechanisms involved in discounting of monetary and primary rewards. WIREs Cogn Sci 2013, 4:523-545. doi: 10.1002/wcs.1246 CONFLICT OF INTEREST: The authors have declared no conflicts of interest for this article. For further resources related to this article, please visit the WIREs website.

Steep temporal reward discounting in ADHD-Combined type: Acting upon feelings

Difficulty waiting plays a primary role in symptoms of attention-deficit/hyperactivity disorder (ADHD), in particular, impulsivity. Current theories suggest that relatively strong preferences for small immediate rewards as observed in ADHD-Combined type are the result of delay-related negative feelings. However, the measurement of difficulty waiting is typically limited to objective choices between small immediate and large delayed rewards. This study aimed at extending the measurement of difficulty waiting in ADHD-Combined type with ratings about subjective feelings. Children and adolescents (ages 6-17) with ADHD-Combined type (n=25), ADHD-Inattentive type (n=20) and matched typically developing participants (n=37) performed temporal reward discounting tasks, and completed a Visual Analogue Scale of subjectively experienced ease/difficulty waiting. Although those with ADHD-Combined type demonstrated relatively steep temporal reward discounting, as reported elsewhere (Scheres et al., 2010), there were no group differences for subjectively experienced ease/difficulty waiting. Additionally, correlations between subjective and objective measures of difficulty waiting were significantly higher in the ADHD-Combined type group than in the control group. These findings suggest that (a) those with ADHD-Combined type do not choose impulsively because they have more negative feelings about waiting than controls; (b) choices in the ADHD-Combined type group are more in accordance with/driven by their feelings than choices made by participants in the control group.

Dopaminergic involvement in effort-based but not impulsive reward processing in smokers

BackgroundA reduction in reward responsivity and an increase in temporal discounting of rewards are both evident in smokers during acute abstinence compared to satiation. However, it is not yet known whether these processes can be modulated pharmacologically in smokers, other than with nicotine or tobacco. MethodsA double-blind placebo controlled crossover design assessed the effects of 0.5mg pramipexole, a dopamine D2/D3 agonist, in smokers following 2h of abstinence. Reward responsivity was measured using an effort-based card sorting task. Temporal discounting of monetary reward was assessed using Area Under the Curve (AUC) analysis, and affective and subjective effects were indexed. ResultsOn placebo, smokers showed an equivalent speed of card sorting when a financial incentive was provided compared to when it was not. Conversely, more cards were sorted under rewarded compared to non-rewarded trials after pramipexole, indicating an improvement in reward responsivity. Temporal discounting of monetary reward was not affected by pramipexole. Drug treatment also decreased positive affect and increased drowsiness. ConclusionsA single dose of pramipexole can enhance effort-based reward responsivity, but does not alter temporal discounting in smokers. These findings highlight pharmacological correlates of reward processing deficits in nicotine dependence and offer potential targets for their treatment.