Discounting Of Rewards Research Articles

Adverse Parenting Is Indirectly Linked to Delayed Reward Discounting via Blunted RSA Reactivity: the Protective Role of a Shift-and-Persist Coping Strategy

Exposure to adverse parenting experiences, either from being reared under conditions of threat or deprivation, is a major source of chronic stress that is linked to youths’ dysregulated stress responses and impulsive decision-making. Many youth, however, adapt well and develop optimal decision-making despite the experience of adversity. Research has shown that the use of cognitive strategies to cope with stress, such as accepting and adapting to an acute stressor (i.e., shifting) and effortfully pursuing future goals (i.e., persisting), can mitigate the influence of adverse parenting experiences on impulsive decision-making and risk behaviors. However, scarce research examines the role of shift-and-persist in mitigating the neurobiological mechanisms underlying the effects of adverse parenting on youths’ decision-making. The current study investigated whether shift-and-persist strategy can buffer the indirect link between adverse parenting and youths’ impulsive decision-making via dysregulated stress reactivity gauged by blunted respiratory sinus arrhythmia (RSA) reactivity. Data were obtained from a longitudinal sample of mostly minority (75.2% African American) low-SES preadolescents and their primary caregivers (N = 101). The youths’ decision-making was indexed by an assessment of delayed reward discounting (DRD). Results indicated that the threat dimension of adverse parenting was significantly linked to youths’ blunted RSA reactivity. Further, employing a shift-and-persist coping strategy significantly mitigated the indirect effect of threatening parenting behaviors on youths’ impulsive DRD via blunted RSA reactivity. Findings from the current study can inform prevention and intervention programs that aim to promote the development of resilience among youth who experience adverse and threatening rearing environments.

Rationally inattentive intertemporal choice

Discounting of future rewards is traditionally interpreted as evidence for an intrinsic preference in favor of sooner rewards. However, temporal discounting can also arise from internal uncertainty in value representations of future events, if one assumes that noisy mental simulations of the future are rationally combined with prior beliefs. Here, we further develop this idea by considering how simulation noise may be adaptively modulated by task demands, based on principles of rational inattention. We show how the optimal allocation of mental effort can give rise to the magnitude effect in intertemporal choice. In a re-analysis of two prior data sets, and in another experiment, we reveal several behavioral signatures of this theoretical account, tying choice stochasticity to the magnitude effect. We conclude that some aspects of temporal discounting may result from a cognitively plausible adaptive response to the costs of information processing.

Intrasexually Competing Women Do Not Differentially Discount Future Appearance Enhancing Products

Men have been shown to discount future monetary rewards in several mating contexts, including intrasexual competition. Women, on the other hand, have not displayed differential discounting of future monetary rewards in intrasexual competition contexts. Here, we test the hypothesis that the failure to find delay discounting in women in intrasexual competition contexts may be due to the monetary nature of the reward. Women do not compete with other women for access to men based on monetary status; rather, women compete in other areas, including youthfulness and beauty. In a series of studies, we tested whether women will more steeply discount future appearance enhancing products following an intrasexual competition prime. Results indicated that the intrasexual competition prime did induce intrasexual competition, yet women did not show enhanced discounting a future gain for appearance enhancing products. We suggest that this failure to find enhanced discounting for a future gain may be due to the differential reproductive assurance for women, suggesting sensitivity to the different abundance of mating opportunities for women relative to men, as proposed by Bateman’s principle.

Temporal discounting does not influence body mass index

The prevalence of obesity has driven searches for cognitive or behavioural economic factors related to Body Mass Index (BMI). One candidate is delay discounting: those who prefer smaller sooner rewards over larger but later rewards are hypothesised to have higher BMI. The findings in the literature are mixed however, with meta analyses suggesting only a very small correlation between discounting and BMI. Here we present novel empirical data (N=381) and Bayesian analyses which suggest no such relationship between discounting of either monetary or weight loss rewards and BMI. We also find evidence against our novel proposal that discounting moderates the rate of BMI gain over time. We also present our data in the context of a random effects Bayesian meta-analytical result which does suggest the presence of a small correlation overall. The strength of the correlation is so weak (2.25% shared variance) that its practical significance may be minor to non existent. However because we found decisive evidence for unaccounted for study-level variance, due to study heterogeneity, we argue that we should treat such meta-analytic correlations with extreme caution. While the relationship between discounting and health outcomes such as BMI remain theoretically appealing, our empirical and meta-analytic results suggest we should be cautious in inferring a correlational, let alone a causal, role for discounting processes in driving BMI or moderating BMI gain with age.

Intact value-based decision-making during intertemporal choice in women with remitted anorexia nervosa? An fMRI study

Extreme restrictive food choice in anorexia nervosa is thought to reflect excessive self-control and/or abnormal reward sensitivity. Studies using intertemporal choice paradigms have suggested an increased capacity to delay reward in anorexia nervosa, and this may explain an unusual ability to resist immediate temptation and override hunger in the long-term pursuit of thinness. It remains unclear, however, whether altered delay discounting in anorexia nervosa constitutes a state effect of acute illness or a trait marker observable after recovery. We repeated the analysis from our previous fMRI investigation of intertemporal choice in acutely underweight patients with anorexia nervosa in a sample of weight-recovered women with anorexia nervosa (n = 36) and age-matched healthy controls (n = 36) who participated in the same study protocol. Follow-up analyses explored functional connectivity separately in both the weight-recovered/healthy controls sample and the acute/healthy controls sample. In contrast to our previous findings in acutely underweight patients with anorexia nervosa, we found no differences between weight-recovered patients with anorexia nervosa and healthy controls at either behavioural or neural levels. New analysis of data from the acute/healthy controls sample sample revealed increased coupling between dorsal anterior cingulate cortex and posterior brain regions as a function of decision difficulty, supporting the hypothesis of altered neural efficiency in the underweight state. This was a cross-sectional study, and the results may be task-specific. Although our results underlined previous demonstrations of divergent temporal reward discounting in acutely underweight patients with anorexia nervosa, we found no evidence of alteration in patients with weight-recovered anorexia nervosa. Together, these findings suggest that impaired valuebased decision-making may not constitute a defining trait variable or “scar” of the disorder.

Is it time? Episodic imagining and the discounting of delayed and probabilistic rewards in young and older adults

Remembering and imagining specific, personal experiences can help shape our decisions. For example, cues to imagine future events can reduce delay discounting (i.e., increase the subjective value of future rewards). It is not known, however, whether such cues can also modulate other forms of reward discounting, such as probability discounting (i.e., the decrease in the subjective value of a possible reward as the odds against its occurrence increase). In addition, it is unclear whether there are age-related differences in the effects of cueing on either delay or probability discounting. Accordingly, young and older adult participants were administered delay and probability discounting tasks both with and without cues to imagine specific, personally meaningful events. As expected, cued episodic imagining decreased the discounting of delayed rewards. Notably, however, this effect was significantly less pronounced in older adults. In contrast to the effects of cueing on delay discounting, personally relevant event cues had little or no effect on the discounting of probabilistic rewards in either young or older adults; Bayesian analysis revealed compelling support for the null hypothesis that event cues do not modulate the subjective value of probabilistic rewards. In sum, imagining future events appears only to affect decisions involving delayed rewards. Although the cueing effect is smaller in older adults, nevertheless, it likely contributes to how adults of all ages evaluate delayed rewards and thus, it is, in fact, about time.

Multidimensional latent structure of risk-related phenotypes in healthy young adults.

Risk-taking behavior can result in a range of maladaptive behaviors such as illicit substance use, unsafe driving, and high-risk sexual behavior. Perception of risk and preference for engaging in risky behaviors have been measured using both self-report measures and a range of behavioral tasks designed for the purpose, and these may predict future risk-taking behavior. However, the interrelationships between these measures and the latent constructs underlying them are poorly understood. In the present study, we examined data from over 1,000 men and women who completed measures of risk-related behaviors, including self-reports of perception of risk, propensity to engage in risky behaviors, and incentivized performance on tasks that involve risk. We conducted principal component analyses (PCAs) to understand the underlying latent structure of these measures. A PCA with the full sample revealed 5 distinct components, corresponding to measures of (a) health/ethical risks, (b) discounting of uncertain rewards, (c) risk of personal finances, (d) preferences in recreational hobbies and social interactions that involve risk, and (e) behavior involving risks in interpersonal interactions. Although we found sex differences on several of the measures, the sex-adjusted PCA components were similar to those of the unadjusted full sample PCA. These findings add to a growing literature revealing different components of the broad category of risk perception and risk-taking behaviors. A better understanding of the multidimensionality of risk preference will help lay the foundation for more refined measures, develop better predictors of future risk-taking behavior, and ultimately to study the genetic or other biological basis of risk-taking. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Steep Discounting of Future Rewards as an Impulsivity Phenotype: A Concise Review.

This chapter provides an overview over the behavioral economic index of impulsivity known as delay discounting. Specifically, delay discounting refers to an individual's preference for smaller immediate rewards over a larger delayed rewards. The more precipitously an individual discounts future rewards, the more impulsive they are considered to be. First, the chapter reviews the nature of delay discounting as a psychological process and juxtaposes it with nominally similar processes, including other facets of impulsivity. Second, the chapter reviews the links between delay discounting and numerous health behaviors, including addiction, attention deficit/hyperactivity disorder, and obesity. Third, the determinants of individual variation in delay discounting are discussed, including both genetic and environmental contributions. Finally, the chapter evaluates delay discounting as a potentially modifiable risk factor and the status of clinical interventions designed to reduce delay discounting to address deficits in self-control in a variety of maladaptive behaviors.

A Reinforcer Pathology perspective on relapse.

The Reinforcer Pathology theory proposes conditions under which drugs emerge as excessively preferred reinforcers compared to other available reinforcers among drug users. The theory highlights 2 key variables as important determinants of drug preference: (a) excessive preference for immediate rewards (high discounting of future rewards); and (b) excessive valuation for addictive reinforcers (e.g., drugs). Two iterations of the Reinforcer Pathology theory exist with the latest iteration (i.e., Reinforcer Pathology 2.0) specifying that the temporal window of reinforcer integration (measured by delay discounting) is a determinant of reinforcer value. The 2 iterations of the Reinforcer Pathology theory are described. A novel insight and understanding of abstinence and relapse from a Reinforcer Pathology perspective, limitations, and future directions are discussed. The Reinforcer Pathology theory continues the long-standing efforts to scientifically understand and better define novel concepts and methods to further translational research and improve treatment outcomes. Exploring the complementary relation between the Reinforcer Pathology perspective and other current approaches could have a valuable effect.

DISCOUNTING OF SHARED REWARDS IN PIGEONS

Pigeons chose between shared (communal) and unshared patches. In the shared patch, the pigeon would share 14 food pellets with zero to seven other pigeons. In the unshared patch, the pigeon had access to food pellets by itself, in an amount that depended on the given trial. The number of food pellets in the unshared patch was increased or decreased depending on the pigeon’s choices, and the indifference point (the subjective value of the shared reward), here the pigeon chose the shared and unshared patches equally often, was reached. The subjective value of the shared rewards decreased systematically as the number of pigeons sharing the rewards increased. The discounting of the shared rewards was well- described using both hyperbolic and exponential functions. The results suggest that food sharing is a discounting factor, and the mathematical function that describes the social discounting of hypothetical monetary rewards in humans described social discounting in pigeons receiving real rewards.

Adaptive early classification of temporal sequences using deep reinforcement learning

In this article, we address the problem of early classification (EC) of temporal sequences with adaptive prediction times. We frame EC as a sequential decision making problem and we define a partially observable Markov decision process (POMDP) fitting the competitive objectives of classification earliness and accuracy. We solve the POMDP by training an agent for EC with deep reinforcement learning (DRL). The agent learns to make adaptive decisions between classifying incomplete sequences now or delaying its prediction to gather more measurements. We adapt an existing DRL algorithm for batch and online learning of the agent’s action value function with a deep neural network. We propose strategies of prioritized sampling, prioritized storing and random episode initialization to address the fact that the agent’s memory is unbalanced due to (1): all but one of its actions terminate the process and thus (2): actions of classification are less frequent than the action of delay. In experiments, we show improvements in accuracy induced by our specific adaptation of the algorithm used for online learning of the agent’s action value function. Moreover, we compare two definitions of the POMDP based on delay reward shaping against reward discounting. Finally, we demonstrate that a static naive deep neural network, i.e. trained to classify at static times, is less efficient in terms of accuracy against speed than the equivalent network trained with adaptive decision making capabilities.

Vigor of reaching, walking, and gazing movements: on the consistency of interindividual differences.

Movement vigor is an important feature of motor control that is thought to originate from cortico-basal ganglia circuits and processes shared with decision-making, such as temporal reward discounting. Accordingly, vigor may be related to one's relationship with time, which may, in turn, reflect a general trait-like feature of individuality. While significant interindividual differences of vigor have been typically reported for isolated motor tasks, little is known about the consistency of such differences across tasks and movement effectors. Here, we assessed interindividual consistency of vigor across reaching (both dominant and nondominant arm), walking, and gazing movements of various distances within the same group of 20 participants. Given distinct neural pathways and biomechanical specificities of each movement modality, a significant consistency would corroborate the trait-like aspect of vigor. Vigor scores for dominant and nondominant arm movements were found to be highly correlated across individuals. Vigor scores of reaching and walking were also significantly correlated across individuals, indicating that people who reach faster than others also tend to walk faster. At last, vigor scores of saccades were uncorrelated with those of reaching and walking, reaffirming that the vigor of stimulus-elicited eye saccades is distinct. These findings highlight the trait-like aspect of vigor for reaching movements with either arms and, to a lesser extent, walking.NEW & NOTEWORTHY Robust interindividual differences of movement vigor have been reported for arm reaching and saccades. Beyond biomechanics, personality trait-like characteristics have been proposed to account for those differences. Here, we examined for the first time the consistency of interindividual differences of vigor during dominant/nondominant arm reaching, walking, and gazing to assess the trait-like aspect of vigor. We found a significant consistency of vigor within our group of individuals for all tested tasks/effectors except saccades.

Cocaine-related alterations in fronto-parietal gray matter volume correlate with trait and behavioral impulsivity

BackgroundChronic cocaine use is associated with structural brain abnormalities within prefrontal regions implicated in impulsivity. Despite high levels of impulsivity among persons who use cocaine, it is not known how reductions in gray matter volume (GMV) may relate to trait and behavioral measures of impulsivity. MethodsThe sample included 39 active cocaine users (COC+) and 40 controls with no history of cocaine use (COC−). Participants had a brain scan on a 3 T MRI machine and completed out-of-scanner measures of trait impulsivity and delayed reward discounting. Whole-brain voxel-based morphometry was used to compare GMV between COC+ and COC−. Within regions that differed between groups, voxelwise correlations were conducted to examine the relationship between GMV and impulsivity. ResultsIn a whole-brain analysis, COC+ had broad reductions in GMV compared to COC− in bilateral frontal, parietal, occipital, and cerebellar regions. Lower GMV correlated with trait impulsivity in lateral prefrontal regions and with delayed reward discounting in medial prefrontal regions, while lower GMV correlated with both measures in the posterior parietal cortex. COC+ demonstrated significantly higher impulsivity than COC− on all measures, but the nature of the correlation with GMV was similar in both groups. ConclusionsReflecting the multi-faceted nature of impulsivity, these results show that trait and behavioral measures of impulsivity map differentially onto altered brain morphology. While the brain-behavior patterns were similar in COC+ and COC−, suggesting that impulsivity varies on a continuous spectrum, cocaine-related abnormalities in frontal-parietal brain systems may contribute to heightened impulsivity.

Vagal signaling and the somatic marker hypothesis: The effect of transcutaneous vagal nerve stimulation on delay discounting is modulated by positive mood

Controlling impulsivity and delaying gratifications are key features of effective self-control. Delay Discounting (DD) indexes the ability to delay rewards and previous research has shown that discounting is influenced by affective states such as mood. According to the Somatic Marker Hypothesis (SMH), afferent somatic signals, such as mood, are carried by the vagus and can influence decision making. In the current study, we employed transcutaneous vagus nerve stimulation (tVNS), a novel non-invasive brain stimulation technique that stimulates the auricular branch of the afferent vagus nerve (located in the outer ear), to assess its effects on decision impulsivity, while taking into account individuals' mood and resting-state HRV as a possible confounding factor. Employing a within-subjects cross-over design, 94 participants received active or sham tVNS while performing delay discounting in two separate sessions. As compared to sham, active tVNS increased discounting, but only for individuals reporting lower positive mood, regardless of the level of negative mood reported. We evidence that the effect of tVNS on reward discounting depends on the level of positive mood. This result suggests that positive mood state might be a proxy of task-relevant arousal, likely influencing the effectiveness of afferent vagal stimulation on self-control processes, as temporal discounting.

DEATH ANXIETY AND FINANCIAL DECISION-MAKING IN AGING: A STUDY FROM THE HUMAN CONNECTOME PROJECT AGING (HCP-A)

While research addressing late-life death anxiety (the fear of death or the dying process) has focused on end-of-life care decision-making, few have studied the effect of late-life death anxiety on financial decision-making. This is particularly relevant to financial decision-making as older adults are more vulnerable to fraud and deception. The aim of this study was to determine how age and death anxiety affect financial decision-making in a sample of older adults of 60-93 years of age (N = 102), who participated in the HCP-A project at UCLA. To study this relationship, we used a delayed reward discounting task to model financial decision-making, where higher rates of discounting indicate a greater preference for immediate, smaller monetary rewards and lower rates of discounting indicate more future-oriented planning. To account for age-related cognitive decline, cognitive functioning was assessed using the NIH Toolbox. We hypothesized that the presence of death anxiety will increase discounting of future rewards in older adults. Results from a univariate ANOVA showed an interaction between age, death anxiety, and delayed reward discounting. Specifically, older adults with self-reported death anxiety showed greater preference for immediate, smaller monetary rewards. By controlling for cognition, these findings suggest that death anxiety moderates decision-making in late-life adults and may add to our understanding of why older adults are more susceptible to financial abuse. These results suggest a need to consider death anxiety as a moderating variable when developing and implementing policies and services that are geared towards older adults.

The effect of executive function on adherence with a cardiac secondary prevention program and its interaction with an incentive-based intervention

Participation in secondary prevention programs such as cardiac rehabilitation (CR) reduces morbidity, mortality, and hospitalizations while improving quality of life. Executive function (EF) is a complex set of cognitive abilities that control and regulate behavior. EF predicts many health-related behaviors, but how EF interacts with interventions to improve treatment adherence is not well understood. The objective of this study is to examine if EF predicts CR treatment adherence and how EF interacts with an intervention to improve adherence. Data were collected from 2013 to 2018 in Vermont, USA. 130 Medicaid-enrolled individuals who had experienced a qualifying cardiac event were enrolled in a controlled clinical trial and randomized 1:1 to receive financial incentives for completing secondary prevention sessions or to usual care. In this secondary analysis, effects of EF on CR adherence (defined as completing ≥30/36 sessions) were examined in 112 participants (57 usual care, 55 intervention) who completed an EF battery. Delay-discounting, a measure of impulsivity, predicted CR adherence (p = 0.01) and interacted with the incentive intervention, such that those who exhibited greater discounting of future rewards benefitted more from the intervention than those who discounted less (F(1, 104) = 5.23, p = 0.02). Better cognitive flexibility, measured with the trail-making-task, also predicted CR adherence (p = 0.02). While EF has been associated with adherence to a variety of treatment regimens, this interaction between an incentive-based intervention to promote treatment adherence and EF is novel. This work illustrates the value of considering individual differences in EF when designing and implementing interventions to promote health-related behavior change.

Abnormal cognitive effort allocation and its association with amotivation in first-episode psychosis.

Abnormal effort-based decision-making represents a potential mechanism underlying motivational deficits (amotivation) in psychotic disorders. Previous research identified effort allocation impairment in chronic schizophrenia and focused mostly on physical effort modality. No study has investigated cognitive effort allocation in first-episode psychosis (FEP). Cognitive effort allocation was examined in 40 FEP patients and 44 demographically-matched healthy controls, using Cognitive Effort-Discounting (COGED) paradigm which quantified participants' willingness to expend cognitive effort in terms of explicit, continuous discounting of monetary rewards based on parametrically-varied cognitive demands (levels N of N-back task). Relationship between reward-discounting and amotivation was investigated. Group differences in reward-magnitude and effort-cost sensitivity, and differential associations of these sensitivity indices with amotivation were explored. Patients displayed significantly greater reward-discounting than controls. In particular, such discounting was most pronounced in patients with high levels of amotivation even when N-back performance and reward base amount were taken into consideration. Moreover, patients exhibited reduced reward-benefit sensitivity and effort-cost sensitivity relative to controls, and that decreased sensitivity to reward-benefit but not effort-cost was correlated with diminished motivation. Reward-discounting and sensitivity indices were generally unrelated to other symptom dimensions, antipsychotic dose and cognitive deficits. This study provides the first evidence of cognitive effort-based decision-making impairment in FEP, and indicates that decreased effort expenditure is associated with amotivation. Our findings further suggest that abnormal effort allocation and amotivation might primarily be related to blunted reward valuation. Prospective research is required to clarify the utility of effort-based measures in predicting amotivation and functional outcome in FEP.

Demand characteristics in episodic future thinking II: The role of cues and cue content in changing delay discounting.

Episodic future thinking (EFT), or imagining experiencing one's future, reduces discounting of future rewards, but the mechanisms of this effect are unclear. We examined the role of cues to engage in EFT, possible demand characteristics inherent to those cues (prompting awareness of the hypothesized effects of EFT), and changes in temporal horizon (how far one thinks into the future) in these reductions in delay discounting. In Experiment 1, cues prompting participants to engage in EFT during the discounting task were presented or withheld: EFT decreased discounting only when cues were present. In a time-perspective task in which cues were not presented, EFT did not broaden temporal horizon; however, episodic recent thinking (the putative control activity) shortened temporal horizons. In Experiment 2, cues were manipulated in a way designed to make them either more theoretically relevant (prompting episodic thought) or more prone to demand characteristics (prompting attention to the correspondence in delays between EFT and discounting tasks). Only theoretically relevant cues reduced delay discounting. These results do not support the hypothesis that EFT reduces delay discounting because of demand characteristics. Instead, they suggest limitations on the generalizability of EFT effects in uncued contexts and suggest that mechanisms other than temporal horizon may be responsible for its effects. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Detection of a High-Turnover Serotonin Circuit in the Mouse Brain Using Mass Spectrometry Imaging.

SummaryMonoamine neurotransmitters are released by specialized neurons regulating behavioral, motor, and cognitive functions. Although the localization of monoaminergic neurons in the brain is well known, the distribution and kinetics of monoamines remain unclear. Here, we generated a murine brain atlas of serotonin (5-HT), dopamine (DA), and norepinephrine (NE) levels using mass spectrometry imaging (MSI). We found several nuclei rich in both 5-HT and a catecholamine (DA or NE) and identified the paraventricular nucleus of the thalamus (PVT), where 5-HT and NE are co-localized. The analysis of 5-HT fluctuations in response to acute tryptophan depletion and infusion of isotope-labeled tryptophan in vivo revealed a close kinetic association between the raphe nuclei, PVT, and amygdala but not the other nuclei. Our findings imply the existence of a highly dynamic 5-HT-mediated raphe to PVT pathway that likely plays a role in the brain monoamine system.

Hunger increases delay discounting of food and non-food rewards

How do our valuation systems change to homeostatically correct undesirable psychological or physiological states, such as those caused by hunger? There is evidence that hunger increases discounting for food rewards, biasing choices towards smaller but sooner food reward over larger but later reward. However, it is not understood how hunger modulates delay discounting for non-food items. We outline and quantitatively evaluate six possible models of how our valuation systems modulate discounting of various commodities in the face of the undesirable state of being hungry. With a repeated-measures design, an experimental hunger manipulation, and quantitative modeling, we find strong evidence that hunger causes large increases in delay discounting for food, with an approximately 25% spillover effect to non-food commodities. The results provide evidence that in the face of hunger, our valuation systems increase discounting for commodities, which cannot achieve a desired state change as well as for those commodities that can. Given that strong delay discounting can cause negative outcomes in many non-food (consumer, investment, medical, or inter-personal) domains, the present findings suggest caution may be necessary when making decisions involving non-food outcomes while hungry.