Small Reward Research Articles

Waiting and working for rewards: Attention-Deficit/Hyperactivity Disorder is associated with steeper delay discounting linked to amygdala activation, but not with steeper effort discounting

ObjectiveChildren and adolescents with ADHD have a relatively strong preference for smaller immediate rewards over larger delayed rewards (steep delay discounting). It is unknown whether such steep discounting of rewards is specific for delayed rewards, i.e., supporting the delay aversion account of ADHD, or whether it is also present for effortful rewards, i.e., representing general reward insensitivity. Therefore, this study examined behavioral and BOLD responses during delay discounting (DD) and effort discounting (ED) in ADHD. MethodThirty adolescents with ADHD and 28 controls (12–17 years) were scanned while performing a DD-ED task (fMRI findings were based on 21 and 25 participants, respectively). During DD, participants were presented with a series of choices between a small reward delivered immediately and a larger reward delivered after 5–25s. During ED, participants were presented with choices between a small reward that was delivered after exerting 15% of their maximal hand grip strength and a larger reward delivered after exerting 30–90% of their strength. ResultsAnalyses on the subjective values of delayed and effortful rewards and on the Area Under the discounting Curves (AUCs) indicated that adolescents with ADHD showed steeper discounting than controls for DD, but not for ED. This was accompanied by a slightly stronger delay dose–response relationship in the amygdala for adolescents with ADHD who reported to be more delay averse in daily life. ConclusionTogether, these results–steeper DD in the ADHD group and a stronger delay dose–response relationship in the amygdala, while no evidence for group differences in ED was found–support the delay aversion account of ADHD.

Consummatory successive negative contrast in rats: Assessment through orofacial taste reactivity responses

Rats exposed to a downshift from a large reward (32% sucrose) to a small reward (4% sucrose) show an abrupt and transient reduction in consumption in comparison with animals that are always exposed to the small reward. This effect is called consummatory Successive Negative Contrast (cSNC) and involves negative affective consequences that lead to an aversive emotional, cognitive and behavioral state of frustration. There are few previous works that have investigated the hedonic alterations that undergo an unexpected incentive devaluation. The hedonic impact of fluids can be reliably assessed by examining the orofacial reactions of acceptance and rejection in the taste reactivity (TR) test. This study addressed in male adult Wistar rats the hedonic impact of incentive devaluation in an adapted cSNC protocol. Specifically, the orofacial responses to a sucrose solution infused into the oral cavity were measured. It was observed that animals exposed to reward devaluation, from a 32% to a 4% sucrose solution, showed a decrease in the duration of appetitive responses (tongue protrusions, mouth movements, paw licks) as compared with subjects which only experienced the low concentration of sucrose. The results are consistent with the hypothesis that incentive devaluation in a cSNC not only results in reduced intake, but also in a reduction in the hedonic value or palatability of the devalued reward.

An improved within-session self-adjusting delay discounting procedure for the study of choice impulsivity in rats.

Delay-discounting procedures involving choice between small immediate rewards and large delayed rewards are used to study impulsivity in rodents. Improving existing procedures may provide new insights into the neurobiological mechanisms underlying decision-making processes. To develop a novel delay-discounting procedure that adjusts the delay value within individual sessions based on the rat's most recent choices. Compared to previously developed procedure, we required a more consistent demonstration of preference, five consecutive choices of the large or small reward, a criterion that is more likely to reflect deliberate choice by the animal, as opposed to two consecutive choices. In addition, delays were changed in steps of 5s (rather than 1s), because 5-s increments should be more easily discriminated and may produce a more distinct effect on choice. We characterized the procedure behaviorally by manipulating the duration of the session and the consecutive choice criterion, and we investigated the stability of the behavior upon interruption of training. We also characterized the procedure pharmacologically by investigating the effects of dopaminergic compounds. Our procedures allowed obtaining two complementary measures of delay discounting: (1) the percentage of choices of the delay option and (2) the mean adjusting delay, an index of the delay that animals choose more frequently. We found that our procedure rapidly establishes a baseline of choice behavior that remains stable over time and is highly sensitive to manipulations of the dopaminergic system. This procedure may provide a useful tool for investigating the neurobiology of inter-temporal choice and decision-making.

Effects of reward and punishment on learning from errors in smokers

BackgroundPunishing errors facilitates adaptation in healthy individuals, while aberrant reward and punishment sensitivity in drug-dependent individuals may change this impact. Many societies have institutions that use the concept of punishing drug use behavior, making it important to understand how drug dependency mediates the effects of negative feedback for influencing adaptive behavior. MethodsUsing an associative learning task, we investigated differences in error correction rates of dependent smokers, compared with controls. Two versions of the task were administered to different participant samples: One assessed the effect of varying monetary contingencies to task performance, the other, the presence of reward as compared to avoidance of punishment for correct performance. ResultsWhile smokers recalled associations that were rewarded with a higher value 11% more often than lower rewarded locations, they did not correct higher punished locations more often. Controls exhibited the opposite pattern. The three-way interaction between magnitude, feedback type and group was significant, F(1,48) = 5.288, p =0.026, ɳ2p =0.099. Neither participant group corrected locations offering reward more often than those offering avoidances of punishment. The interaction between group and feedback condition was not significant, F(1,58) = 0.0, p =0.99, ɳ2p =0.001. ConclusionsThe present results suggest that smokers have poorer learning from errors when receiving negative feedback. Moreover, larger rewards reinforce smokers' behavior stronger than smaller rewards, whereas controls made no distinction. These findings support the hypothesis that dependent smokers may respond to positively framed and rewarded anti-smoking programs when compared to those relying on negative feedback or punishment.

Differential effects of real versus hypothetical monetary reward magnitude on risk-taking behavior and brain activity

Human decisions are more easily affected by a larger amount of money than a smaller one. Although numerous studies have used hypothetical money as incentives to motivate human behavior, the validity of hypothetical versus real monetary rewards remains controversial. In the present study, we used event-related potential (ERP) with the balloon analogue risk task to investigate how magnitudes of real and hypothetical monetary rewards modulate risk-taking behavior and feedback-related negativity (FRN). Behavioral data showed that participants were more risk averse after negative feedback with increased magnitude of real monetary rewards, while no behavior differences were observed between large and small hypothetical monetary rewards. Similarly, ERP data showed a larger FRN in response to negative feedback during risk taking with large compared to small real monetary rewards, while no FRN differences were observed between large and small hypothetical monetary rewards. Moreover, FRN amplitude differences correlated with risk-taking behavior changes from small to large real monetary rewards, while such correlation was not observed for hypothetical monetary rewards. These findings suggest that the magnitudes of real and hypothetical monetary rewards have differential effects on risk-taking behavior and brain activity. Real and hypothetical money incentives may have different validity for modulating human decisions.

Two facets of patience in young children: Waiting with and without an explicit reward

Patience, or the ability to tolerate delay, is typically studied using delay of gratification (DoG) tasks. However, among other factors (e.g., type of reward), the use of a reward to test patience is affected by an individual’s motivation to obtain the reward (e.g., degree of preference for the small vs. large reward). In addition, DoG tasks do not assess the extent to which an individual can wait in the absence of an explicit reward—or what we term “patience as a virtue.” Accordingly, the current study used a new measure of patience—the “pure waiting paradigm”—in which 3- to 5-year-old children waited 3 min with nothing to do and with no explicit reward. We then examined the relation between performance on this task (as assessed by children’s spontaneous patient behaviors) and performance on two DoG tasks (candy and video rewards). Significant correlations were found between DoG performance and patient behaviors in the pure waiting paradigm, especially when controlling for motivation. These results and methodology show for the first time a direct link between patience as a virtue and DoG performance and also provide new insights about the study of patience in children.

Dissociating absolute and relative reward- and punishment-related electrocortical processing: An event-related potential study

The meaning of reward and punishment signals depends on context. Receiving a small reward where a larger reward could have been obtained can be considered a punishment, while a small loss in the context of avoiding a larger loss can be experienced as a reward. The aim of this study was to investigate the electrophysiological processes associated with absolute and relative reward and punishment signals. Twenty healthy right-handed volunteers performed a decision-making task and were instructed to judge which of two neutral objects was the most expensive. The received outcome was presented together with the non-received outcome for the alternative choice. The feedback-related potentials P200, FRN and P300 were recorded in response to absolute (i.e., received) outcome and relative (i.e., received in the context of the alternative) outcome. Absolute rewards yielded higher P200 amplitudes as compared to relative rewards, while the P200 amplitude was largest for relative as compared to absolute punishments. The P300 amplitude showed a main effect of valence with larger amplitudes for more positive relative and absolute outcomes. No effect of absolute or relative outcome was observed for the feedback-related negativity (FRN). Our findings suggest distinct processes associated with context-dependent and context-independent processing during feedback processing.

Financial Rewards Do Not Stimulate Coproduction: Evidence from Two Experiments

AbstractWestern governments are increasingly trying to stimulate citizens to coproduce public services by, among other strategies, offering them financial incentives. However, there are competing views on whether financial incentives stimulate coproduction. While some argue that financial incentives increase citizens' willingness to coproduce, others suggest that incentives decrease their willingness (i.e., crowding out). To test these competing expectations, the authors designed a set of experiments that offered subjects a financial incentive to assist municipalities in helping refugees integrate. The experiment was first conducted among university students within a laboratory setting. Then, the initial findings were replicated and extended among a general adult sample. Results suggest that small financial rewards have no effect: they neither increase nor decrease people's willingness to coproduce. When the offered amount is increased substantially, willingness to coproduce increases only marginally. Hence, financial incentives are not a very cost‐efficient instrument to stimulate coproduction.

Domestic demand-side response on district heating networks

ABSTRACT Results are presented from a field study that deployed demand-shifting technology on a sample of 28 homes connected to a district heating (DH) network in England over the winter of 2015/16. The study’s aim was to improve the load factor of the participating households. This has the potential to improve the attractiveness of DH and accelerate the roll out of DH networks. Capital costs are lowered by reducing required boiler capacity and pipework sizes. Operational costs are reduced by increasing the coverage of the primary plant and reducing heat losses and pumping energy. The interventions were found to increase the load factor of the participating homes from 0.29 to 0.44. This led to an increased energy demand of approximately 3%; however, the estimated network cost savings exceed this amount. While some participants noted the altered operation of their heating systems and expressed concern, the majority indicated they would be willing to participate in a commercial scheme for a small financial reward. In addition to specific insights for the deployment of demand shifting on DH networks, the results provide general lessons for the utilization of building thermal inertia for demand shifting.

Effect of Motion Perception on Intertemporal Choice Is Associated With the Altered Time Perception.

Intertemporal choice refers to the choice between receiving a small immediate reward or a large delayed one. Previous studies have indicated that time perception plays a critical role in the intertemporal choice, and it could be affected by the features of the target stimulus in the time reproduction task, such as speed of movement and state of motion. However, there is no evidence about whether backward or forward motion perception could alter the intertemporal choice. Thus, in our current study, 29 participants were asked to perform two tasks in a random order. One was the intertemporal choice task after viewing videos containing moving elements with forward/backward directions as well as stationary ones, and another was the time perception task. We found that the discounting rate in intertemporal choice was significantly larger in backward motion condition than in both forward motion and stationary conditions, indicating that backward motion perception made participants more myopic (specifically, more likely to choose the smaller immediate reward instead of the large delayed one) during their decision-makings. Meanwhile, participants overestimated the temporal duration in a time perception task in backward motion condition compared to the other two conditions. Furthermore, the Pearson's correlation analysis showed that the changes of the intertemporal choice induced by backward motion perception could be associated with the altered time perception. As far as we know, we provide the first evidence on influence of motion perception on the intertemporal choice as well as its possible cognitive correlates, which extend previous studies on cognitive basis of the intertemporal choices.

Activation of the cannabinoid system in the nucleus accumbens affects effort-based decision making

Effort-based decision making addresses how we make an action choice based on an integration of action and goal values. The nucleus accumbens (NAc) is implicated in allowing an animal to overcome effort constraints to obtain greater benefits, and it has been previously shown that cannabis derivatives may affect such processes. Therefore, in this study, we intend to evaluate the involvement of the cannabinoid system in the entire NAc on effort-based decision making. Rats were trained in a T-maze cost-benefit decision making the task in which they could choose either to climb a barrier to obtain a large reward in one arm or run into the other arm without a barrier to obtaining a small reward. Following training, the animals were bilaterally implanted with guide cannulae in the NAc. On test day, rats received cannabinoid agonist (Win 55,212-2; 2, 10 and 50μM) and/or antagonist (AM251; 45μM), afterward percentage of large reward choice and latency of reward attainment were investigated. Results revealed that the administration of cannabinoid agonist led to decrease of large reward choice percentage such that the animals preferred to receive a small reward with low effort instead of receiving a large reward with high effort. The administration of antagonist solely did not affect effort-based decision making, but did attenuate the Win 55,212-2-induced impairments in effort allocation. In agonist-treated animals, the latency of reward collection increased. Moreover, when the effort was equated on both arms, the animals returned to choosing large reward showing that obtained results were not caused by spatial memory impairment. Our finding suggested that activation of the cannabinoid system in the NAc impaired effort-based decision making and led to rats were less willing to invest the physical effort to gain large reward.

Interplay between cooperation-enhancing mechanisms in evolutionary games with tag-mediated interactions

Cooperation is fundamental for the long-term survival of biological, social, and technological networks. Previously, mechanisms for the enhancement of cooperation, such as network reciprocity, have largely been studied in isolation and with often inconclusive findings. Here, we present an evolutionary, multiagent-based, and spatially explicit computer model to specifically address the interactive interplay between such mechanisms. We systematically investigate the effects of phenotypic diversity, network structure, and rewards on cooperative behavior emerging in a population of reproducing artificial decision makers playing tag-mediated evolutionary games. Cooperative interactions are rewarded such that both the benefits of recipients and costs of donators are affected by the reward size. The reward size is determined by the number of cooperative acts occurring within a given reward time frame. Our computational experiments reveal that small reward frames promote unconditional cooperation in populations with both low and high diversity, whereas large reward frames lead to cycles of conditional and unconditional strategies at high but not at low diversity. Moreover, an interaction between rewards and spatial structure shows that relative to small reward frames, there is a strong difference between the frequency of conditional cooperators populating rewired versus non-rewired networks when the reward frame is large. Notably, in a less diverse population, the total number of defections is comparable across different network topologies, whereas in more diverse environments defections become more frequent in a regularly structured than in a rewired, small-world network of contacts. Acknowledging the importance of such interaction effects in social dilemmas will have inevitable consequences for the future design of cooperation-enhancing protocols in large-scale, distributed, and decentralized systems such as peer-to-peer networks.

Duration of extinction trials as a determinant of instrumental extinction in terrestrial toads (Rhinella arenarum).

Instrumental learning guides behavior toward resources. When such resources are no longer available, approach to previously reinforced locations is reduced, a process called extinction. The present experiments are concerned with factors affecting the extinction of acquired behaviors in toads. In previous experiments, total reward magnitude in acquisition and duration of extinction trials were confounded. The present experiments were designed to test the effects of these factors in factorial designs. Experiment 1 varied reward magnitude (900, 300, or 100s of water access per trial) and amount of acquisition training (5 or 15 daily trials). With total amount of water access equated in acquisition, extinction with large rewards was faster (longer latencies in 900/5 than 300/15), but with total amount of training equated, extinction with small rewards was faster (longer latencies in 100/15 than 300/15). Experiment 2 varied reward magnitude (1200 or 120s of water access per trial) while holding constant the number of acquisition trials (5 daily trials) and the duration of extinction trials (300s). Extinction performance was lower with small, rather than large reward magnitude (longer latencies in 120/300 than in 1200/300). Thus, instrumental extinction depends upon the amount of time toads are exposed to the empty goal compartment during extinction trials.

Can social news websites pay for content and curation? The SteemIt cryptocurrency model

SteemIt is a Reddit-like social news site that pays members for posting and curating content. It uses micropayments backed by a tradeable currency, exploiting the Bitcoin cryptocurrency generation model to finance content provision in conjunction with advertising. If successful, this paradigm might change the way in which volunteer-based sites operate. This article investigates 925,092 new members’ first posts for insights into what drives financial success in the site. Initial blog posts on average received US$0.01, although the maximum accrued was US$20,680.83. Longer, more sentiment-rich or more positive comments with personal information received the greatest financial reward in contrast to more informational or topical content. Thus, there is a clear financial value in starting with a friendly introduction rather than immediately attempting to provide useful content, despite the latter being the ultimate site goal. Follow-up posts also tended to be more successful when more personal, suggesting that interpersonal communication rather than quality content provision has driven the site so far. It remains to be seen whether the model of small typical rewards and the possibility that a post might generate substantially more are enough to incentivise long-term participation or a greater focus on informational posts in the long term.

Functional connectivity between anterior cingulate cortex and orbitofrontal cortex during value-based decision making

There is a growing body of evidence showing that the anterior cingulate (ACC) and the orbitofrontal (OFC) cortex are both essential for reinforcement-guided decision making. Focusing on functional connectivity approach through coherence, we studied whether communication between the ACC and OFC through neural synchronization is a necessary stage for performing value-based decision making. We used a T-maze task with a differential reward (Large vs. small reward) and cost (long vs. short waiting time) and simultaneously recorded local field potentials (LFP) from the ACC and OFC. Task-dependent synchronization in theta/low beta (4–20 Hz) frequency bands were observed between areas when rats chose the higher over the lower reward. This synchronization was significantly poorer when rats chose lower rewards or passively performed the task. High-gamma (80–100 Hz) synchrony between areas was also observed, however, it was not dependent on the animal’s decision. Our results propose that synchronization between the ACC and OFC in the low-frequency range is necessary during value-based decision making.

The effect of hydrocortisone administration on intertemporal choice

Intertemporal choices – decisions involving trade-offs of outcomes at different points in time – are often made under stress. Stress activates the hypothalamic–pituitary–adrenal (HPA) axis, resulting in the release of corticosteroids. Recent studies provide evidence that corticosteroids can induce rapid non-genomic effects focused on immediate resolution of the stressful situation, followed by slower genomic effects focused on long-term recovery after stress. It remains unknown, however, how corticosteroids affect intertemporal choice. We randomly assigned healthy men to receive either 10 mg hydrocortisone or a placebo before measuring intertemporal choice. To target time-dependent effects, hydrocortisone was administered either 195 or 15 min before choice elicitation, while a placebo was administered at the other timepoint, in a double-blind design. Intertemporal choices were elicited by offering subjects decisions between small rewards available sooner vs. large rewards available later. We demonstrate a time-dependent effect of hydrocortisone administration on intertemporal choice: when tested 15 min after hydrocortisone administration, subjects showed a strongly increased preference for the small, soon reward over the larger, delayed reward. In contrast, this effect was not found when testing occurred 195 min after hydrocortisone administration. Together, these results suggest that the physiological effects of acute, but not delayed, stress may increase temporal discounting.

Adolescent Exposure to Methylphenidate Increases Impulsive Choice Later in Life.

Background: The psychostimulant methylphenidate (MPH) is known to temporarily reduce impulsive choice and promote self-control. What is not sufficiently understood is how repeated treatment with MPH affects impulsive choice in the long run, and whether any such effect is contingent on exposure at certain developmental stages.Methods: Using an animal model for impulsive choice, we examined first whether giving MPH through early adolescence alters delay discounting, an operational measure of impulsive choice, later in adulthood. We then tested whether equivalent long-term effects are observed if exposure to the drug occurred during adulthood. Starting on postnatal day 25 or postnatal day 60, male rats received one of a range of doses of MPH for 10 consecutive days. Twenty-six days later, all rats were trained to choose between a lever that produced a small immediate reward and a lever that produced a large reward after a range of delays.Results: Rats showed a long-term decrease in the selection of the delayed larger reward when treated with moderate doses of MPH during early adolescence, but not when treated with the lower or higher doses. In contrast, no differences were observed in the selection of the delayed larger reward in animals that were treated with various doses of MPH during adulthood.Conclusions: Our findings suggest effects of MPH on impulsive choice that are contingent on dosage and on the developmental period of exposure. When administered during adolescence, moderate doses of MPH increase impulsive choice long after the end of treatment, whereas these same doses administered during adulthood were without effect

An Initial Examination of Performance on Two Versions of the Iowa Gambling Task.

To examine differences between two versions of the Iowa Gambling Task (IGT). A total of 282 undergraduate students completed one of two versions of the IGT: the original version from 1994 (n = 132), or the 2007 version available through Psychological Assessment Resources (PAR) (n = 150). PAR (2007 version) IGT participants decided more advantageously (i.e., selected more from the small immediate reward but long-term positive/gain decks than the large immediate reward but long-term negative/loss decks) than original IGT participants during Trials 21-60. This difference was likely due to fewer Deck B selections by the PAR IGT participants during the early (Trials 1-40) and later (Trials 41-100) trials. The PAR IGT may result in a greater ability to make future-oriented, advantageous decisions more quickly than on the original IGT. Implications for future assessment of decision-making impairments in clinical and research settings are discussed.

Preschoolers’ Distributive Justice When They Distribute Few Total Rewards in a Third‐Party Task

AbstractFive‐year‐old children were presented with a story in which one character made three origami stars and another made nine. Then the participants were asked to distribute small rewards to each character (“Three” or “Four” candies), and were told to use all the candies (“All” instruction), or that “You can leave some candies, if you don't want to use all the candies (‘Partial’ instruction).” In Study 1, almost all children distributed the candies equally (Four). Some participants (28.1%) refused the All instruction for Three, and then equally distributed the candies. They conducted equality allocations in three or all four of the conditions, giving a ratio for “Thorough Egalitarians” of 20–30%. Study 2 used looking‐time measures to assess participants’ judgment. Children looked longer at the amounts of production of two characters in condition Three than in condition Four. Allocation patterns were almost identical to Study 1, but the condition “Partial‐Three” results were not reproduced. The paper discusses the types of egalitarian behavior and the associated production and reward quantities observed.

The Promise and Pitfalls of Using Crowdsourcing in Research Prioritization for Back Pain: Cross-Sectional Surveys.

BackgroundThe involvement of patients in research better aligns evidence generation to the gaps that patients themselves face when making decisions about health care. However, obtaining patients’ perspectives is challenging. Amazon’s Mechanical Turk (MTurk) has gained popularity over the past decade as a crowdsourcing platform to reach large numbers of individuals to perform tasks for a small reward for the respondent, at small cost to the investigator. The appropriateness of such crowdsourcing methods in medical research has yet to be clarified.ObjectiveThe goals of this study were to (1) understand how those on MTurk who screen positive for back pain prioritize research topics compared with those who screen negative for back pain, and (2) determine the qualitative differences in open-ended comments between groups.MethodsWe conducted cross-sectional surveys on MTurk to assess participants’ back pain and allow them to prioritize research topics. We paid respondents US $0.10 to complete the 24-point Roland Morris Disability Questionnaire (RMDQ) to categorize participants as those “with back pain” and those “without back pain,” then offered both those with (RMDQ score ≥7) and those without back pain (RMDQ <7) an opportunity to rank their top 5 (of 18) research topics for an additional US $0.75. We compared demographic information and research priorities between the 2 groups and performed qualitative analyses on free-text commentary that participants provided.ResultsWe conducted 2 screening waves. We first screened 2189 individuals for back pain over 33 days and invited 480 (21.93%) who screened positive to complete the prioritization, of whom 350 (72.9% of eligible) did. We later screened 664 individuals over 7 days and invited 474 (71.4%) without back pain to complete the prioritization, of whom 397 (83.7% of eligible) did. Those with back pain who prioritized were comparable with those without in terms of age, education, marital status, and employment. The group with back pain had a higher proportion of women (234, 67.2% vs 229, 57.8%, P=.02). The groups’ rank lists of research priorities were highly correlated: Spearman correlation coefficient was .88 when considering topics ranked in the top 5. The 2 groups agreed on 4 of the top 5 and 9 of the top 10 research priorities.ConclusionsCrowdsourcing platforms such as MTurk support efforts to efficiently reach large groups of individuals to obtain input on research activities. In the context of back pain, a prevalent and easily understood condition, the rank list of those with back pain was highly correlated with that of those without back pain. However, subtle differences in the content and quality of free-text comments suggest supplemental efforts may be needed to augment the reach of crowdsourcing in obtaining perspectives from patients, especially from specific populations.