Reward Size Research Articles

Measure for measure: Effects of money exposure, reward size and loss aversion on cheating

Measure for measure: Effects of money exposure, reward size and loss aversion on cheating

When should bees be flower constant? An agent-based model highlights the importance of social information and foraging conditions.

Many bee species show flower constancy, that is, a tendency to visit flowers of one type during a foraging trip. Flower constancy is important for plant reproduction, but the benefits of constancy to bees is unclear. Social bees, which often use communication about food sources, show particularly strong flower constancy. We aimed to better understand the benefits of flower constancy in social bees and how these benefits depend on foraging conditions. We hypothesised that sharing social information increases the benefits of flower constancy because social foragers share information selectively about high-quality food sources, thereby reducing the need to sample alternatives. We developed an agent-based model that allowed us to simulate bee colonies with and without communication and flower constancy in different foraging environments. By varying key environmental parameters, such as food source numbers and reward size, we explored how the costs and benefits of flower constancy depend on the foraging landscape. Flower constancy alone performed poorly in all environments, while indiscriminate flower choice was often the most successful strategy. However, communication improved the performance of flower constant colonies considerably in most environments. This combination was particularly successful when high-quality food sources were abundant and competition was weak. Our findings help explain why social bees tend to be more flower constant than solitary bees and suggest that flower constancy can be an adaptive strategy in social bees. Simulations suggest that anthropogenic changes of foraging landscapes will have different effects on the foraging performance of bees that vary in flower constancy.

Value-Based Pricing for Patent-Protected Medicines Over the Product Life Cycle: Pricing Anomalies in the “Age of Cures” and Their Implications for Dynamic Efficiency

ObjectivesConventional cost-effectiveness analysis (CEA) for the value-based pricing of new medicines largely ignores the implications of limited market exclusivity (ie, patent-protection periods plus any exclusivity granted by regulators). This paper explores the implications of this methodological shortcoming, which produces several pricing anomalies with potentially unintended effects on research and development (R&D) incentives. MethodsWe illustrate these implications by comparing 4 stylized examples of increasing complexity, from short-term cures for acute conditions to long-term cures for rare, health-catastrophic conditions. Results(1) Conventional-CEA will project a different result than an adjusted CEA that considers generic or biosimilar entry; (2) free and flexible pricing of long-term treatments (eg, statins for hypercholesterolemia) or repeated-dose cures (eg, insulin for type 1 diabetes) for chronic conditions will likely result in predictable price increases at the end of the exclusivity period that may be perceived as unjustified or unsupported; and (3) one-time administration “cures” (eg, gene therapy for spinal muscular atrophy) have the potential to allocate a large share of the social surplus to the manufacturer over the product lifetime, which may or may not be dynamically efficient per se, but may also inadvertently disadvantage the development of valuable long-term treatments or repeated-dose cures for chronic conditions. ConclusionsWe highlight the need for additional research on long-term solutions to these issues that would aim to promote dynamically efficient global R&D. More work is needed on the following: (1) relationships between social surplus allocation and the amount and composition of global R&D, as we may be as likely to be encouraging excessive R&D in some areas as to be undersupplying it in others; and (2) relating the size of the surplus reward to R&D cost and, thus, the return on investment.

Developmental risk sensitivity theory: the effects of socio-economic status on children's risky gain and loss decisions.

Evolutionary developmental theories propose that early environments shape human risk preferences. Developmental risk sensitivity theory (D-RST) focuses on the plasticity of risk preferences during childhood and makes predictions about the effect of reward size based on a child's social environment. By contrast, prospect theory predicts risk aversion for gains and risk seeking for losses regardless of environment or status. We presented 4 to 10-year-olds (n = 194) with a set of trials in which they chose between a certain amount and a chance to receive more or nothing. Two trials were equal expected value choices that differed by stake size and two were unequal expected value choices. Children either received gain trials or loss trials. Social environment was assessed using socio-economic status (SES) and subjective social status. Results confirmed the predictions of D-RST for gains based on SES. Children from lower-SES families differentiated between the high- and low-value trials and made more risky decisions for the high-value reward compared with higher-SES children. Children from higher-SES families were more risk averse for both trial types. Decisions for loss trials did not conform completely to either theory. We discuss the results in relation to evolutionary developmental theories.

Bumblebees retrieve only the ordinal ranking of foraging options when comparing memories obtained in distinct settings.

Are animals' preferences determined by absolute memories for options (e.g. reward sizes) or by their remembered ranking (better/worse)? The only studies examining this question suggest humans and starlings utilise memories for both absolute and relative information. We show that bumblebees' learned preferences are based only on memories of ordinal comparisons. A series of experiments showed that after learning to discriminate pairs of different flowers by sucrose concentration, bumblebees preferred flowers (in novel pairings) with (1) higher ranking over equal absolute reward, (2) higher ranking over higher absolute reward, and (3) identical qualitative ranking but different quantitative ranking equally. Bumblebees used absolute information in order to rank different flowers. However, additional experiments revealed that, even when ranking information was absent (i.e. bees learned one flower at a time), memories for absolute information were lost or could no longer be retrieved after at most 1 hr. Our results illuminate a divergent mechanism for bees (compared to starlings and humans) of learned preferences that may have arisen from different adaptations to their natural environment.

Critical periods when dopamine controls behavioral responding during Pavlovian learning.

Learning the association between rewards and predictive cues is critical for appetitive behavioral responding. The mesolimbic dopamine system is thought to play an integral role in establishing these cue-reward associations. The dopamine response to cues can signal differences in reward value, though this emerges only after significant training. This suggests that the dopamine system may differentially regulate behavioral responding depending on the phase of training. The purpose of this study was to determine whether antagonizing dopamine receptors elicited different effects on behavior depending on the phase of training or the type of Pavlovian task. Separate groups of male rats were trained on Pavlovian tasks in which distinct audio cues signaled either differences in reward size or differences in reward rate. The dopamine receptor antagonist flupenthixol was systemically administered prior to either the first ten sessions of training (acquisition phase) or the second ten sessions of training (expression phase), and we monitored the effect of these manipulations for an additional ten training sessions. We identified acute effects of dopamine receptor antagonism on conditioned responding, the latency to respond, and post-reward head entries in both Pavlovian tasks. Interestingly, dopamine receptor antagonism during the expression phase produced persistent deficits in behavioral responding only in rats trained on the reward size Pavlovian task. Together, our results illustrate that dopamine's control over behavior in Pavlovian tasks depends upon one's prior training experience and the information signaled by the cues.

EE462 Value-Based Pricing for Patent-Protected Medicines: Pricing Anomalies in the "Age of Cures” and Their Implications for Dynamic Efficiency

The application of a conventional cost-effectiveness analysis (C-CEA) for the value-based pricing of new medicines has largely ignored the implications of a fixed patent life or data exclusivity period. We believe that this creates several anomalies: 1) C-CEA will produce a different result than an actual CEA that considers generic or biosimilar entry; 2) the pricing of long-term treatments for chronic conditions and “repeated-dose cures” in a free and flexible pricing environment is likely to result in predictable price increases at the end of the exclusivity period that will upset patients and the public; and 3) “one-time administration cures” have the potential to capture a greater share of the consumer surplus over the product lifetime, which may be efficient per se, but also may inadvertently disadvantage the development of valuable long-term therapies for chronic conditions or repeated-dose cures. Emergence of the Institute for Clinical and Economic Review, which is arguably a form of “privately supplied regulation” of US drug prices and relies on C-CEA, may be producing inadvertent anomalies that distort prices and research and development (R&D) incentives. A demand exists for long-term solutions to these issues that would promote dynamically efficient global R&D. More work is needed on 1) the relationship between surplus reward and the amount and composition of global R&D, because we may be as likely to be encouraging excessive R&D in some areas and undersupplying it in others, and 2) the size of the surplus reward to R&D cost and to return on investment.

Neuronal correlates of selective attention and effort in visual area V4 are invariant of motivational context.

Task demands can differentially engage two fundamental attention components: selectivity (spatial bias) and effort (total nonselective attentional intensity). The relative contributions and interactions of these components in modulating neuronal signals remain unknown. We recorded V4 neurons while monkeys’ spatially selective attention and effort were independently controlled by adjusting either task difficulty or reward size at two locations. Neurons were robustly modulated by either selective attention or effort. Notably, increasing overall effort to improve performance at a distant site reduced neuronal responses even when performance was unchanged for receptive field stimuli. This interaction between attentional selectivity and effort was evident in single-trial spiking and can be explained by divisive normalization of spatially distributed behavioral performance at the single-neuron level. Changing motivation using task difficulty or reward produced indistinguishable effects. These results provide a cellular-level mechanism of how attention components integrate to modulate sensory processing in different motivational contexts.

Declared Support and Clientelism

Recent studies of clientelism predominantly focus on how elites use rewards to influence vote choices and turnout. This article shifts attention toward citizens and their choices beyond the ballot box. Under what conditions does clientelism influence citizens’ decisions to express political preferences publicly? When voters can obtain post-election benefits by declaring support for victorious candidates, their choices to display political paraphernalia on their homes or bodies may reflect more than just political preferences. We argue that various factors—such as the size of rewards and punishments, the competitiveness of the election, and whether multiple candidates employ clientelism—affect citizens’ propensity to declare support in response to clientelist inducements. Building on insights from fieldwork, formal analyses reveal how and why such factors can distort patterns of political expression observed during electoral campaigns. We conduct an experiment in Brazil, which predominantly corroborates predictions about declared support and clientelism.

Tribute to E. J. Capaldi: Celebration of a Psychological Scientist

Abstract E. J. (John) Capaldi (1928–2020) made numerous contributions to experimental psychology in his long career at the University of Texas at Austin and Purdue University. He was a pioneer in the area of animal learning and cognition, known for his sequential theory of partial reinforcement extinction effects. His research in this area and in memory and counting phenomena was conducted for the most part with rats running in straight alley mazes under various sequences of trial outcomes (e.g., reward, nonreward, variations in reward size). John's other interests included the sequential theory applied to Pavlovian conditioning, evolutionary theory, and psychology of science. More generally, John was a scientist to the core who served as a role model to those who knew him. His career centered on the values of science and rigorous critical thinking based in empirical data, coupled with a curiosity and openness to different and new ideas in psychology.

Reward signals in the cerebellum: Origins, targets, and functional implications.

The cerebellum has long been proposed to play a role in cognitive function, although this has remained controversial. This idea has received renewed support with the recent discovery that signals associated with reward can be observed in the cerebellar circuitry, particularly in goal-directed learning tasks involving an interplay between the cerebellar cortex, basal ganglia, and cerebral cortex. Remarkably, a wide range of reward contingencies-including reward expectation, delivery, size, and omission-can be encoded by specific circuit elements in a manner that reflects the microzonal organization of the cerebellar cortex. The facts that reward signals have been observed in both the mossy fiber and climbing fiber input pathways to the cerebellar cortex and that their convergence may trigger plasticity in Purkinje cells suggest that these interactions may be crucial for the role of the cerebellar cortex in learned behavior. These findings strengthen the emerging consensus that the cerebellum plays a pivotal role in shaping cognitive processing and suggest that the cerebellum may combine both supervised learning and reinforcement learning to optimize goal-directed action. We make specific predictions about how cerebellar circuits can work in concert with the basal ganglia to guide different stages of learning.

Sex Differences in Behavioral Responding and Dopamine Release during Pavlovian Learning.

Learning associations between cues and rewards require the mesolimbic dopamine system. The dopamine response to cues signals differences in reward value in well trained animals. However, these value-related dopamine responses are absent during early training sessions when cues signal differences in the reward rate. These findings suggest cue-evoked dopamine release conveys differences between outcomes only after extensive training, though it is unclear whether this is unique to when cues signal differences in reward rate, or whether this is also evident when cues signal differences in other value-related parameters such as reward size. To address this, we used a Pavlovian conditioning task in which one audio cue was associated with a small reward (one pellet) and another audio cue was associated with a large reward (three pellets). We performed fast-scan cyclic voltammetry to record changes in dopamine release in the nucleus accumbens of male and female rats throughout learning. While female rats exhibited higher levels of conditioned responding, a faster latency to respond, and elevated post-reward head entries relative to male rats, there were no sex differences in the dopamine response to cues. Multiple training sessions were required before cue-evoked dopamine release signaled differences in reward size. Reward-evoked dopamine release scaled with reward size, though females displayed lower reward-evoked dopamine responses relative to males. Conditioned responding related to the decrease in the peak reward-evoked dopamine response and not to cue-evoked dopamine release. Collectively, these data illustrate sex differences in behavioral responding as well as in reward-evoked dopamine release during Pavlovian learning.

Individual Differences in Frustrative Nonreward Behavior for Sucrose in Rats Predict Motivation for Fentanyl under Progressive Ratio.

Frustrative nonreward (FN) is a construct in the Negative Valence Systems domain of the Research Domain Criteria (RDoC) from the National Institute of Mental Health. An organism’s response to frustrating situations (e.g., inability to obtain an expected reward) has broad implications for a variety of neuropsychiatric conditions, including substance use disorders. The current project developed a first of its kind rat operant behavioral model of FN based loosely on the human Point Subtraction Aggression Paradigm (PSAP). The current study shows that individual differences in FN for sucrose pellets are consistent across sessions at baseline and that the task is sensitive to reward size in male rats. More importantly, high FN behavior for sucrose predicts early “breaking” for intravenous fentanyl self-administration under a progressive ratio (PR) schedule. These results solidify frustration/ FN as an important factor for substance use disorders in addition to craving, impulsivity, and habit.

The divergent impact of reward magnitude on goal eagerness and effort investment

When attempting to motivate an individual, does increasing the magnitude of the reward always have a positive impact on one’s effort investment? Through six experiments, we demonstrate that although individuals working for a large (vs. small) reward are more eager to commence the task (Studies 1–5) and more likely to persist when encountering an opportunity to terminate the pursuit (Study 2), the intensity of their effort investment in the goal remains relatively unchanged regardless of the magnitude of the reward (Studies 1–5). The insensitivity of effort investment to the magnitude of the reward disappears in the following contexts: when the participants are reminded of the reward size during the goal striving phase (Study 3), when individuals make a plan prior to commencing the task (Study 4), and when a partial reward is allocated to each goal-directed action (Study 5). This divergent impact appears because the psychological processes and information that govern pre-initiation goal eagerness may differ from those that control post-initiation effort investment.

Single-trial dynamics of hippocampal spatial representations are modulated by reward value

Reward value is known to modulate learning speed in spatial memory tasks, but little is known about its influence on the dynamical changes in hippocampal spatial representations. Here, we monitored the trial-to-trial changes in hippocampal place cell activity during the acquisition of place-reward associations with varying reward size. We show a faster reorganization and stabilization of the hippocampal place map when a goal location is associated with a large reward. The reorganization is driven by both rate changes and the appearance and disappearance of place fields. The occurrence of hippocampal replay activity largely followed the dynamics of changes in spatial representations. Replay patterns became more selectively tuned toward behaviorally relevant experiences over the course of learning via the refined contributions of specific cell subpopulations. These results suggest that high reward value enhances memory retention by accelerating the formation and stabilization of the hippocampal cognitive map and selectively enhancing its reactivation during learning.

Single caudate neurons encode temporally discounted value for formulating motivation for action.

The term 'temporal discounting' describes both choice preferences and motivation for delayed rewards. Here we show that neuronal activity in the dorsal part of the primate caudate head (dCDh) signals the temporally discounted value needed to compute the motivation for delayed rewards. Macaque monkeys performed an instrumental task, in which visual cues indicated the forthcoming size and delay duration before reward. Single dCDh neurons represented the temporally discounted value without reflecting changes in the animal's physiological state. Bilateral pharmacological or chemogenetic inactivation of dCDh markedly distorted the normal task performance based on the integration of reward size and delay, but did not affect the task performance for different reward sizes without delay. These results suggest that dCDh is involved in encoding the integrated multi-dimensional information critical for motivation.

Dividend or No Dividend in Delegated Blockchain Governance: A Game Theoretic Analysis

Delegated blockchain governance is the frontier of blockchain design issues that is to improve the scalability of blockchain networks. Delegated proof of stake (DPoS) blockchains such as EOS must select a few super nodes for transaction verification. In particular, the blockchain community has debated recently on whether dividend should be allowed when electing super nodes, which might be considered as unethical or unfair, leading to an open research topic and a new research gap that has theoretical value for both academia and industry. In this paper, we build a theoretical framework to study how dividend affects user decisions and welfare in a DPoS blockchain. Based on game theory, we propose a three-player Hotelling model with two policies to study the behaviors of voters and candidates. We first use a static game analysis to show that the benefits of dividend for voters and for candidates vary with the size of reward and there exists an interval, in which the zero dividend policy would be good for the welfare of both candidates and voters. Then, we use an evolutionary game analysis to examine the process dynamics of super node selection, we find that the campaign strategy of candidates has to do with the size of platform reward given to the candidates by the blockchain platform. Especially in the asymmetric case, we find that higher level of platform reward turns to benefit candidates with large number of votes even more. Our findings are instrumental for designing dividend policies in DPoS blockchains and can stimulate more potential research in blockchain governance.

The ecology of competition: A theory of risk-reward environments in adaptive decision making.

In many choice environments, risks and rewards-or probabilities and payoffs-seem tightly coupled such that high payoffs only occur with low probabilities. An adaptive mind can exploit this association by, for instance, using a potential reward's size to infer the probability of obtaining it. However, a mind can only adapt to and exploit an environmental structure if it is ecologically reliable, that is if it is frequent and recurrent. We develop the competitive risk-reward ecology theory (CET) that establishes how the ecology of competition can make the association of high rewards with low probabilities ubiquitous. This association occurs because of what is known as the ideal free distribution (IFD) principle. The IFD states that competitors in a landscape of resource patches distribute themselves proportionally to the gross total amount of resources in the patches. CET shows how this principle implies a risk-reward structure: an inverse relationship between probabilities and payoffs. It also identifies boundary conditions for the risk-reward structure, including heterogeneity of resources, computational limits of competitors, and scarcity of resources. Finally, a set of empirical studies (N = 1,255) demonstrate that people's beliefs map onto properties predicted by CET and change as a function of the environment. In sum, grounding people's inferences in CET demonstrates how the behaviors of a boundedly rational mind can be better predicted once accounts of the mind and the environment are fused. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Neuronal Activity in the Posterior Cingulate Cortex Signals Environmental Information and Predicts Behavioral Variability during Trapline Foraging.

Animals engage in routine behavior to efficiently navigate their environments. This routine behavior may be influenced by the state of the environment, such as the location and size of rewards. The neural circuits tracking environmental information and how that information impacts decisions to deviate from routines remain unexplored. To investigate the representation of environmental information during routine foraging, we recorded the activity of single neurons in posterior cingulate cortex (PCC) in 2 male monkeys searching through an array of targets in which the location of rewards was unknown. Outside the laboratory, people and animals solve such traveling salesman problems by following routine traplines that connect nearest-neighbor locations. In our task, monkeys also deployed traplining routines; but as the environment became better known, they deviate from them despite the reduction in foraging efficiency. While foraging, PCC neurons tracked environmental information but not reward and predicted variability in the pattern of choices. Together, these findings suggest that PCC may mediate the influence of information on variability in choice behavior.SIGNIFICANCE STATEMENT Many animals seek information to better guide their decisions and update behavioral routines. In our study, subjects visually searched through a set of targets on every trial to gather two rewards. Greater amounts of information about the distribution of rewards predicted less variability in choice patterns, whereas smaller amounts predicted greater variability. We recorded from the posterior cingulate cortex, an area implicated in the coding of reward and uncertainty, and discovered that these neurons signaled the expected information about the distribution of rewards instead of signaling expected rewards. The activity in these cells also predicted the amount of variability in choice behavior. These findings suggest that the posterior cingulate helps direct the search for information to augment routines.

Momentary subjective well-being depends on learning and not reward.

Subjective well-being or happiness is often associated with wealth. Recent studies suggest that momentary happiness is associated with reward prediction error, the difference between experienced and predicted reward, a key component of adaptive behaviour. We tested subjects in a reinforcement learning task in which reward size and probability were uncorrelated, allowing us to dissociate between the contributions of reward and learning to happiness. Using computational modelling, we found convergent evidence across stable and volatile learning tasks that happiness, like behaviour, is sensitive to learning-relevant variables (i.e. probability prediction error). Unlike behaviour, happiness is not sensitive to learning-irrelevant variables (i.e. reward prediction error). Increasing volatility reduces how many past trials influence behaviour but not happiness. Finally, depressive symptoms reduce happiness more in volatile than stable environments. Our results suggest that how we learn about our world may be more important for how we feel than the rewards we actually receive.