Reward Size Research Articles

Simulation of Coherent Excavator Operations in Earthmoving Tasks Based on Reinforcement Learning

Earthwork operations are critical to construction projects, with their safety and efficiency influenced by factors such as operator skill and working hours. Pre-construction simulation of these operations is essential for optimizing outcomes, providing key training for operators and improving safety awareness and operational efficiency. This study introduces a hierarchical cumulative reward mechanism that decomposes complex operational behaviors into simple, fundamental actions. The mechanism prioritizes reward function design elements, including order, size, and form, thus simplifying excavator operation simulation using reinforcement learning (RL) and enhancing policy network reusability. A 3D model of a hydraulic excavator was constructed with six degrees of freedom—comprising the boom, arm, bucket, base, and left/right tracks. The Proximal Policy Optimization (PPO) algorithm was applied to train four basic behaviors: scraping, digging, throwing, and turning back. Motion simulation was successfully achieved using diggable terrain resources. Results demonstrate that the simulated excavator, powered by RL neural networks, can perform coordinated actions and maintain smooth operational performance. This research offers practical implications by rapidly illustrating the full operational process before construction, delivering immersive movies, and enhancing worker safety and operational efficiency.

Incentivizing interdependent tasks: Evidence from a real-effort experiment

Workers are often required to jointly work on interdependent tasks that are cognitively complex. Yet, relatively little empirical evidence exists on how to effectively incentivize the workforce in such settings. Using a novel real-effort experiment, we assess the effects of team-based incentives (TBI) and individual-based incentives (IBI) on productivity. We provide evidence that TBI yield higher productivity than IBI, both when holding the incentives for the worker constant, and when holding the marginal cost of output for the firm constant. Various measures point towards workers engaging in lower quality work under IBI. We also provide new evidence that performance does not necessarily increase with the size of the monetary rewards. Overall, these results underscore the benefits of incentivizing the workforce in teams in settings in which workers’ payoff are interdependent and tasks are complex.

Cracking and Packing Information about the Features of Expected Rewards in the Orbitofrontal Cortex.

The orbitofrontal cortex (OFC) is crucial for tracking various aspects of expected outcomes, thereby helping to guide choices and support learning. Our previous study showed that the effects of reward timing and size on the activity of single units in OFC were dissociable when these attributes were manipulated independently ( Roesch et al., 2006). However, in real-life decision-making scenarios, outcome features often change simultaneously, so here we investigated how OFC neurons in male rats integrate information about the timing and identity (flavor) of reward and respond to changes in these features, according to whether they were changed simultaneously or separately. We found that a substantial number of OFC neurons fired differentially to immediate versus delayed reward and to the different reward flavors. However, contrary to the previous study, selectivity for timing was strongly correlated with selectivity for identity. Taken together with the previous research, these results suggest that when reward features are correlated, OFC tends to "pack" them into unitary constructs, whereas when they are independent, OFC tends to "crack" them into separate constructs. Furthermore, we found that when both reward timing and flavor were changed, reward-responsive OFC neurons showed unique activity patterns preceding and during the omission of an expected reward. Interestingly, this OFC activity is similar and slightly preceded the ventral tegmental area dopamine (VTA DA) activity observed in a previous study ( Takahashi et al., 2023), consistent with the role of OFC in providing predictive information to VTA DA neurons.

Neurons of Macaque Frontal Eye Field Signal Reward-Related Surprise.

The frontal eye field (FEF) plays a well-established role in the control of visual attention. The strength of an FEF neuron's response to a visual stimulus presented in its receptive field is enhanced if the stimulus captures spatial attention by virtue of its salience. A stimulus can be rendered salient by cognitive factors as well as by physical attributes. These include surprise. The aim of the present experiment was to determine whether surprise-induced salience would result in enhanced visual-response strength in the FEF. Toward this end, we monitored neuronal activity in two male monkeys while presenting first a visual cue predicting with high probability that the reward delivered at the end of the trial would be good or bad (large or small) and then a visual cue announcing the size of the impending reward with certainty. The second cue usually confirmed but occasionally violated the expectation set up by the first cue. Neurons responded more strongly to the second cue when it violated than when it confirmed expectation. The increase in the firing rate was accompanied by a decrease in spike-count correlation as expected from capture of attention. Although both good surprise and bad surprise induced enhanced firing, the effects appeared to arise from distinct mechanisms as indicated by the fact that the bad-surprise signal appeared at a longer latency than the good-surprise signal and by the fact that the strength of the two signals varied independently across neurons.

Synchronous Measurements of Extracellular Action Potentials and Neurochemical Activity with Carbon Fiber Electrodes in Nonhuman Primates.

Measuring the dynamic relationship between neuromodulators, such as dopamine, and neuronal action potentials is imperative to understand how these fundamental modes of neural signaling interact to mediate behavior. We developed methods to measure concurrently dopamine and extracellular action potentials (i.e., spikes) in monkeys. Standard fast-scan cyclic voltammetric (FSCV) electrochemical (EChem) and electrophysiological (EPhys) recording systems are combined and used to collect spike and dopamine signals, respectively, from an array of carbon fiber (CF) sensors implanted in the monkey striatum. FSCV requires the application of small voltages at the implanted sensors to measure redox currents generated from target molecules, such as dopamine. These applied voltages create artifacts at neighboring EPhys measurement sensors which may lead to misclassification of these signals as physiological spikes. Therefore, simple automated temporal interpolation algorithms were designed to remove these artifacts and enable accurate spike extraction. We validated these methods using simulated artifacts and demonstrated an average spike recovery rate of 84.5%. We identified and discriminated cell type-specific units in the monkey striatum that were shown to correlate to specific behavioral task parameters related to reward size and eye movement direction. Synchronously recorded spike and dopamine signals displayed contrasting relations to the task variables, suggesting a complex relationship between these two modes of neural signaling. Future application of our methods will help advance our understanding of the interactions between neuromodulator signaling and neuronal activity, to elucidate more detailed mechanisms of neural circuitry and plasticity mediating behaviors in health and in disease.

Neurobiological foundations and clinical relevance of effort-based decision-making.

Applying effort-based decision-making tasks provides insights into specific variables influencing choice behaviors. The current review summarizes the structural and functional neuroanatomy of effort-based decision-making. Across 39 examined studies, the review highlights the ventromedial prefrontal cortex in forming reward-based predictions, the ventral striatum encoding expected subjective values driven by reward size, the dorsal anterior cingulate cortex for monitoring choices to maximize rewards, and specific motor areas preparing for effort expenditure. Neuromodulation techniques, along with shifting environmental and internal states, are promising novel treatment interventions for altering neural alterations underlying decision-making. Our review further articulates the translational promise of this construct into the development, maintenance and treatment of psychiatric conditions, particularly those characterized by reward-, effort- and valuation-related deficits.

Pupillary Responses Reflect Dynamic Changes in Multiple Cognitive Factors During Associative Learning in Primates.

Associative learning involves complex interactions of multiple cognitive factors. While adult subjects can articulate these factors verbally, for model animals such as macaques, we rely on behavioral outputs. In our study, we used pupillary responses as an alternative measure to capture these underlying cognitive changes. We recorded the dynamic changes in the pupils of three male macaques when they learned the associations between visual stimuli and reward sizes under the classical Pavlovian experimental paradigm. We found that during the long-term learning process, the gradual changes in the pupillary response reflect the changes in the cognitive state of the animals. The pupillary response can be explained by a linear combination of components corresponding to multiple cognitive factors. These components reflect the impact of visual stimuli on the pupils, the prediction of reward values associated with the visual stimuli, and the macaques' understanding of the current experimental reward rules. The changing patterns of these factors during interday and intraday learning clearly demonstrate the enhancement of current reward-stimulus association and the weakening of previous reward-stimulus association. Our study shows that the dynamic response of pupils can serve as an objective indicator to characterize the psychological changes of animals, understand their learning process, and provide important tools for exploring animal behavior during the learning process.

Exploring the Factors that Drive Clinical Negligence Claims: Stated Preferences of Those Who Have Experienced Unintended Harm.

Better understanding of the factors that influence patients to make a financial claim for compensation is required to inform policy decisions. This study aimed to assess the relative importance of factors that influence those who have experienced a patient safety incident (PSI) to make a claim for compensation. Participants completed an online discrete choice experiment (DCE) involving 10 single profile tasks where they chose whether or not to file a claim. DCE data were modelled using logistic, mixed logit and latent class regressions; scenario analyses, external validity, and willingness to accept were also conducted. A total of 1029 participants in the United Kingdom responded to the survey. An appropriate apology and a satisfactory investigation reduced the likelihood of claiming. Respondents were more likely to claim if they could hold those responsible accountable, if the process was simple and straightforward, if the compensation amount was higher, if the likelihood of compensation was high or uncertain, if the time to receive a decision was quicker, and if they used the government compensation scheme. Men are more likely to claim for low impact PSIs. The actions taken by the health service after a PSI, and people's perceptions about the probability of success and the size of potential reward, can influence whether a claim is made. Results show the importance of giving an appropriate apology and conducting a satisfactory investigation. This stresses the importance around how patients are treated after a PSI in influencing the clinical negligence claims that are made.

Financial incentives to stop smoking: Potential financial consequences of different reward schedules.

Financial incentives to stop smoking (FISS) programs have been implemented internationally to encourage people who smoke to quit smoking. However, such programs require that the financial reward structure and its resulting effects on smoking quit rates are considered. We analyzed a number of scenarios for FISS reward schedules for current smoking individuals in Ireland, with a view to identify the potential implications in terms of financial consequences and expected effects. Using national QuitManager services data 2021-2023, we defined smoking quit rates for smokers currently using the national Health Services Executive stop smoking services in Ireland. Smoking quit rates at 4, 12 and 52 weeks were defined, and additionally defined by sex, age and education level. Using scenarios assuming different FISS reward sizes, structures and targeted population sub-groups, we estimated the number of additional quitters, budget impact, and incremental cost-effectiveness ratio. A FISS program, if implemented for a cohort of 3500 smokers can result in a budget impact ranging €250000 - €870000. The cost-effectiveness trade-off between different payment schedules and the expected effect size suggested that FISS are cost-effective even at a moderate effect size. A FISS program implemented to approximately 20000 smokers nationally would cost between €2.0 million and €4.8 million, subject to the chosen reward schedule. Across social groups, FISS is more cost-effective for females, individuals in the youngest age group, and individuals with a medium level of education. This analysis highlights the importance of considering different FISS schedules and potential quit effects, when designing such programs. We highlight that FISS programs should be targeted at certain social groups to achieve highest long-term smoking cessation rates. We also identified important challenges that decision-makers face when designing the reward structure of FISS programs. The acceptability or otherwise of the FISS structures may differ among stakeholders and should be explored.

Dollar or percentage? The effect of reward presentation on referral likelihood

PurposeAddressing the significant differences between referral programs and traditional promotional marketing, this paper aims to investigate and examine the impact of how reward-related information is presented within referral programs and how it interacts with reward size and reward allocation.Design/methodology/approachThis study adopts framing effect and equity theory to build the relationship between reward presentation, reward size and reward allocation. Then, two scenario-based experimental studies are designed and conducted on Amazon Mechanical Turk.FindingsThe results show that there is no direct impact of reward presentation on referral likelihood, while the effect relies on reward size. As the levels of reward size increase, the referral likelihood gradually shifts from percentage form to dollar form as perceived size mediates the interaction effect on referral likelihood. Further, adding information about reward allocation also indicate the different impacts of equity and inequity on influencing the above findings.Originality/valueThe study contributes to the literature by introducing reward presentation and emphasizes its impact on individual’s behavior decisions in the context of referral programs. This study extends and broadens the scope and effectiveness of the framing effect on traditional promotional marketing strategies, while also bridging the gap in the literature by examining the combined role of information about rewards.

The role of risk preference in referral reward programs

ABSTRACT This study explores the impacts of the referrer’s risk preference and reward size on the number of referrals and type of friends referred (i.e., tie strength). Risk preference (i.e., appetite for risk) is found to be related positively to the number of referrals and the percentage of weak ties reached, regardless of the reward. The effect differences between risk preference groups are greater if a medium-sized reward is offered, however, mainly due to the greater concavity exhibited by people with lower risk preference in their utility function.

Dopamine receptor activation regulates reward expectancy signals during cognitive control in primate prefrontal neurons

Dopamine neurons respond to reward-predicting cues but also modulate information processing in the prefrontal cortex essential for cognitive control. Whether dopamine controls reward expectation signals in prefrontal cortex that motivate cognitive control is unknown. We trained two male macaques on a working memory task while varying the reward size earned for successful task completion. We recorded neurons in lateral prefrontal cortex while simultaneously stimulating dopamine D1 receptor (D1R) or D2 receptor (D2R) families using micro-iontophoresis. We show that many neurons predict reward size throughout the trial. D1R stimulation showed mixed effects following reward cues but decreased reward expectancy coding during the memory delay. By contrast, D2R stimulation increased reward expectancy coding in multiple task periods, including cueing and memory periods. Stimulation of either dopamine receptors increased the neurons’ selective responses to reward size upon reward delivery. The differential modulation of reward expectancy by dopamine receptors suggests that dopamine regulates reward expectancy necessary for successful cognitive control.

Atomoxetine and reward size equally improve task engagement and perceptual decisions but differently affect movement execution

Our ability to engage and perform daily activities relies on balancing the associated benefits and costs. Rewards, as benefits, act as powerful motivators that help us stay focused for longer durations. The noradrenergic (NA) system is thought to play a significant role in optimizing our performance. Yet, the interplay between reward and the NA system in shaping performance remains unclear, particularly when actions are driven by external incentives (reward). To explore this interaction, we tested four female rhesus monkeys performing a sustained Go/NoGo task under two reward sizes (low/high) and three pharmacological conditions (saline and two doses of atomoxetine, a NA reuptake inhibitor: ATX-0.5 mg/kg and ATX-1 mg/kg).We found that increasing either reward or NA levels equally enhanced the animal's engagement in the task compared to low reward saline; the animals also responded faster and more consistently under these circumstances. Notably, we identified differences between reward size and ATX. When combined with ATX, high reward further reduced the occurrence of false alarms (i.e., incorrect go trials on distractors), implying that it helped further suppress impulsive responses. In addition, ATX (but not reward size) consistently increased movement duration dose-dependently, while high reward did not affect movement duration but decreased its variability. We conclude that noradrenaline and reward modulate performance, but their effects are not identical, suggesting differential underlying mechanisms. Reward might energize/invigorate decisions and action, while ATX might help regulate energy expenditure, depending on the context, through the NA system.

When Cash is Not King: An Examination of the Relative Effectiveness of Tangible versus Cash Rewards on Whistleblower Reporting Behavior

SYNOPSISWe investigate the effects of incentive type (i.e., cash versus. tangible) in motivating whistleblower behavior. While prior research indicates that cash rewards are an effective means for motivating whistleblower reporting, research has yet to examine the relative effectiveness of tangible incentives (e.g., gift cards, incentive travel, and merchandise) in promoting these prosocial behaviors. Motivated by mental accounting theory, our study experimentally tests and finds that the type of reward offered (cash versus. tangible) interacts with reward size to predict whistleblower reporting behavior. Specifically, whistleblower reporting was less (more) sensitive to changes in reward size when small tangible (cash) rewards were offered. These findings suggest that tangible (i.e., noncash) rewards can increase both the efficiency and effectiveness of whistleblower incentive programs and should be of considerable interest to managers, corporate boards, audit committees, and those charged with corporate governance.

VIP interneurons in sensory cortex encode sensory and action signals but not direct reward signals

Vasoactive intestinal peptide (VIP) interneurons in sensory cortex modulate sensory responses based on global exploratory behavior and arousal state, but their function during non-exploratory, goal-directed behavior is not well understood. In particular, whether VIP cells are activated by sensory cues, reward-seeking actions, or directly by reinforcement is unclear. We trained mice on a Go/NoGo whisker touch detection task that included a delay period and other features designed to separate sensory-evoked, action-related, and reward-related neural activity. Mice had to lick in response to a whisker stimulus to receive a variable-sized reward. Using two-photon calcium imaging, we measured ΔF/F responses of L2/3 VIP neurons in whisker somatosensory cortex (S1) during behavior. In both expert and novice mice, VIP cells were strongly activated by whisker stimuli and goal-directed actions (licking), but not by reinforcement. VIP cells showed somatotopic whisker tuning that was spatially organized relative to anatomical columns in S1, unlike lick-related signals which were spatially widespread. In expert mice, lick-related VIP responses were suppressed, not enhanced, when a reward was delivered, and the amount of suppression increased with reward size. This reward-related suppression was not seen in novice mice, where reward delivery was not yoked to licking. These results indicate that besides arousal and global state variables, VIP cells are activated by local sensory features and goal-directed actions, but not directly by reinforcement. Instead, our results are consistent with a role for VIP cells in encoding the expectation of reward associated with motor actions.

Designing gamified interactions with self-service technology at restaurants

This research aims to (1) investigate the effects/mechanisms of gamifying restaurant self-service technologies (SSTs) on post-SST usage behavior; and (2) understand how this gamification effect varies as a function of the game result (win vs. loss), reward type (monetary vs. altruistic), and reward size (small vs. large). Two simulation experiments were conducted to achieve these objectives. Our findings suggest that gamified (vs. non-gamified) SST interactions yield higher behavioral outcomes. The results further indicate that menu knowledge, which can be acquired through gamified SSTs, is a necessary condition for impulsive menu ordering. Moreover, we discovered that participants who win (vs. lose) in the game expressed greater enjoyment, perceived value, and post-SST behavioral outcomes, but only when a monetary (vs. altruistic) reward was offered. These findings contribute to the literature by providing evidence of the impacts of gamification, game results, and reward type on menu ordering and behavioral intention in restaurant settings.

Conflicting constraints on male mating success shape reward size in pollen-rewarding plants.

Pollen-rewarding plants face two conflicting constraints: They must prevent consumptive emasculation while remaining attractive to pollen-collecting visitors. Small pollen packages (the quantity of pollen available in a single visit) may discourage visitors from grooming (reducing consumptive loss) but may also decrease a plant's attractiveness to pollen-collecting visitors. What package size best balances these two constraints? We modeled the joint effects of pollinators' grooming behaviors and package size preferences on the optimal package size (i.e., the size that maximizes pollen donation). We then used this model to examine Darwin's conjecture that selection should favor increased pollen production in pollen-rewarding plants. When package size preferences are weak, minimizing package size reduces grooming losses and should be favored (as in previous theoretical studies). Stronger preferences select for larger packages despite the associated increase to grooming loss because loss associated with nonremoval of smaller packages is even greater. Total pollen donation increases with production (as Darwin suggested). However, if floral visitation declines or packages size preference increases with overall pollen availability, the fraction of pollen donated may decline as per-plant pollen production increases. Hence, increasing production may result in diminishing returns. Pollen-rewarding plants can balance conflicting constraints on pollen donation by producing intermediate-sized pollen packages. Strictly pollen-rewarding plants may have responded to past selection to produce more pollen in total, but diminishing returns may limit the strength of that selection.

Monetary rewards and self-selection in design crowdsourcing contests: Managing participation, contribution appropriateness, and winning trade-offs

Does the size of monetary rewards have differential effects on a crowdsourcing contest member's likelihood of (i) participation (i.e., submitting contribution(s) to the contest), (ii) making appropriate contribution(s) that are accepted for inclusion in the contest, and (iii) winning the contest? In this paper, we develop a theoretical framework to unravel the role of extrinsic motivation through monetary rewards in influencing an individual's performance outcomes. We hypothesize that the size of monetary rewards increases the probability of participation but reduces the likelihood that individual participants will submit appropriate contribution(s) or win the contest. These trade-off effects of monetary rewards can be mitigated significantly through participation of individuals with skills matching the contest requirements and having past experience in crowdsourcing contests. We obtain support for our hypotheses using a unique dataset of 93,677 individual-contest dyads consisting of 50,227 individual contest members who registered for 183 contests on eYeka, a design crowdsourcing platform during the period of 2007–2012. The results imply that increasing the size of monetary rewards needs to be complemented by increasing the participation of individuals with skills match and past experience in order to mitigate the trade-off effects.

On-demand ride-sourcing markets with cryptocurrency-based fare-reward scheme

Cryptocurrency-based fare-reward scheme has been emerging from on-demand ride-sourcing platforms. The reward scheme provides each passenger with some units of cryptocurrency as rewards for use of ride service. The passengers can redeem the rewards for various redemption options. The redemption options offer the passengers economic benefits to offset a portion of costs of using ride services. In this way, the economic benefits serve as incentives for the passengers to repeatedly use ride services, which leads to increase in passenger demand for the platforms. From a passenger perspective, the reward scheme has a distinctive feature in that economic benefits can fluctuate according to changes in price of cryptocurrency in the future. Therefore, the passengers perceive the benefits differently based on their own risk preference and expectation of future price changes. It creates a challenge for the platforms because the optimal decisions for ride services vary by the value that the passengers place for their cryptocurrency rewards. To tackle this challenge, this paper analyzes the impacts of the reward scheme on on-demand ride-sourcing markets. The analysis is done with a mathematical model where a ride-sourcing platform makes decisions for trip fare, vehicle fleet size and cryptocurrency reward size. The reward decision is available under the assumption that the platform secures as many units of cryptocurrency as it rewards to passengers at their equilibrium price level. The model defines passengers’ perceived value of cryptocurrency rewards in terms of risk aversion level and expectation of future price growth of cryptocurrency. Afterwards, we investigate equilibrium and optimal properties of monopoly, social-optimum and second-best optimum ride-sourcing markets. Our analysis gives several key insights on the markets. First, the markets always locate at normal regime where interactions between passenger demand and vehicle supply for ride services converges to a stable equilibrium state. Second, this study shows that a ride-sourcing platform would make a decision to reward if passengers expect larger economic benefits from future redemption of cryptocurrency rewards than their current value. Third, it is shown that the quality of ride services would get worse because average pick-up time for passengers and drivers gets longer with more passenger retention. Fourth, this paper finds that the markets have more room to improve profit or social welfare as passengers expect larger speculative gains from reward redemption.

Effects of methylphenidate on impulsive choice and delay aversion in Lewis rats.

Attention-deficit/hyperactivity disorder (ADHD), a common behavioral disorder in children and young adults, is characterized by symptoms of impulsivity, inattention, and hyperactivity. The purpose of this study was to evaluate the Lewis rat strain as a model of ADHD by testing their impulsive choices. Lewis rats were compared to their source strain, the Wistar rat, on an impulsive choice task. Rats completed the tasks on and off methylphenidate, a commonly prescribed medication for ADHD. Off methylphenidate, Lewis rats made more impulsive choices than Wistar rats. Analyses of acquisition of choice behavior suggested that both strains were able to discriminate reward sizes, but Lewis rats still chose the smaller-sooner option more than the larger-later (LL) option when the delays to reward were the same. This may be due to an aversion to the LL lever, which was associated with the longest delays to reward. Higher doses of methylphenidate increased LL choices in Lewis rats but decreased LL choices in Wistar rats. Altogether, these results suggest Lewis rats may be a viable model for ADHD in individuals whose symptoms are characterized by impulsive choices.