Actual Reward Research Articles

Is the tail wagging the dog? Understanding the value of incentives in peer review

Incentives for efforts expended in peer review remain controversial, as one of us [https://www.morressier.com/o/event/655b8f316515150019d5f90b/sessions/6569ca7c8d0e640013427ec3?eventId=655b8f316515150019d5f90b] reported at the recent APE Meeting in Berlin 2024. Publishers and researchers have different views on if, and how, peer review should and could be rewarded. We surveyed researchers regularly performing peer review for journals as well as editors and journal managers who assign articles in order to look at the use of incentives and how these influence behavior. Incentives for peer review remain relatively rare overall: Results show that more than 80% of editorial board members said they ‘did not reward’ or ‘rewarded in less than 25% of cases’, while, at the same time, the same proportion of respondents felt that rewarding peer reviewers would be a good idea. The majority of our respondents felt that giving ‘something was better than nothing’ and that a token incentive would help. In terms of actual rewards, APC tokens are both wanted (52%) and awarded (42%) while certificates are given far more (43%) than they are actually wanted (12%). Surprisingly, money is given out far less (4%) than wanted (27%); this reward is not actually desired as much as one might expect.

An overview of reward prediction error and its links with dopamine

Reward prediction error (RPE) refers to the discrepancy between expected and actual rewards received during an event, signaling the difference between what was predicted and what actually happened. Dopaminergic neuron encodes RPE signals in the brain, and is responsible for updating reward expectations and influencing decision-making processes. The relationship between RPE and dopamine has led to research in understanding reward-driven learning and its implications on cognition and behavior. In this review, I will provide an overview of the principles, and task models used to quantify RPE. I will also discuss about the neural mechanisms underlying RPE generation, with a particular focus on the role of dopamine in encoding and transmitting RPE signals and enhance learning. Overall, this review highlights the importance of RPE in reward processing, decision-making, memory and so on, emphasizing its connection to dopamine.

Predictions about reward outcomes in rhesus monkeys.

Human infants and nonhuman animals respond to surprising events by looking longer at unexpected than expected situations. These looking responses provide core cognitive evidence that nonverbal minds make predictions about possible outcomes and detect when these predictions fail to match reality. We propose that this phenomenon has crucial parallels with the processes of reward prediction error, indexing the difference between expected and actual reward outcomes. Most work on reward prediction errors to date involves neurobiological techniques that cannot be implemented in many relevant populations, so we developed a novel behavioral task to assess monkeys' predictions about reward outcomes using looking time responses. In Study 1, we tested how semi-free-ranging monkeys (n = 210) responded to positive error (more rewards than expected), negative error (less rewards than expected), and a number control. We found that monkeys looked longer at a given reward when it was unexpectedly large or small, compared to when the same quantity was expected. In Study 2, we compared responses in the positive error condition in monkeys ranging from infancy to old age (n = 363), to assess lifespan changes in sensitivity to reward predictions. We found that adolescent monkeys showed heightened responses to unexpected rewards, similar to patterns seen in humans, but showed no changes during aging. These results suggest that monkeys' looking responses can be used to track their predictions about rewards, and that monkeys share some developmental signatures of reward sensitivity with humans, providing a new approach to access cognitive processes underlying reward-based decision making. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

The interactive effect of organizational identification and reward type on reward valuation

AbstractRecent management trends highlight two techniques firms use to motivate employee effort: (1) fostering employees' organizational identification (OI) and (2) offering employees tangible rewards such as gift cards instead of cash rewards. We use three studies to examine how OI affects employees' reward valuation and how such effects differ depending on the reward type. Study 1 is an experiment, demonstrating that increasing OI increases the emphasis participants place on a reward's symbolic value, which then increases the total value of the reward—to a larger extent when the reward is tangible than when it is cash. Study 2 is an experiment, providing evidence that Study 1 results are robust to using a tangible reward that is not socially consumed, that is selected either by the firm or by the employee, and that is either a good or poor fit with the employee's personal preference. Finally, Study 3 is a survey, asking respondents about actual rewards they received from their current employer and capturing their actual OI with their current employer. Results in Study 3 are inferentially similar to those in Study 1 and Study 2, albeit stronger for rewards of smaller monetary value. Collectively, these results highlight the particular benefit of strong OI on how employees value tangible rewards relative to cash rewards, which should be of interest to incentive system designers.

Identify Elements of the VTA Projecting RPE Circuit Neurons inMouse Mus musculus Prefrontal Cortex

The Reward Prediction Error (RPE) model was presented decades ago but still has elements adding to the theory asthe mystery of neuron signaling pathways unravels. Following the projections of the dopaminergic neurons, the RPEcircuit originates from the ventral tegmental area (VTA), goes to the nucleus accumbens (NAc) reaches a higherorder processing center, which is the prefrontal cortex (PFC). Furthermore, downstream GABAergic neurons will beactivated according to the “error,” which indicates the value difference between the anticipated and actual rewards. Thisstudy aims to investigate the intermediate steps of the circuit, if any, and the role these intermediate neurons played instimulating the GABAergic neurons after the sensory inputs are integrated and processed at the PFC. Numerous studieshave investigated the function and mechanism of the VTA-NAc-PFC connections, whereas this experiment focuses onthe second half of the circuit. At least one, and possibly many, PFC residing, VTA projecting neurons with long axonscan be identified.

Eyes over stomach: companion dogs choose the larger quantity by sight, irrespective of the actual reward eaten

Abstract Domestic dogs tested in a free-choice task between two sets of food prefer that of a larger quantity. A recent study pointed out the critical role of the first trial. Dogs succeeded only after being allowed at least once to eat the selected food. Here we explore the importance of the actual experience of consuming the selected reward on dogs’ capability to redirect their choice to solve numerical discriminations. Dogs tested in the 2 vs 4 and 1 vs 8 comparisons could never consume the quantity selected but always obtained a single dogs’ treat. Despite never experiencing a difference in the eaten quantity, dogs discriminated between the two sets, preferring the larger. Whereas they behaved at chance at first choice, they successfully redirected their preference toward the larger set already on their second trial. We discuss our results in terms of motivational biases toward the larger quantity that can bear relevant ecological value.

Effects of perceived vs actual frequency of rewards on orthodontic patient attitudes and compliance.

To examine the longitudinal association of different reward schedules on patient compliance (as measured by oral hygiene assessments). The cross-sectional associations of actual vs perceived rewards frequency on patient attitudes were also examined. 138 patients undergoing treatment at a university orthodontic clinic were surveyed to collect information on perceived frequency of rewards, likelihood of making patient referrals, and attitudes toward reward programs and orthodontic treatment. Oral hygiene assessment from the most recent appointment and actual frequency of rewards were obtained from patient charts. Among participants, 44.9% were male, age ranged from 11 to 18 (mean = 14.9 ± 1.7) years; treatment time ranged from 9 to 56 (mean = 23.2 ± 9.8) months. Mean perceived frequency of rewards was 48% while actual frequency of rewards was 19.6%. There were no significant differences in attitudes by actual reward frequency (P > .10). However, those who perceived always receiving rewards were significantly more likely to have more positive opinions of reward programs (P = .004 and P = .024). Age- and treatment-time adjusted analyses showed that always receiving actual rewards was associated with odds of good oral hygiene 3.8 times (95% CI = 1.13, 13.09) higher than those never/rarely receiving actual rewards, but there was no association between perceived rewards and odds of good oral hygiene. Actual and perceived reward frequencies were significantly and positively correlated (r = 0.40, P < .001). It is beneficial to give rewards to patients as often as possible to maximize compliance (as shown by hygiene ratings) and foster positive attitudes.

Dynamic Data Transaction in Crowdsensing Based on Multi-Armed Bandits and Shapley Value

Crowdsensing gradually forms a big data market where workers are willing to trade reusable data with different data collectors. It is challenging for the data collector to choose the transaction party due to the changeable value of the data, while determining the transaction price is also a tough issue. In this paper, we research the dynamic data transaction in crowdsensing. The contribution of the new data to the collector is modeled as the Shapley value, with each worker as a player in the cooperative game. The data collector then judges the contribution of the worker and determines the transaction object. To maximum the profit in the transaction, the collector will dynamically adjust the offering price to workers. The contextual bandit model is utilized in the price decision, with each candidate price as an arm and the time-variant data value as the context. Based on the classic LinUCB learning policy, we learn the mapping of the observed data value and the reward, and estimate the optimal reward in current transaction. The simulation on the data demonstrates that the actual reward got by the collector is close to the maximum reward he can get, which verifies the effectiveness of our scheme.

A VTA GABAergic computational model of dissociated reward prediction error computation in classical conditioning

Classical Conditioning is a fundamental learning mechanism where the Ventral Striatum is generally thought to be the source of inhibition to Ventral Tegmental Area (VTA) Dopamine neurons when a reward is expected. However, recent evidences point to a new candidate in VTA GABA encoding expectation for computing the reward prediction error in the VTA. In this system-level computational model, the VTA GABA signal is hypothesised to be a combination of magnitude and timing computed in the Peduncolopontine and Ventral Striatum respectively. This dissociation enables the model to explain recent results wherein Ventral Striatum lesions affected the temporal expectation of the reward but the magnitude of the reward was intact. This model also exhibits other features in classical conditioning namely, progressively decreasing firing for early rewards closer to the actual reward, twin peaks of VTA dopamine during training and cancellation of US dopamine after training.

Dopamine errors drive excitatory and inhibitory components of backward conditioning in an outcome-specific manner

For over two decades, phasic activity in midbrain dopamine neurons was considered synonymous with the prediction error in temporal-difference reinforcement learning.1-4 Central to this proposal is the notion that reward-predictive stimuli become endowed with the scalar value of predicted rewards. When these cues are subsequently encountered, their predictive value is compared to the value of the actual reward received, allowing for the calculation of prediction errors.5,6 Phasic firing of dopamine neurons was proposed to reflect this computation,1,2 facilitating the backpropagation of value from the predicted reward to the reward-predictive stimulus, thus reducing future prediction errors. There are two critical assumptions of this proposal: (1) that dopamine errors can only facilitate learning about scalar value and not more complex features of predicted rewards, and (2) that the dopamine signal can only be involved in anticipatory cue-reward learning in which cues or actions precede rewards. Recent work7-15 has challenged the first assumption, demonstrating that phasic dopamine signals across species are involved in learning about more complex features of the predicted outcomes, in a manner that transcends this value computation. Here, we tested the validity of the second assumption. Specifically, we examined whether phasic midbrain dopamine activity would be necessary for backward conditioning-when a neutral cue reliably follows a rewarding outcome.16-20 Using a specific Pavlovian-to-instrumental transfer (PIT) procedure,21-23 we show rats learn both excitatory and inhibitory components of a backward association, and that this association entails knowledge of the specific identity of the reward and cue. We demonstrate that brief optogenetic inhibition of VTADA neurons timed to the transition between the reward and cue reduces both of these components of backward conditioning. These findings suggest VTADA neurons are capable of facilitating associations between contiguously occurring events, regardless of the content of those events. We conclude that these data may be in line with suggestions that the VTADA error acts as a universal teaching signal. This may provide insight into why dopamine function has been implicated in myriad psychological disorders that are characterized by very distinct reinforcement-learning deficits.

Spontaneous recovery from overexpectation in an insect

In associative learning in mammals, it is widely accepted that learning is determined by the prediction error, i.e., the error between the actual reward and the reward predicted by the animal. However, it is unclear whether error-based learning theories are applicable to the learning occurring in other non-mammalian species. Here, we examined whether overexpectation, a phenomenon that supports error-based learning theories, occurs in crickets. Crickets were independently trained with two different conditioned stimuli (CSs), an odour and a visual pattern, that were followed by an appetitive unconditioned stimulus (US). Then the two CSs were presented simultaneously as a compound, followed by the same US. This treatment resulted in a reduced conditioned response to the odour CS when tested immediately after training. However, the response to the CS was partially recovered after 1 day. These results are the first to show overexpectation and its spontaneous recovery in an invertebrate species. While the results showing overexpectation are in agreement with the prediction by the Rescorla-Wagner model, a major form of error-based learning theories, the ones showing spontaneous recovery are not. Our results suggest that conventional error-based learning models account for some, but not for all essential features of Pavlovian conditioning in crickets.

Conditional Regulation of Blood Pressure in Response to Emotional Stimuli by the Central Nucleus of the Amygdala in Rats.

Humans and animals can determine whether a situation is favorable to them and act accordingly. For this, the autonomic tuning of the cardiovascular system to supply energy to active skeletal muscles through the circulatory system is as important as motor control. However, how the autonomic cardiovascular responses are regulated in dynamically changing environments and the neuronal mechanisms underlying these responses remain unclear. To resolve these issues, we recorded the blood pressure and heart rate of head-restrained rats during dynamically changing appetitive and aversive classical conditioning tasks. The rats displayed various associations between conditioned stimuli and unconditioned stimuli in appetitive (sucrose water), neutral (no outcome), and aversive (air puff) blocks. The blood pressure and heart rate in the appetitive block gradually increased in response to the reward-predicting cue and the response to the actual reward vigorously increased. The reward-predictive response was significantly higher than the responses obtained in the neutral and aversive condition blocks. To investigate whether the reward-predictive pressor response was caused by orofacial movements such as anticipatory licking behavior, we separately analyzed high- and low-licking trials. The conditioned pressor response was observed even in trials with low-licking behaviors. Blood pressure and heart rate responses to the air puff-predicting cue in the aversive block were not significantly different from the responses in the neutral block. The conditioned blood pressure response rapidly changed with condition block switching. Furthermore, to examine the contribution of the amygdala as an emotion center to these conditioned responses, we bilaterally microinjected a GABAA receptor agonist, muscimol, into the central nucleus of the amygdala. Pharmacological inactivation of the central nucleus of the amygdala significantly decreased the reward-predictive pressor responses. These results suggest that the blood pressure is adaptively and rapidly regulated by emotional conditioned stimuli and that the central nucleus of the amygdala participates in regulating the pressor response in dynamically changing situations.

A systematic review of factors associated with illegal drug driving

Drug driving is a serious problem worldwide that can increase the risk of road crashes. This systematic review seeks to identify factors associated with drug driving (i.e., driving after consuming drugs other than alcohol) to highlight gaps in existing knowledge and inform the design of more effective countermeasures. A search of the literature was conducted for the period January 1, 2005 to July 31, 2021 using six different databases. The search protocol followed PRISMA guidelines and was registered in PROSPERO (#CRD42021234616). Studies that met inclusion criteria compared drug drivers with either non-drug drivers, alcohol-only drivers or drug drivers from an earlier time period, to identify factors specifically associated with drug driving, rather than common to all drivers. Two hundred and nineteen publications met the inclusion criteria and were included within the review. Based on the findings, a logic model was developed that presents the factors associated with drug driving. Various sociodemographic, psychosocial and legal factors emerged as the main factors associated with illegal drug driving. At the sociodemographic and psychological levels, drug drivers were more likely to be single, young males who often drive after using cannabis and who score high on sensation-seeking and impulsivity scales. The key social factor found to be associated with drug driving was peer acceptance/disapproval of the behaviour. At the legal level, the review suggested that the effectiveness of current enforcement approaches to drug driving vary among jurisdictions around the world due to differences in the level of perceived certainty of apprehension and the chances of punishment avoidance. Future research into the anticipated and actual rewards for drug driving is needed to inform the development of more effective countermeasures.

Neural underpinnings of food choice and consumption in obesity.

Obesity is associated with unhealthy food choices. Food selection is driven by the subjective valuation of available options, and the perceived and actual rewards accompanying consumption. These cognitive operations are mediated by brain regions including the ventromedial prefrontal cortex (vmPFC), dorsal anterior cingulate cortex (dACC), and ventral striatum (vStr). This study investigated the relationship between body mass index (BMI) and functional activations in the vmPFC, dACC, and vStr during food selection and consumption. After overnight fasting, 26 individuals (BMI: 18-40 kg/m2) performed a food choice task while being scanned with functional magnetic resonance imaging (fMRI). Each trial involved selecting one beverage from a pair of presented options, followed by delivery of a 3 mL aliquot of the selected option using an MR-compatible gustometer. We also tracked subjective preference for each beverage throughout the experiment. During food choice, individuals with greater BMI had less activation in the dorsolateral prefrontal cortex when selecting a high-value option and less vmPFC activation upon its consumption. Independent of BMI, during food choice the dACC and anterior insula elicited higher activation when a less preferred beverage was selected. Activation of the dACC and a broader frontoparietal network was also observed when deciding between options more similar in value. During consumption, receipt of a more preferred beverage was associated with greater vmPFC response, and attenuation of the dACC. An individual's preference for a food option modulates the brain activity associated with choosing and consuming it. The relationship between food preference and underlying brain activity is altered in obesity, with reduced engagement of cognition-related regions when presented with a highly valued option, but a blunted response in reward-related regions upon consumption.

The \u201cEconomic Battle\u201d Now and Then: (E)valuation Patterns of Distributive Justice in Cuban State-Socialism

This article disentangles and explores some commonly made assumptions about egalitarian state-socialist ideologies. Based on the conceptual framework of the multiprinciple approach of justice, it presents the results of an in-depth analysis of (e)valuation patterns of distributive justice in Cuban state-socialism. The analysis mainly focuses on ideational conceptions of distributive justice (just rewards), but it also accounts for distribution outcomes and resulting (in)equalities (actual rewards). The results of the comparative case study of the Cuban framework of institutions and political leaders’ views in two periods of time, the early 1960s and the 2010s, point to (e)valuation patterns that are generally labelled as egalitarian, such as the allocation rules of outcome equality and (non-functional) needs. However, contrary to common assumptions about egalitarian state-socialist ideologies, the results also point to several other patterns, including equity rules as well as functional and productivist allocation rules. I argue that many of these (e)valuation patterns, in their connection to the discursive storyline of the Cuban economic battle, are indeed compatible with egalitarian state-socialist ideology.

A new paradigm for Prosocial Behavior and Reciprocity, assessed in WT and HET rats for the DAT gene

Most behavioral studies on animals focus on observation of individual subjects. Current paradigms of sociability set aside the social-operant dimension, i.e. acting in favor of another conspecific. We focused on prosocial behavior and reciprocity of male, adult Wild-Type (WT) and Heterozygous (HET) rats for the dopamine-transporter (DAT) gene. MethodThe experiment consisted of 24 rats, of WT (n = 12) and HET (n = 12) genotypes. During training, rats were daily introduced, individually, into an apparatus hosting a suspended syringe, which they learnt to push in order to obtain food therein. Then, twice daily along several weeks, we introduced two rats separated by a grid in the same structure: by syringe-pushing, each subject had the opportunity to donate and receive donations of food. We tested pairs with similar versus different genotype. Eventually, we replaced food reward with polystyrene pieces, to understand if they pushed for actual reward or like a habit. ResultsIn general, WT rats had better performance, regardless of reward type, than HET ones. When we crossed partner rats’ genotype (WT-HET pairs), WT rats pushed at peak levels, regardless of food pellet received back (in fact, HET companions pushed less). Couples of WT rats achieved better results than HET ones even when polystyrene, instead of food, was used. Thus,WT rats seem to be a better model for altruistic behavior than HET ones. For this reason, HET rats could represent a model for studies on altered prosocial behavior, to understand the role of DAT gene for impaired social mechanisms.

Cue-Evoked Dopamine Promotes Conditioned Responding during Learning

Dopamine neurons mediate the association of conditioned stimuli (CS) with reward (unconditioned stimuli, US) by signaling the discrepancy between predicted and actual reward during the US. Some theoretical models suggest that learning is also influenced by the salience or associability of the CS. A hallmark of CS associability models is that they can explain latent inhibition, i.e., the observation that novel CS are more effectively learned than familiar CS. Novel CS are known to activate dopamine neurons, but whether those responses affect associative learning has not been investigated. Here, we used fiber photometry to characterize dopamine responses to inconsequential familiar and novel stimuli. Using bidirectional optogenetic modulation during conditioning, we then show that CS-evoked dopamine promotes conditioned responses. This suggests that Pavlovian conditioning is influenced by CS dopamine, in addition to US reward prediction errors. Accordingly, the absence of dopamine responses to familiar CS might explain their slower learning in latent inhibition.

A Biased-Randomized Learnheuristic for Solving the Team Orienteering Problem with Dynamic Rewards

In this paper we discuss the team orienteering problem (TOP) with dynamic inputs. In the static version of the TOP, a fixed reward is obtained after visiting each node. Hence, given a limited fleet of vehicles and a threshold time, the goal is to design the set of routes that maximize the total reward collected. While this static version can be efficiently tackled using a biased-randomized heuristic (BR-H), dealing with the dynamic version requires extending the BR-H into a learnheuristic (BR-LH). With that purpose, a ‘learning’ (white-box) mechanism is incorporated to the heuristic in order to consider the variations in the observed rewards, which follow an unknown (black-box) pattern. In particular, we assume that: (i) each node in the network has a ‘base’ or standard reward value; and (ii) depending on the node’s position inside its route, the actual reward value might differ from the base one according to the aforementioned unknown pattern. As new observations of this black-box pattern are obtained, the white-box mechanism generates better estimates for the actual rewards after each new decision. Accordingly, better solutions can be generated by using this predictive mechanism. Some numerical experiments contribute to illustrate these concepts.

The contribution of striatal pseudo-reward prediction errors to value-based decision-making

Most studies that have investigated the brain mechanisms underlying learning have focused on the ability to learn simple stimulus-response associations. However, in everyday life, outcomes are often obtained through complex behavioral patterns involving a series of actions. Parallel learning systems might be important to reduce the complexity of the learning problem in such scenarios, as proposed in the framework of hierarchical reinforcement learning (HRL). The key feature of HRL is the decomposition of complex sets of action into subgoals. These subgoals are associated with the computation of pseudo-reward prediction errors (PRPEs), which allow the reinforcement of actions that led to a subgoal before the final goal itself is achieved. Here we wanted to test the hypothesis that, despite not carrying any rewarding value per se, pseudo-rewards might generate a bias in choice behavior in the absence of any advantage. Second, we also hypothesized that this bias might be related to the strength of PRPE striatal representations. In order to test these ideas, we developed a novel decision-making paradigm to assess reward prediction errors (RPEs) and PRPEs in two studies (fMRI study: n = 20; behavioral study: n = 19). Our results show that the participants developed a preference for the most pseudo-rewarding option throughout the task, even though it did not lead to more monetary rewards. fMRI analyses revealed that this preference was predicted by individual differences in the relative striatal sensitivity to PRPEs vs RPEs. Together, our results indicate that pseudo-rewards generate learning signals in the striatum and subsequently bias choice behavior despite their lack of association with actual reward.

Reconsolidation impairment of reward memory by stimulating stress response.

Research in memory reconsolidation has raised hope for new treatment options of persistent psychiatric disorders like substance dependence and post-traumatic stress disorder (PTSD). While animal research showed successful memory modification by interfering with reconsolidation, human research requires less invasive techniques. In our pilot study, we aimed to reduce appetitive memory reconsolidation of a newly acquired reward memory by exerting a stressor. Thirty healthy participants were randomly assigned to two groups performing a monetary reward paradigm at a personal computer. Day 1 was considered to allow for memory acquisition; on day 2, the experimental group was exposed to a frightening stimulus in the reconsolidation window; and day 3 again served to determine reward memory effects. Measures of reward memory were reaction times to reward announcing stimuli (ie, showing instrumental behavior), actual reward gained, and electrodermal response as a measure for reward anticipation. We found significantly smaller reaction time improvements to reward stimuli over time in the experimental group, as well as reduced achievements in monetary reward. Electrodermal response to reward announcing stimuli was lower in the experimental group after intervention, whereas it was higher in the untreated group. Thus, we argue in favor of the reconsolidation hypothesis, assuming our intervention had successfully interfered with the reconsolidation process. This points towards future treatment options that interfere with an addiction memory.