Reward Expectation Research Articles

Building and Breaking the Chain: A Model of Reward Prediction Error Integration and Segmentation of Memory.

Prediction errors drive reinforcement learning and organize episodic memory into distinct contexts, but do these effects interact? Here, we review the roles of midbrain dopamine, the locus coeruleus, and the hippocampus in event cognition to propose and simulate the theoretical influence of two prediction error signals in integrating versus segmenting events in memory. We suggest that signed reward prediction errors can build mental models of reward environments, increasing the contextual similarity (integration) of experiences with stronger, more stable reward expectations. On the other hand, unsigned reward prediction errors can signal a new model of the environment, generating a contextual shift (segmentation) between experiences that crossed them. We moreover predicted that these differences in contextual similarity give rise to distinct patterns of temporal-order memory. We combined these ideas in a computational model to account for a seemingly paradoxical pattern of temporal-order memory where greater representational distance helps order memory within context but impairs it across contexts. We found that simulating signed reward prediction error integration and unsigned reward prediction error segmentation differentially enabled the model to perform associative chaining, which involved reactivating items between two tested probes to assist with sequential retrieval. In summary, our simulations provide a unifying explanation for the varied ways that neuromodulatory systems may alter event cognition and memory.

Reinforcement learning in motor skill acquisition: using the reward positivity to understand the mechanisms underlying short- and long-term behavior adaptation.

According to reinforcement learning, humans adjust their behavior based on the difference between actual and anticipated outcomes (i.e., prediction error) with the main goal of maximizing rewards through their actions. Despite offering a strong theoretical framework to understand how we acquire motor skills, very few studies have investigated reinforcement learning predictions and its underlying mechanisms in motor skill acquisition. In the present study, we explored a 134-person dataset consisting of learners' feedback-evoked brain activity (reward positivity; RewP) and motor accuracy during the practice phase and delayed retention test to investigate whether these variables interacted according to reinforcement learning predictions. Results showed a non-linear relationship between RewP and trial accuracy, which was moderated by the learners' performance level. Specifically, high-performing learners were more sensitive to violations in reward expectations compared to low-performing learners, likely because they developed a stronger representation of the skill and were able to rely on more stable outcome predictions. Furthermore, contrary to our prediction, the average RewP during acquisition did not predict performance on the delayed retention test. Together, these findings support the use of reinforcement learning models to understand short-term behavior adaptation and highlight the complexity of the motor skill consolidation process, which would benefit from a multi-mechanistic approach to further our understanding of this phenomenon.

A common stay-on-goal mechanism in anterior cingulate cortex for information and effort choices.

Humans and non-humans alike often make choices to gain information, even when the information cannot be used to change the outcome. Prior research has shown the anterior cingulate cortex (ACC) is important for evaluating options involving reward-predictive information. Here we studied the role of ACC in information choices using optical inhibition to evaluate the contribution of this region during specific epochs of decision making. Rats could choose between an uninformative option followed by a cue that predicted reward 50% of the time vs. a fully informative option that signaled outcomes with certainty, but was rewarded only 20% of the time. Reward seeking during the informative S+ cue decreased following ACC inhibition, indicating a causal contribution of this region in supporting reward expectation to a cue signaling reward with certainty. Separately in a positive control experiment and in support of a known role for this region in sustaining high-effort behavior for preferred rewards, we observed reduced lever presses and lower breakpoints in effort choices following ACC inhibition. The lack of changes in reward latencies in both types of decisions indicate the motivational value of rewards remained intact, revealing instead a common role for ACC in maintaining persistence toward certain and valuable rewards.

Role of dopamine in reward expectation and predictability during execution of action sequences.

Reward-associated cues serve different functions depending on whether they precede or terminate action sequences. Cues that precede action sequences and signal opportunity for reward could serve as GO signals to initiate the sequence, whereas sequence termination cues could serve as response feedback by signaling reward delivery. Reward expectation during sequence execution depends on these cues and might condition whether behavior is habitual or goal-directed. However, it remains unknown how sequence initiation and termination cues differentially affect reward expectation and contribute to habit learning. Further, while mesolimbic dopamine plays a key role in cue-induced reward expectation and sequence learning, how dynamic changes in dopamine signals differ depending on the response strategy is unclear. Here, we determined how mesolimbic DA signals change over training during cue-mediated sequence learning, depending on the type of cue and the nature of behavioral control. We found sequence initiation and termination cues differentially affect reward expectation during action sequences, with the termination cue contributing to habit and automaticity. Distinct response strategies induced by sequence initiation and termination cues induced differential changes in mesolimbic DA signals that captured variations in reward expectation along sequence execution. Notably, habit-like behavior induced by the sequence termination cue was associated with a rapid shift in DA signals from reward retrieval to the cue. This habit-like behavior was reflected in behavioral inflexibility and attenuated DA reward prediction error signals. Finally, using optogenetics, we provide evidence that phasic DA activity elicited by the sequence termination cue is critical for the development of habit-like behavior.

Effect of pre-session discrimination training on performance in a judgement bias test in dogs

Spatial judgement bias tests (JBTs) can involve teaching animals that a bowl provides a reward in one location but does not in another. The animal is then presented with the bowl placed between the rewarded and the unrewarded locations (i.e., ambiguous locations) and their latency to approach reflects expectation of reward or ‘optimism’. Some suggest that greater ‘optimism’ indicates better welfare. Performance in JBTs, however, may also indicate a learning history independently from welfare determinants. We hypothesized that dogs’ ‘optimism’ in a follow-up JBT may be impacted by a learning treatment involving additional trials of a different discrimination task. Once enrolled, companion dogs (n = 16) were required to complete three study phases: (1) a pre-treatment JBT, (2) a learning treatment, and (3) a post-treatment JBT. During the JBTs, dogs were presented with five locations: one rewarded, one unrewarded, and three ambiguous (all unrewarded). Dogs were randomly assigned to a trial-based learning task—a nose-touch to the palm of the hand. In the Experimental discrimination treatment phase (n = 8), dogs were presented with two hands in each trial and only rewarded for touching one specific hand. In the Control treatment phase (n = 8), dogs were presented with one hand per trial in alternating sequence and were yoked to dogs in the Experimental group to receive the same number of rewarded and unrewarded trials (to control for possible frustration). Using a repeated measures mixed model with JBT repeated within dog, we found no difference in the change in approach latency to the ambiguous locations between the dogs across treatments. ‘Optimism’ as measured in this JBT was not altered by the additional discrimination trials used in our study.

Mice monitor their timing errors

Animals often engage in representationally guided goal-directed behaviors. These behaviors are thus also subjected to representational uncertainty (e.g. timing uncertainty during waiting), which has been previously shown to adaptively guide behaviors normatively. These observations raise the question of whether non-human animals can track the direction and magnitude of their timing errors (i.e. temporal error monitoring). Only a few studies have investigated this question without addressing the key components of temporal error monitoring (e.g. due to differential reinforcement of metacognitive judgments and primary task representation). We conducted the critical test of temporal error monitoring in mice by developing a novel behavioral task that involved temporal production that exponentially favored temporal accuracy and minimized the contribution of sensorimotor noise. The response rate for an upcoming probabilistic reward following the timing performance was used as a proxy for confidence. We found that mice exhibited high reward expectancy after accurate and low reward expectancy after inaccurate timing performance. The reward expectancy decreased as a function of deviations from the target interval for the short and long reproductions; pointing to the symmetrical sensitivity of metacognition to shorter/longer than target responses. These findings suggest a complete temporal error monitoring ability for mice with human-like metacognitive features.

Physically stressed bees expect less reward in an active choice judgement bias test.

Emotion-like states in animals are commonly assessed using judgment bias tests that measure judgements of ambiguous cues. Some studies have used these tests to argue for emotion-like states in insects. However, most of these results could have other explanations, including changes in motivation and attention. To control for these explanations, we developed a novel judgment bias test, requiring bumblebees to make an active choice indicating their interpretation of ambiguous stimuli. Bumblebees were trained to associate high or low rewards, in two different reward chambers, with distinct colours. We subsequently presented bees with ambiguous colours between the two learnt colours. In response, physically stressed bees were less likely than control bees to enter the reward chamber associated with high reward. Signal detection and drift diffusion models showed that stressed bees were more likely to choose low reward locations in response to ambiguous cues. The signal detection model further showed that the behaviour of stressed bees was explained by a reduction in the estimated probability of high rewards. We thus provide strong evidence for judgement biases in bees and suggest that their stress-induced behaviour is explained by reduced expectation of higher rewards, as expected for a pessimistic judgement bias.

Investigating the cognitive and affective dynamics of social media addiction: Insights from peer contexts.

Informed by the interaction of person-affect-cognition-execution (I-PACE) theory, the present studies examined the association between peer rejection, peer popularity, and social media addiction (SMA) at both between-person and within-person levels. Two distinct processes, the fear-driven/compensation-seeking process and the reward-driven process were explored. In Study 1, using a cross-sectional sample of high school students (N = 318), both processes were supported via different cognitive mediators. Support for the fear-driven/compensation-seeking process was demonstrated by finding that avoidance expectancy was a significant cognitive mediator between peer-nominated rejection and SMA. In turn, the reward-driven process was supported by the significant mediation of reward expectancy between peer-nominated popularity and SMA. In Study 2, using ecological momentary assessment with college students (N = 54), we found the fear-driven/compensation-seeking process partially supported through both between-person and within-person mediations. Specifically, negative affect and social media craving were two affective mediators that linked peer rejection and addictive social media use behaviors. On the other hand, the reward-driven process was predominantly supported by within-person mediations, in which positive affect and social media craving were found to be mediators of the relationship between peer popularity and addictive social media use behaviors. The results underscore that adolescents experiencing rejection tend to use social media to avoid negative feelings and compensate for interpersonal deficits, while adolescents experiencing popularity tend to use social media to maintain positive feelings and gain social rewards. Implications for the assessment, case formulation, and treatment of SMA in counseling practice are discussed. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Placebo effects beyond dopamine.

The role of dopamine in reward expectancy has led to the hypothesis that it is crucial for forming treatment expectations and placebo effects. However, a new study in PLOS Biology presents robust evidence against the causal role of dopamine in these processes.

Fulfillment or status: Job seekers’ reward expectations towards family and non-family employers

Limited anecdotal and empirical evidence suggests that family ownership status – whether a company is family-owned or publicly held – may influence job seekers’ intentions to apply. Drawing from the theory of person-organization fit and utilizing qualitative comparative analysis (QCA), we conducted an exploratory study analyzing survey data from 756 senior-level job seekers in Switzerland. Our analysis identified distinct cohorts of job seekers who consistently applied to family-owned versus non-family-owned companies. We discovered two profiles of applicants who seem to favor family employers: Fulfillment Seekers and Reward Seekers. Despite their differences, both profiles share an intrinsic need for praise. In contrast, we identified one profile of non-family firm applicants, Hero Followers, who prioritize neither praise nor pride but have a strong desire to work for an admired superior – the very thing Fulfillment Seekers and Reward Seekers deprioritize. Our findings suggest that distinct groups of job seekers are motivated by different reward combinations: Pay, Perks, Personas, Pride, which they expect to find in either family or non-family companies. Family firms could benefit from emphasizing these rewards in their recruitment processes, compensation schemes, and employer branding strategies.

The modulation of reward expectancy on the processing of near-miss outcomes: An ERP study

A near-miss is a situation in which a gambler almost wins but falls short by a small margin, which motivates gambling by making it feel like success is within reach. Existing research has extensively investigated the influence of contextual information on near-miss outcome processing; however, the impact of reward expectancy has received limited attention thus far. To address this gap, we utilized the wheel of fortune task and event-related potential technique (ERP) to quantify the electrophysiological responses associated with gambling outcomes at different levels of reward expectancy. Behaviorally, near-miss outcomes elicited a greater occurrence of counterfactual thoughts, feelings of regret, and heightened anticipation of rewards for subsequent trials compared to full-miss outcomes. ERP findings indicated that in contrast to full-miss outcomes, near-miss outcomes diminished feedback-related negativities (FRNs) and amplified P300s when reward expectancy was low, but amplified FRNs and diminished P300s when reward expectancy was high. These findings provide valuable insights into the neural mechanisms underlying the processing of outcome proximity and reward expectancy.

Evidence for the Beneficial Effect of Reward on Working Memory: A Meta-Analytic Study.

Rewards act as external motivators and can improve performance in various cognitive tasks. However, previous research demonstrated mixed findings regarding the effect of reward on working memory (WM) performance, and the question of whether reward enhances WM performance is arguable. It remains unclear how the effect of reward on WM can be influenced by various factors, such as types of reward and experimental paradigms. In this meta-analytic study, we systematically investigated the effect of reward on WM by analyzing data from 51 eligible studies involving a total of 1767 participants. Our results showed that reward robustly enhanced WM performance, with non-monetary rewards inducing more benefits than monetary rewards. This may be because, while both types of reward could induce extrinsic motivation, non-monetary rewards enhanced intrinsic motivation while monetary rewards reduced it. Notably, all three reward methods-reward binding, reward expectation, and subliminal reward-effectively improved WM performance, with the reward binding paradigm exhibiting the greatest effects. This finding suggests that the reward effect can be attributed to both increasing the total amount of WM resources and improving the flexibility of resource reallocation. Moreover, the type of WM, the experimental paradigms, and the outcome measures are three moderators that should be jointly considered when assessing the reward effects on WM. Overall, this meta-analytic study provides solid evidence that reward improves WM performance and reveals possible mechanisms underlying these improvements.

IMPULSE BUYING AND RETAILER LOYALTY PROGRAMS: ASSESS THE INFLUENCE OF LOYALTY PROGRAMS, REWARDS, AND CUSTOMER INCENTIVES ON IMPULSE BUYING

Comprehending the link between impulse buying and retailer loyalty programs is crucial in retail marketing. These programs, designed to encourage repeat purchases and foster customer loyalty with personalized rewards, profoundly impact impulsive behaviors. This study uses a mixed methods approach to assess the effectiveness of loyalty strategies across diverse consumer demographics. Rewards such as discounts, points, or exclusive access cultivate a sense of belonging and exclusivity, influencing purchasing decisions by triggering immediate incentives and urgency through limited time offers. Emotional and psychological factors drive impulse buying. The expectation of rewards, the excitement of earning points, and the sense of exclusivity all play roles in impulsive purchases. Retailers can capitalize on these motivators by tailoring programs accordingly. Demographic variables such as age, income, and shopping habits influence program effectiveness, with younger consumers typically favoring digital rewards and older consumers showing a preference for traditional discounts. Qualitative research delves into motivations and perceptions, providing rich insights into how loyalty programs influence impulse buying behaviors. This holistic approach offers valuable insights for refining marketing strategies, increasing customer loyalty, and promoting impulse buying through personalized rewards, real-time offers, and gamification.

Surfacing of Latent Time Memories Supports the Representational Basis of Timing Behavior in Mice

Animals can adapt their reward expectancy to changes in delays to reward availability. When temporal relations are altered, associative models of interval timing predict that the original time memory is lost due to the updating of the underlying associative weights, whereas the representational models render the preservation of the original time memory (as previously demonstrated in the extinction of conditioned fear). The current study presents the critical test of these theoretical accounts by training mice with two different intervals in a consecutive fashion (short → long or long → short) and then testing timing behaviors during extinction where neither temporal relation is in effect. Mice that were trained with the long interval first clustered their anticipatory responses around the average of two intervals (indirect higher-order manifestation of two memories in the form of temporal averaging), whereas mice trained with the short interval first clustered their responses either around the short or long interval (direct manifestation of memory representations by their independent indexing). We assert that the original memory representation formed during training with the long interval “metrically affords” the integration of subsequent experiences with a shorter interval, allowing their co-activation during extinction. The original memory representation formed during training with the short interval would not metrically afford such integration and thus result in the formation of a new (mutually exclusive) time memory representation, which does not afford their co-activation during extinction. Our results provide strong support for the representational account of interval timing. We provide a new theoretical account of these findings based on the “metric affordances” of the original memory representation formed during training with the original intervals.

A distinct circuit for biasing visual perceptual decisions and modulating superior colliculus activity through the mouse posterior striatum.

The basal ganglia play a key role in visual perceptual decisions. Despite being the primary target in the basal ganglia for inputs from the visual cortex, the posterior striatum's (PS) involvement in visual perceptual behavior remains unknown in rodents. We reveal that the PS direct pathway is largely segregated from the dorsomedial striatum (DMS) direct pathway, the other major striatal target for visual cortex. We investigated the role of the PS in visual perceptual decisions by optogenetically stimulating striatal medium spiny neurons in the direct pathway (D1-MSNs) of mice performing a visual change-detection task. PS D1-MSN activation robustly biased visual decisions in a manner dependent on visual context, timing, and reward expectation. We examined the effects of PS and DMS direct pathway activation on neuronal activity in the superior colliculus (SC), a major output target of the basal ganglia. Activation of either direct pathway rapidly modulated SC neurons, but mostly targeted different SC neurons and had opposite effects. These results demonstrate that the PS in rodents provides an important route for controlling visual decisions, in parallel with the better known DMS, but with distinct anatomical and functional properties.

An overview of reward prediction error and its links with dopamine

Reward prediction error (RPE) refers to the discrepancy between expected and actual rewards received during an event, signaling the difference between what was predicted and what actually happened. Dopaminergic neuron encodes RPE signals in the brain, and is responsible for updating reward expectations and influencing decision-making processes. The relationship between RPE and dopamine has led to research in understanding reward-driven learning and its implications on cognition and behavior. In this review, I will provide an overview of the principles, and task models used to quantify RPE. I will also discuss about the neural mechanisms underlying RPE generation, with a particular focus on the role of dopamine in encoding and transmitting RPE signals and enhance learning. Overall, this review highlights the importance of RPE in reward processing, decision-making, memory and so on, emphasizing its connection to dopamine.

Unveiling Unexpected Selfless Collectivism: Exploring Variances in Moral Beliefs Across Cultural Values.

The impact of moral beliefs on individuals' moral judgments, particularly within the framework of contrasting cultural values such as collectivism and individualism, continues to be a subject of interest. This research delves into whether individuals with distinct values display differences in moral beliefs, with a specific focus on selflessness. Through experiment 1 and 2, we scrutinize the moral judgments of individuals with diverse values concerning pro-social behaviors driven by selflessness and in-group favoritism. Experiment 3 investigates the underlying mechanisms by examining reward expectations. The findings from experiments 1 and 2 indicate that collectivists exhibit significantly higher moral judgment levels for selfless pro-social behaviors compared to individualists, highlighting a disparity in moral beliefs between the two value systems. Experimental 2 also found that collectivists' moral judgments rose and fell as rescuers' selflessness increased or decreased. However, individualist moral judgment did not change significantly with behavioral selflessness. The results reveal that collectivists hold higher reward expectations for selfless behavior, which contributes to their heightened moral judgment of selfless behavior. While it may be assumed that collectivists' moral beliefs would be dominated by in-group preferences, aligning with the conceptual in-group preference of collectivism, this study unexpectedly found evidence of collectivists' pursuit of selflessness in their moral beliefs. These findings offer initial evidence of distinct patterns of moral beliefs associated with collectivism and individualism, shedding light on potential reasons that lead collectivists to possess stronger moral beliefs about selflessness.

A Comprehensive Analysis of Ariyah, Hibah, Hadiah, and Samsarah: Theoretical and Practical Perspectives

This literature review article presents an analysis of the concepts of Ariyah, Grants, Gifts, and Samsarah from a theoretical and practical perspective to understand the complexity and legal framework governing these transactions in an Islamic context. The data is analyzed in depth to identify key elements and relationships between concepts. Ariyah (borrowing) is the loan of goods without remuneration, Hibah (grant) is the voluntary gift of a living person to another, Hadiah (gift) without expectation of reward, and Samsarah (brokerage) is connecting between parties involved in a transaction (such as buyers and sellers) in which the intermediary is given a certain commission. Despite having fundamental differences in definition and application, these four concepts play an important role in society. From a theoretical perspective, each concept has a legal basis and principles governing it. In practical terms, the application of Ariyah, Grants, Gifts, and Samsarah is seen in various aspects of daily life, from trade transactions to social relations. Understanding these concepts helps individuals and organizations manage relationships and transactions more effectively according to the principles of sharia law.

Are lateralized and bold fish optimistic or pessimistic?

Cognitive bias is defined as the influence of emotions on cognitive processes. The concept of the cognitive judgement bias has its origins in human psychology but has been applied to animals over the past 2 decades. In this study we were interested in determining if laterality and personality traits, which are known to influence learning style, might also be correlated with a cognitive bias in the three-spined sticklebacks (Gasterosteus aculeatus). We used the judgement bias test with the go/no-go procedure where fish were first trained to discriminate between a black and white card and, after reaching a minimum learning criterion, tested their response to an ambiguous card (grey). Optimistic subjects were expected to have a high expectation of reward associated with an ambiguous stimulus, whereas pessimistic subjects a high expectation of non-reward. We used an emergence and a mirror test to quantify boldness and laterality, respectively. We hypothesised that male, bolder and more strongly lateralized fish would be more optimistic than female, shy and less strongly lateralised fish. We found that males and more strongly lateralized fish were more optimistic than females and less strongly lateralized fish. In addition, bold males were more optimistic than shy males as we predicted, but females showed the opposite pattern. Finally, fish trained on the black colour card learned the training task faster than those trained on a white card. Our results indicate that both laterality and personality traits are linked to animals’ internal states (pessimistic or optimistic outlooks) which likely has broad implications for understanding animal behaviour particularly in a welfare context.

Neural basis of reward expectancy inducing proactive aggression.

Proactive aggression refers to deliberate and unprovoked behavior, typically motivated by personal gain or expected reward. Reward expectancy is generally recognized as a critical factor that may influence proactive aggression, but its neural mechanisms remain unknown. We conducted a task-based functional magnetic resonance imaging (fMRI) experiment to investigate the relationship between reward expectancy and proactive aggression. 37 participants (20 females, mean age = 20.8 ± 1.42, age range = 18-23 years) completed a reward-harm task. In the experiment, reward valence expectancy and reward possibility expectancy were manipulated respectively by varying amounts (low: 0.5-1.5 yuan; high: 10.5-11.5 yuan) and possibilities (low: 10%-30%; high: 70%-90%) of money that participants could obtain by choosing to aggress. Participants received fMRI scans throughout the experiment. Brain activation regions associated with reward expectancy mainly involve the middle frontal gyrus, lingual gyrus, inferior temporal gyrus, anterior cuneus, caudate nucleus, inferior frontal gyrus, cingulate gyrus, anterior central gyrus, and posterior central gyrus. Associations between brain activation and reward expectancy in the left insula, left middle frontal gyrus, left thalamus, and right middle frontal gyrus were found to be related to proactive aggression. Furthermore, the brain activation regions primarily involved in proactive aggression induced by reward expectancy were the insula, inferior frontal gyrus, inferior temporal gyrus, pallidum, and caudate nucleus. Under conditions of high reward expectancy, participants engage in more proactive aggressive behavior. Reward expectancy involves the activation of reward- and social-cognition-related brain regions, and these associations are instrumental in proactive aggressive decisions.