Outcome Valence Research Articles

Behavioural and neural limits in competitive decision making: The roles of outcome, opponency and observation

To understand the boundaries we set for ourselves in terms of environmental responsibility during competition, we examined a neural index of outcome valence (feedback-related negativity; FRN) in relation to an early index of visual attention (N1), a later index of motivational significance (P3), and, eventual behaviour. In Experiment 1 (n = 36), participants either were (play) or were not (observe) responsible for action selection. In Experiment 2 (n = 36), opponents additionally either could (exploitable) or could not (unexploitable) be beaten. Various failures in reinforcement learning expression were revealed including large-scale approximations of random behaviour. Against unexploitable opponents, N1 determined the extent to which negative and positive outcomes were perceived as distinct categories by FRN. Against exploitable opponents, FRN determined the extent to which P3 generated neural gain for future events. Differential activation of the N1 – FRN – P3 processing chain provides a framework for understanding the behavioural dynamism observed during competitive decision making.

“Bigger” or “better”: the roles of magnitude and valence in “affective bias”

ABSTRACT Negative affective biases are thought to be a key symptom driving and upholding many psychiatric disorders. When presented with ambiguous information, anxious individuals, for example, tend to anticipate lower rewards than asymptomatic individuals (Aylward et al., 2019. Translating a rodent measure of negative bias into humans: the impact of induced anxiety and unmedicated mood and anxiety disorders. Psychological Medicine). The assumption is that this is because anxious individuals assume “worse” outcomes. However, predictions are often made about high and low rewards, so it is not clear whether the bias is due to the valence (the “worse” option) or just magnitude (the lower number). We therefore explored the roles of valence and magnitude in a translational measure of negative affective bias. We adapted a two-alternative forced choice (2AFC) “reward-reward” task into a “punishment-punishment” paradigm, and followed up with “high reward-high punishment” and “low reward-high punishment” variants. The results from the “punishment-punishment” paradigm – a bias towards higher punishments in healthy controls – suggest that it is outcome magnitude that is important. However, this is qualified by the other variants which indicate that both valence and magnitude are important. Overall, our results temper the assumption that negative affective biases observed in tasks using numeric outcomes are solely as a result of subjective outcome valence.

Age Differences in Intertemporal Choice: The Role of Task Type, Outcome Characteristics, and Covariates.

Prior research has revealed age differences in the preferred timing of monetary outcomes, but results are inconsistent across studies. The present study examined the role of task type, outcome characteristics, and a range of theoretically implicated covariates that may contribute to variations in age effects. Two types of intertemporal choice paradigms (temporal discounting and sequence construction) were administered to a diverse life-span sample (n = 287, aged 18-87). The design experimentally manipulated outcome delay (months vs years), amount (hundreds vs thousands), and valence (gain vs loss) while statistically controlling for a range of potential covariates including demographics, affect, personality, time perspective, subjective health, and numeracy. In the temporal discounting task, no significant age differences were observed and this pattern did not differ by outcome delay, amount, or valence. In the sequence-construction task, age was associated with a preference for sequences of decreasing impact in the gain condition but not in the loss condition, whereas outcome delay and amount did not moderate age effects. Age patterns in discounting and sequences preferences remained unchanged after controlling for covariates. These findings converge with prior studies reporting weak or null effects of age in temporal discounting tasks and suggest that inconsistent results are not due to variations in outcome valence, delay, or amount across studies. Findings also add to the scarce evidence for age differences sequence-preferences. After discussing methodological limitations, we consider implications for future research and practice.

Decomposing the effects of context valence and feedback information on speed and accuracy during reinforcement learning: a meta-analytical approach using diffusion decision modeling

Reinforcement learning (RL) models describe how humans and animals learn by trial-and-error to select actions that maximize rewards and minimize punishments. Traditional RL models focus exclusively on choices, thereby ignoring the interactions between choice preference and response time (RT), or how these interactions are influenced by contextual factors. However, in the field of perceptual decision-making, such interactions have proven to be important to dissociate between different underlying cognitive processes. Here, we investigated such interactions to shed new light on overlooked differences between learning to seek rewards and learning to avoid losses. We leveraged behavioral data from four RL experiments, which feature manipulations of two factors: outcome valence (gains vs. losses) and feedback information (partial vs. complete feedback). A Bayesian meta-analysis revealed that these contextual factors differently affect RTs and accuracy: While valence only affects RTs, feedback information affects both RTs and accuracy. To dissociate between the latent cognitive processes, we jointly fitted choices and RTs across all experiments with a Bayesian, hierarchical diffusion decision model (DDM). We found that the feedback manipulation affected drift rate, threshold, and non-decision time, suggesting that it was not a mere difficulty effect. Moreover, valence affected non-decision time and threshold, suggesting a motor inhibition in punishing contexts. To better understand the learning dynamics, we finally fitted a combination of RL and DDM (RLDDM). We found that while the threshold was modulated by trial-specific decision conflict, the non-decision time was modulated by the learned context valence. Overall, our results illustrate the benefits of jointly modeling RTs and choice data during RL, to reveal subtle mechanistic differences underlying decisions in different learning contexts.

Regulatory effect of implicit acceptance during outcome evaluation: The temporal dynamics in an event-related potential study

The acceptance of emotion is important for humans' wellbeing and social functioning. Despite its regulatory advantages, the temporal dynamics of acceptance for regulating decision-related emotion remains unclear. For this purpose, Event-related potentials were recorded for outcome presentation, when participants either in explicit or implicit acceptance condition performed a Gambling Task. Results showed that acceptance effectively regulated emotional experiences, irrespective of how it was realized (explicit/implicit). Compared to viewing condition, explicit acceptance increased overall amplitudes of feedback-related negativity (FRN,180-240 ms) at the early stage and reduced P3 amplitude (240–440 ms) in general at the late stage, regardless of feedback valence or magnitude. By contrast, implicit acceptance did not influence the FRN amplitudes but increased the P3 amplitudes globally, an effect unaffected by feedback valence and magnitude. In addition, the P3 amplitude for explicit acceptance was negatively correlated with the ratio of risky choices, regardless of outcome valence. These results suggest that explicit acceptance is associated with cognitive conflict and resource depletion, while these adverse effects are not engendered during implicit acceptance. These regulatory effects are independent of specific feedback valence and magnitude. These findings highlight the role of implicit acceptance in cognitive demanding context, such as decision-making.

The effect of others’ outcome valence on spontaneous gift-giving behavior

PurposeIn social networking services gift giving, the decision to send a gift is often initiated by spontaneous news about others, who may have recently experienced fortune or misfortune. The purpose of this paper is to show that the valence of the other’s event can affect the empathy experienced by the giver and that the level of empathy affects gift selection behavior.Design/methodology/approachStudy 1 investigated the relationship between empathy and valence of other’s event and the underlying mechanism of changes in self-esteem. Study 2 explored how different levels of empathies lead to different gift selection behavior. Study 3 replicated the results of Study 2 using a different measurement approach.FindingsAcross the three studies, findings consistently suggest that the empathy arising from unexpected news of the others’ fortune was lower compared to that of the others’ misfortune because of threats to self-esteem. In addition, greater empathy prompted gift givers to spend more time and effort in gift selection.Practical implicationsUnderstanding how valence of event experienced by others might motivate givers to engage in selecting a gift online can help retailers increase predictive insights for recommendations.Originality/valueWhile past research focused on ritual gift giving, this research examined spontaneous gift giving. The study is also unique in that the empathy gap between the giver and the receiver is a result of the changes in the psychological state of the giver.

Contextual influence on confidence judgments in human reinforcement learning.

The ability to correctly estimate the probability of one’s choices being correct is fundamental to optimally re-evaluate previous choices or to arbitrate between different decision strategies. Experimental evidence nonetheless suggests that this metacognitive process—confidence judgment- is susceptible to numerous biases. Here, we investigate the effect of outcome valence (gains or losses) on confidence while participants learned stimulus-outcome associations by trial-and-error. In two experiments, participants were more confident in their choices when learning to seek gains compared to avoiding losses, despite equal difficulty and performance between those two contexts. Computational modelling revealed that this bias is driven by the context-value, a dynamically updated estimate of the average expected-value of choice options, necessary to explain equal performance in the gain and loss domain. The biasing effect of context-value on confidence, revealed here for the first time in a reinforcement-learning context, is therefore domain-general, with likely important functional consequences. We show that one such consequence emerges in volatile environments, where the (in)flexibility of individuals’ learning strategies differs when outcomes are framed as gains or losses. Despite apparent similar behavior- profound asymmetries might therefore exist between learning to avoid losses and learning to seek gains.

Sense of agency is modulated by interactions between action choice, outcome valence, and predictability

Sense of agency is a feeling of control over one’s actions to cause sensory events in one’s environment. While previous studies investigated the role of action choice and emotional valence of action outcome in forming implicit agency, the results were not consistent and the relationship between these factors remains unclear. We manipulated both action alternatives available and emotional valences of sounds (either positive or negative) as action outcomes and measured the resultant intentional binding effects in two experiments that differed in predictability of outcome valence. When participants could not predict the valence of action outcomes, they showed stronger sense of agency for negative outcomes determined by their free choice (Experiment 1). Conversely, when participants’ actions caused only outcomes with specific valence, this interaction was not observed (Experiment 2). These findings imply that the implicit processes of agency reflect an integrative context-dependent cognition of consequence of action choice, prior to explicit attribution judgments.

Incidental positive emotion modulates neural response to outcome valence in a monetarily rewarded gambling task.

Choices are influenced by incidental emotions. To understand the neural mechanisms underlying the potential effects of incidental emotions on outcome processing, we conducted two experiments measuring feedback-related negativity (FRN) as a function of outcome (gain and loss) and emotional context. Experiment 1 used happy, neutral, and sad faces. Experiment 2 used pleasant, neutral, and unpleasant emotional scenes. We expected that incidental emotions would influence outcome processing at the behavioral level in line with the cognitive themes associated with each emotion. At a neural level, the effect of emotion based on outcome was expected in FRN. Participants chose one of two gambles presented on either side of an emotional face (Experiment 1) or on the scene (Experiment 2), and were later shown the outcome. Behaviorally, both the experiments showed emotion specific carryover effects on outcome experience in line with the cognitive appraisal tendencies associated with specific emotions. In both experiments, mean amplitude of FRN measured related to the outcome at Fz and FCz showed a significant effect of outcome with larger amplitude for loss compared to gain. The interaction between emotion and outcome was significant at FCz in Experiment 1 and at FPz in Experiment 2. The amplitude difference between loss and gain was larger for positive emotional context compared to neutral and negative emotional contexts, indicating a dopaminergic basis moderating the emotion-outcome processing interaction.

Can I surprise myself? A conceptual framework of surprise self-gifting among consumers

Consumers frequently purchase self-gifts, i.e. gifts for themselves, for a variety of reasons. Recently, certain items bought as self-gifts have begun to incorporate an element of surprise, as in the case of subscription boxes. Little is known to date, however, about how surprise can influence the self-gifting process. Through a qualitative study, we develop a multi-stage framework illustrating how surprise: (a) transforms the initial stages through heightened anticipation and immersion, and (b) bifurcates the final stage depending on the outcome valence, i.e. positive or negative surprise. Implications for manufacturers and retailers of such products and experiences are then discussed.

How actions change liking: The effect of an action's outcome on the evaluation of the action's object.

People like positive objects (e.g., fun activities) and dislike negative objects (e.g., boring activities). However, objects usually do not appear in isolation; They are often objects of an action (e.g., the boring activities were canceled). Using a wide array of stimuli and procedures, 11 experiments (N = 5,573) found that evaluation of objects is biased by the outcome of an action performed on the objects. For example, when participants read that a gene increases the likelihood of possessing the trait kindness (an action with a positive outcome), they evaluated kindness more positively than after reading the gene inhibits the trait (an action with a negative outcome). Conversely, they disliked dishonesty more after reading about genes that increased dishonesty than after reading about genes that decreased dishonesty. The effect was incompatible with logical inference from the information provided. We found evidence that misattribution of the valence of the action's outcome to the action's object contributes to this effect. These findings extend knowledge about the factors that lead to evaluative change. Importantly, the results demonstrate a recursive evaluation process: The valence of the outcome of an action on the object determines the evaluation of the object, but the valence of the outcome is already based on a previous evaluation of the object itself. We discuss the possible implications of our findings to a wide range of research domains, such as moral judgment and economic decisions. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Reference-point centering and range-adaptation enhance human reinforcement learning at the cost of irrational preferences

In economics and perceptual decision-making contextual effects are well documented, where decision weights are adjusted as a function of the distribution of stimuli. Yet, in reinforcement learning literature whether and how contextual information pertaining to decision states is integrated in learning algorithms has received comparably little attention. Here, we investigate reinforcement learning behavior and its computational substrates in a task where we orthogonally manipulate outcome valence and magnitude, resulting in systematic variations in state-values. Model comparison indicates that subjects’ behavior is best accounted for by an algorithm which includes both reference point-dependence and range-adaptation—two crucial features of state-dependent valuation. In addition, we find that state-dependent outcome valuation progressively emerges, is favored by increasing outcome information and correlated with explicit understanding of the task structure. Finally, our data clearly show that, while being locally adaptive (for instance in negative valence and small magnitude contexts), state-dependent valuation comes at the cost of seemingly irrational choices, when options are extrapolated out from their original contexts.

Older adults fail to form stable task representations during model-based reversal inference.

Older adults struggle in dealing with changeable and uncertain environments across several cognitive domains. This has been attributed to difficulties in forming adequate task representations that help navigate uncertain environments. Here, we investigate how, in older adults, inadequate task representations impact on model-based reversal learning. We combined computational modeling and pupillometry during a novel model-based reversal learning task, which allowed us to isolate the relevance of task representations at feedback evaluation. We find that older adults overestimate the changeability of task states and consequently are less able to converge on unequivocal task representations through learning. Pupillometric measures and behavioral data show that these unreliable task representations in older adults manifest as a reduced ability to focus on feedback that is relevant for updating task representations, and as a reduced metacognitive awareness in the accuracy of their actions. Instead, the data suggested older adults' choice behavior was more consistent with a guidance by uninformative feedback properties such as outcome valence. Our study highlights that an inability to form adequate task representations may be a crucial factor underlying older adults' impaired model-based inference.

Preschoolers Focus on Others' Intentions When Forming Sociomoral Judgments.

Many studies suggest that preschoolers initially privilege outcome over intention in their moral judgments. The present findings reveal that, in contrast, even younger preschoolers can privilege intentions when evaluating characters who successfully or unsuccessfully help or hinder a third party in achieving its goal. Following a live-action puppet show originally created for infant populations, children made a forced-choice social judgment (which puppet was liked) and two forced-choice moral judgments (which puppet was nicer, which puppet should be punished), and were asked to explain their punishment allocations. In two experiments (N = 195), 3- and 4-year-olds evaluated characters with distinct intentions to help or to hinder who were associated with either positive or negative outcomes. Both ages judged characters with more positive intentions as nicer, and allocated punishment to characters with more negative intentions; neither of these tendencies depended on the outcomes the characters were associated with. Three-year-olds’ responses were somewhat less consistent than were 4-year-olds’, in that 3-year-olds’ judgments were disrupted by ambiguous harmful intent. Notably, children’s social judgments were less consistent than their moral judgments. In a third and final experiment (N = 100), children evaluated characters with the same intention but who were associated with different outcomes. Children showed inconsistent responding across age and outcome valence, but only 4-year-olds evaluating two characters with positive intentions reliably responded based on outcome. When providing informative responses in all three studies, children most frequently explained their punishment allocations by appealing to the puppet’s (attempted) hindering action or failure to help. These findings raise questions as to what underlies different patterns of response across studies in the literature, and suggests that observing live interactions may facilitate young children’s intention-based moral judgments.

Narcissism and risky decisions: a neurophysiological approach.

Narcissists are prone to risky decision-making, but why? This study tested—via behavioral and event-related potential (ERP) measures—two accounts: deficiencies in error monitoring and deficiencies in action updating. High and low narcissists were engaged in a monetary gambling task by choosing between a high-risk and a low-risk option while the electroencephalogram (EEG) was being recorded. Two ERP components relevant to outcome evaluation—feedback-related negativity (FRN) and P3—were analyzed, with the FRN serving as an index of error monitoring and the P3 as an index of action updating. Generally, high and low narcissists differed in the high-risk condition but not in the low-risk condition. At the behavioral level, high (vs low) narcissists made riskier decisions following high-risk decision outcomes, which was in line with past findings; at the neurophysiological level, while no FRN difference emerged between high and low narcissists, the outcome valence effect (positive vs negative) on the P3 was stronger among low narcissists than high narcissists following high-risk decision outcomes. One possible interpretation of the results is that narcissism is associated with reduced action updating. The findings contribute to the understanding of narcissistic decision-making and self-regulation.

Dorsal Anterior Cingulate Cortices Differentially Lateralize Prediction Errors and Outcome Valence in a Decision-Making Task.

The dorsal anterior cingulate cortex (dACC) is proposed to facilitate learning by signaling mismatches between the expected outcome of decisions and the actual outcomes in the form of prediction errors. The dACC is also proposed to discriminate outcome valence—whether a result has positive (either expected or desirable) or negative (either unexpected or undesirable) value. However, direct electrophysiological recordings from human dACC to validate these separate, but integrated, dimensions have not been previously performed. We hypothesized that local field potentials (LFPs) would reveal changes in the dACC related to prediction error and valence and used the unique opportunity offered by deep brain stimulation (DBS) surgery in the dACC of three human subjects to test this hypothesis. We used a cognitive task that involved the presentation of object pairs, a motor response, and audiovisual feedback to guide future object selection choices. The dACC displayed distinctly lateralized theta frequency (3–8 Hz) event-related potential responses—the left hemisphere dACC signaled outcome valence and prediction errors while the right hemisphere dACC was involved in prediction formation. Multivariate analyses provided evidence that the human dACC response to decision outcomes reflects two spatiotemporally distinct early and late systems that are consistent with both our lateralized electrophysiological results and the involvement of the theta frequency oscillatory activity in dACC cognitive processing. Further findings suggested that dACC does not respond to other phases of action-outcome-feedback tasks such as the motor response which supports the notion that dACC primarily signals information that is crucial for behavioral monitoring and not for motor control.

Utilizing time-frequency amplitude and phase synchrony measure to assess feedback processing in a gambling task.

The neurophysiological mechanisms involved in the evaluation of performance feedback have been widely studied in the ERP literature over the past twenty years, but understanding has been limited by the use of traditional time-domain amplitude analytic approaches. Gambling outcome valence has been identified as an important factor modulating event-related potential (ERP) components, most notably the feedback negativity (FN). Recent work employing time-frequency analysis has shown that processes indexed by the FN are confounded in the time-domain and can be better represented as separable feedback-related processes in the theta (3-7 Hz) and delta (0-3 Hz) frequency bands. In addition to time-frequency amplitude analysis, phase synchrony measures have begun to further our understanding of performance evaluation by revealing how feedback information is processed within and between various brain regions. The current study aimed to provide an integrative assessment of time-frequency amplitude, inter-trial phase synchrony, and inter-channel phase synchrony changes following monetary feedback in a gambling task. Results revealed that time-frequency amplitude activity explained separable loss and gain processes confounded in the time-domain. Furthermore, phase synchrony measures explained unique variance above and beyond amplitude measures and demonstrated enhanced functional integration between medial prefrontal and bilateral frontal, motor, and occipital regions for loss relative to gain feedback. These findings demonstrate the utility of assessing time-frequency amplitude, inter-trial phase synchrony, and inter-channel phase synchrony together to better elucidate the neurophysiology of feedback processing.

Greater mindful eating practice is associated with better reversal learning

Mindfulness-based interventions are thought to reduce compulsive behavior such as overeating by promoting behavioral flexibility. Here the main aim was to provide support for mindfulness-mediated improvements in reversal learning, a direct measure of behavioral flexibility. We investigated whether an 8-week mindful eating intervention improved outcome-based reversal learning relative to an educational cooking (i.e., active control) intervention in a non-clinical population. Sixty-five healthy participants with a wide BMI range (19–35 kg/m2), who were motivated to change their eating habits, performed a deterministic reversal learning task that enabled the investigation of reward- and punishment-based reversal learning at baseline and following the intervention. No group differences in reversal learning were observed. However, time invested in the mindful eating, but not the educational cooking intervention correlated positively with changes in reversal learning, in a manner independent of outcome valence. These findings suggest that greater amount of mindfulness practice can lead to increased behavioral flexibility, which, in turn, might help overcome compulsive eating in clinical populations.

Transient and sustained ERP activity related to feedback processing in the probabilistic selection task

Feedback processing represents a fundamental component of adaptive decision-making. Extensive research using event-related brain potentials (ERPs) has explored the medial frontal feedback negativity (FN), P3a, and P3b components in the context of feedback processing. Additionally, recent studies examining the neural correlates of feedback processing in a gambling task revealed transient and sustained ERP activity over the lateral frontal and posterior regions of the scalp. Consistent with the neuroimaging literature, source analysis has localized this ERP activity to the lateral and medial frontal cortex, as well as the temporal-occipital and medial occipital cortices. The current experiments examine whether the pattern of transient medial frontal and sustained lateral frontal and posterior ERP activity extend to a reinforcement learning task, and may thereby represent a general property of feedback processing. Consistent with a large literature, the ERP data revealed the FN-P3a and P3b. In addition, these data revealed sustained ERP activity over the lateral frontal and posterior regions. These findings contribute to our understanding of the temporal dynamics of feedback processing across broadly distributed cortical and subcortical networks that are sensitive to the valence and probability of positive and negative outcomes.

Attentional bias of students toward negative feedback in bad outcome situations: the mechanism of self-defense

The main goals of the present study were to investigate the effects of outcome valence on attentional bias toward feedback and examine the internal mechanism of self-defense. We systematically manipulated the outcome valence by providing a bogus score in a rational thinking task and recorded the time positive feedback and negative feedback was viewed in experiment 1. We added the intervention of self-affirmation to examine the self-defense mechanism in experiment 2. The results suggest that (1) in good outcome situations, the participants viewed negative feedback longer than positive feedback. There was a tendency to slightly reduce the attention given to negative feedback in bad outcome situations. (2) Self-affirming participants in bad outcome situations increased their viewing time of negative feedback, which supported the activation of defensiveness.