Separate Reward Research Articles

Behavioral and psychiatric correlates of brain responses to social feedback.

Maladaptive responses to peer acceptance and rejection arise in numerous psychiatric disorders in adolescence; yet, homogeneity and heterogeneity across disorders suggest common and unique mechanisms of impaired social function. We tested the hypothesis that social feedback is processed similarly to other forms of feedback (e.g., monetary) by examining the correspondence between the brain's response to social acceptance and rejection and behavioral performance on a separate reward and loss task. We also examined the relationship between these brain responses and depression and social anxiety severity. The sample consisted of one hundred and thirteen 16-21-year olds who received virtual peer acceptance/rejection feedback in an event-related potential (ERP) task. We used temporospatial principal component analysis and identified a component consistent with the reward positivity (RewP) or feedback negativity (FN). RewP to social acceptance was not significantly related to reward bias or the FN to social rejection related to loss avoidance. The relationship between RewP and depression severity, while nonsignificant, was of a similar magnitude to prior studies. Exploratory analyses yielded a significant relationship between lower socioeconomic status (SES) and blunted RewP and between lower SES and heightened loss avoidance and blunted reward bias. These findings build on prior work to improve our understanding of the function of the brain's response to social feedback, while also suggesting a pathway for further study, whereby poverty leads to depression via social and reward learning mechanisms.

In it together: evidence of a preference for the fair distribution of effort in joint action

A growing body of theoretical and empirical work suggests that our sense of fairness implies a preference for divisions of rewards that are proportional to contributions. However, there has been no study testing whether people distribute effort costs according to the expected reward distribution. We hypothesized that when people expect to share the reward of the joint task equally, they will ensure fairness by calibrating their effort investment such as to reduce inequity with respect to joint action partners' effort investment. We developed a task in which participants traded off effort costs against reward. Before making their decision, they observed as their partner performed an effort task. We examined how the perception of the partner's effort modulated effort-based decision-making in joint action, depending on whether participants were in a joint or separate reward structure, and on whether the available reward was known or unknown. Across two lab-based, pre-registered experiments (N = 57), we found support for our hypothesis, and we controlled for other candidate explanations for the observed effort matching effect such as a preference for acting jointly, learning about the value of opportunities afforded by the environment, learning the value of effort and competition.

Effect of Daily Life Reward Loop Functioning on the Course of Depression

Engagement in activities increases positive affect (Reward Path 1), which subsequently reinforces motivation (Reward Path 2), and hence future engagement in activities (Reward Path 3). Strong connections between these three reward loop components are considered adaptive, and might be disturbed in depression. Although some ecological nomentary assessment (EMA) studies have investigated the cross-sectional association between separate reward paths and individuals’ level of depression, no EMA study has looked into the association between individuals’ reward loop strength and depressive symptom course. The present EMA study assessed reward loop functioning (5x/day, 28 days) of 46 outpatients starting depression treatment at secondary mental health services and monitored with the Inventory of Depressive Symptomatology—Self-Report (IDS-SR) during a 7-month period. Results of multilevel regression analyses showed significant within-person associations for Reward Path 1 (b = 0.21, p < .001), Reward Path 2 (b = 0.43, p < .001), and Reward Path 3 (b = 0.20, p < .001). Stronger average reward loops (i.e., within-person mean of all reward paths) did not relate to participants’ improvement in depressive symptoms over time. Path-specific results revealed that Reward Paths 1 and 2 may have partly opposite effects on depressive symptom course. Together, our findings suggest that reward processes in daily life might be best studied separately and that further investigation is warranted to explore under what circumstances strong paths are adaptive or not.

Effort and Appetitive Responding in Depression: Examining Deficits in Motivational and Consummatory Stages of Reward Processing Using the Effort-Doors Task

BackgroundReward sensitivity is a dimensional construct central to understanding the nature of depression. Psychophysiological research on this construct has primarily focused on the reward positivity, an event-related potential (ERP) that indexes consummatory reward sensitivity. This study extended prior research by focusing on ERPs that index the motivational component of reward. MethodsA novel effort-for-reward task was used to elicit motivational and consummatory ERPs. Groups consisting of 34 participants with depression and 32 participants without depression were compared across a range of reward-related ERPs. ResultsParticipants with depression exhibited reduced responsivity to effort completion cues following high effort expenditure, reduced anticipation of rewards after low effort expenditure (i.e., the stimulus preceding negativity), and reduced reward positivity following high effort expenditure. ERPs occurring prior to reward receipt accounted for unique variance in depression status and differentiated between subgroups of depressed individuals. ConclusionsFindings support the utility of leveraging multiple ERPs that index separate reward processing deficits to better characterize depression and depressive subtypes.

Hierarchical, Discontinuous Agent Reinforcement Learning Rewards in Complex Military-Oriented Environments

Artificially intelligent agents are seeing increased adoption in both the video game and simulation industry for training, education, and entertainment purposes. These systems often need realistic and believable opponents that must achieve objectives in the face of competing and contradictory priorities and frequently require the rapid creation of a wide spectrum of agents with disparate behaviors that reflect tactical realism. This in turn drives the need for the dynamic training of such agents from available source data. Approaches to do so have yet to be widely investigated due to the smaller scales of these simulation environments. This paper discusses techniques to quickly design and generate a variety of AI agents that follow desired tactics and procedures, including realistic situations that require trade-off decisions between competing objectives. Techniques described include an investigation into deep reinforcement agents that have separable reward structures and can prioritize and re-prioritize goals based on a hierarchy.

Mapping reward mechanisms by intracerebral self-stimulation in the rhesus monkey (Macaca mulatta).

The objective of the study was to identify brain structures that mediate reward as evidenced by positive reinforcing effects of stimuli on behavior. Testing by intracerebral self-stimulation enabled monkeys to inform whether activation of ~2900 sites in 74 structures of 4 sensorimotor pathways and 4 modulatory loop pathways was positive, negative or neutral. Stimulation was rewarding at 30% of sites, negative at 17%, neutral at 52%. Virtually all (99%) structures yielded some positive or negative sites, suggesting a ubiquitous distribution of pathways transmitting valence information. Mapping of sites to structures with dense versus sparse dopaminergic (DA) or noradrenergic (NA) innervation showed that stimulation of DA-pathways was rewarding or neutral. Stimulation of NA-pathways was not rewarding. Stimulation of association areas was generally rewarding; stimulation of purely sensory or motor structures was generally negative. Reward related more to structures' sensorimotor function than to density of DA-innervation. Stimulation of basal ganglia loop pathways was rewarding except in lateral globus pallidus, an inhibitory structure in the negative feedback loop; stimulation of the cerebellar loop was rewarding in anterior vermis and the spinocerebellar pathway; and stimulation of the hippocampal CA1 loop was rewarding. While most positive sites were in the DA reward system, numerous sites in sparsely DA-innervated posterior cingulate and parietal cortices may represent a separate reward system. DA-density represents concentrations of plastic synapses that mediate acquisition of new synaptic connections. DA-sparse areas may represent innate, genetically programmed reward-associated pathways. Implications of findings in regard to response habituation and addiction are discussed.

WCSAC: Worst-Case Soft Actor Critic for Safety-Constrained Reinforcement Learning

Safe exploration is regarded as a key priority area for reinforcement learning research. With separate reward and safety signals, it is natural to cast it as constrained reinforcement learning, where expected long-term costs of policies are constrained. However, it can be hazardous to set constraints on the expected safety signal without considering the tail of the distribution. For instance, in safety-critical domains, worst-case analysis is required to avoid disastrous results. We present a novel reinforcement learning algorithm called Worst-Case Soft Actor Critic, which extends the Soft Actor Critic algorithm with a safety critic to achieve risk control. More specifically, a certain level of conditional Value-at-Risk from the distribution is regarded as a safety measure to judge the constraint satisfaction, which guides the change of adaptive safety weights to achieve a trade-off between reward and safety. As a result, we can optimize policies under the premise that their worst-case performance satisfies the constraints. The empirical analysis shows that our algorithm attains better risk control compared to expectation-based methods.

Control Over Patch Encounters Changes Foraging Behaviour

Foraging is a common decision problem in natural environments. When new exploitable sites are always available, a simple optimal strategy is to leave a current site when its return falls below a single average reward rate. Here, we examined foraging in a more structured environment, with a limited number of sites that replenished at different rates and had to be revisited. When participants could choose sites, they visited fast-replenishing sites more often, left sites at higher levels of reward, and achieved a higher net reward rate. Decisions to exploit-or-leave a site were best explained with a computational model estimating separate reward rates for each site. This suggests option-specific information can be used to construct a threshold for patch leaving in some foraging settings, rather than a single average reward rate.

Parental feeding practices across child's weight status: Evidence of the Italian validation of the Child Feeding Questionnaire.

The main aim of this study was to assess the psychometric proprieties of the Child Feeding Questionnaire (CFQ) in Italian mothers. Mothers completed the Italian version of the CFQ, and children's anthropometric data were collected. Construct validity of the CFQ was assessed by comparing three different models: (a) a seven correlated factors model in which all items were analysed; (b) a seven correlated factors model with composite items based on the Restriction factor and (c) an eight correlated factors model with a separate Reward factor. Measurement invariance using BMI categories and gender was evaluated. Furthermore, discriminant validity with group comparison was performed between BMI categories and gender. Italy. A total of 1253 6-year-old Italian children (53·9 % male) attending elementary school (1st grade) and their mothers (mean age = 38·22 years; sd = 4·89) participated in this study. The eight-factor model with a separate reward factor provided the best fit for the data. The strict invariance of the CFQ across child BMI categories and gender was confirmed. The CFQ internal consistency was acceptable for most subscales. However, two subscales showed no adequate values. As expected, the CFQ scales showed significant differences between BMI categories, while no gender-related differences were found. The study indicated the Italian version of the CFQ to be factorially valid for assessing parental feeding practices of 6-year-old children across BMI categories. Future research should address low internal consistency in some of the CFQ subscales.

The effect of concurrent reward on aversive information processing in the brain

Neural networks for the processing of appetitive and aversive information, in isolation, have been well characterized. However, how the brain integrates competing signals associated with simultaneous appetitive and aversive information is less clear. In particular, it is unknown how the presence of concurrent reward modulates the processing of an aversive event throughout the brain. Here, we utilized a four-armed bandit task in an fMRI study to measure the representation of an aversive electric shock with and without the simultaneous receipt of monetary reward. Using a region of interest (ROI) approach, we first identified regions activated by the experience of aversive electric shock, and then measured how this shock-related activation is modulated by concurrent reward using independent data. Informed by prior literature and our own preliminary data, analyses focused on the dorsolateral prefrontal cortex, anterior and posterior insula, anterior cingulate cortex, and the thalamus and somatosensory cortex. We hypothesized that the neural response to punishment in these ROIs would be attenuated by the presence of concurrent reward. However, we found no evidence of concurrent reward attenuating the neural response to punishment in any ROI and also no evidence of concurrent punishment attenuating the neural response to reward in exploratory analyses. Altogether, our findings are consistent with the idea that neural networks responsible for the processing of reward and punishment signals are largely independent of one another, and that representations of overall value or utility are arrived at through the integration of separate reward and punishment signals at later stages of information processing.

The relations between temporal and social perceptual biases: Evidence from perceptual matching

We report a new “now-bias” effect on simple perceptual matching between shapes and labels and examined the relation between this now-bias effect and the self-bias previously established with this task (Sui, He, & Humphreys, Journal of Experimental Psychology: Human Perception and Performance, 38, 1105–1117, 2012). The perceptual biases favoring present-relevant and self-relevant information were correlated with each other, suggesting a common underlying mechanism. Nevertheless, temporal biases in decision making, specifically in temporal discounting, correlated with the perceptual self-bias but not with the perceptual now-bias. We suggest that common attentional biases to present-relevant and self-relevant information mediate perceptual prioritization, whereas temporal discounting is likely involved in a separate reward evaluation mechanism that relates to self-bias processes.

An Affect Misattribution Pathway to Perceptions of Intrinsic Reward

Intrinsic rewards are typically thought to stem from an activity's inherent properties and not from separable rewards one receives from it. Yet, people may not consciously notice or remember all the subtle external rewards that correspond with an activity and may misattribute some directly to the activity itself. We propose that perceptions of intrinsic reward can often be byproducts of misattributed causal inference, and present some initial evidence that perceptions of intrinsic reward can in fact increase when words pertaining to an activity are subtly paired with pleasant context cues. Importantly, these effects follow classic boundary conditions of both misattribution and intrinsic motivation, insofar as they were extinguished when participants could make a proper source attribution and/or when the activity became associated with a blatant external reward. We further propose a distinction can be made between authentically “intrinsic” rewards and the illusion of intrinsic rewards caused by misattribut...

The Emerging Neuroscience of Intrinsic Motivation: A New Frontier in Self-Determination Research.

Intrinsic motivation refers to people’s spontaneous tendencies to be curious and interested, to seek out challenges and to exercise and develop their skills and knowledge, even in the absence of operationally separable rewards. Over the past four decades, experimental and field research guided by self-determination theory (SDT; Ryan and Deci, 2017) has found intrinsic motivation to predict enhanced learning, performance, creativity, optimal development and psychological wellness. Only recently, however, have studies begun to examine the neurobiological substrates of intrinsic motivation. In the present article, we trace the history of intrinsic motivation research, compare and contrast intrinsic motivation to closely related topics (flow, curiosity, trait plasticity), link intrinsic motivation to key findings in the comparative affective neurosciences, and review burgeoning neuroscience research on intrinsic motivation. We review converging evidence suggesting that intrinsically motivated exploratory and mastery behaviors are phylogenetically ancient tendencies that are subserved by dopaminergic systems. Studies also suggest that intrinsic motivation is associated with patterns of activity across large-scale neural networks, namely, those that support salience detection, attentional control and self-referential cognition. We suggest novel research directions and offer recommendations for the application of neuroscience methods in the study of intrinsic motivation.

The Merit Economy

A number of studies report, that remuneration - in addition to being a carrier of exchange value - has also signaling component, perceived by workers as a measure of their competence, social rank, or appreciation from society. Empirical and experimental evidence also suggests, that the signaling function is more reliable predictor of the workers’ economic performance, than its exchange value. The first claim of the presented paper is that decoupling the two functions into separate rewards can address certain problems associated with the existing remuneration mechanism, in particular distribution of exchange value, leading to income inequality. In the second claim, the common assumption concerning profit ownership is challenged, by pointing out that profit is not the result of a worker’s effort and therefore its exchange value should be shared in accordance with Rawls’s 2nd principle of justice. At the same time the motivationally salient signaling component of profit is not to be distributed, as it is measure of the rewarded worker’s intrinsic abilities. In the second part of the paper, framework of an economic system is proposed, based on these ideas.

Stress of underworked and overpaid

Purpose Frequent absences from work can be highly disruptive, while also potentially indicating problematic working conditions that can lead to increased withdrawal behavior. The purpose of this paper is to test the predictive capability of an expanded effort-reward imbalance model on employee absenteeism within the context of policing. Design/methodology/approach Three separate reward systems are identified by the effort-reward imbalance model. In this study, the authors assessed these individual components for their contribution to officer withdrawal behavior in the form of absenteeism frequency. Data were gathered from officers within a large Australian police agency. Findings Findings indicate that there was a strong influence of social rewards, such as social support and recognition in the workplace on officer absenteeism rates. Low workload was associated with a higher frequency of absenteeism, suggesting a potential underloading effect. There were a number of significant interactions providing support for the effort-reward imbalance mechanism and the separation of the reward construct. Security rewards were particularly influential and significantly moderated the relationship between effort and absenteeism. Originality/value This paper considers an expanded model of worker strain and contributes a longitudinal assessment of the association between perceived effort and reward systems and worker absenteeism.

An application of an extended effort-reward imbalance model to police absenteeism behaviour

Purpose– Frequent absences from work can be highly disruptive, whilst also potentially indicating problematic working conditions that can lead to increased withdrawal behaviour. The purpose of this paper is to test the predictive capability of an expanded effort-reward imbalance model on employee absenteeism within the context of policing.Design/methodology/approach– Three separate reward systems are identified by the effort-reward imbalance model. In this study, the authors assessed these individual components for their contribution to officer withdrawal behaviour in the form of absenteeism frequency. Data were gathered from a sample of operational officers (n=553) within a large Australian police agency.Findings– Findings indicate that there was a strong influence of social rewards such as social support and recognition in the workplace on officer absenteeism rates. Low workload was associated with a higher frequency of absenteeism suggesting a potential underloading effect. There were a number of significant interactions providing support for the effort-reward imbalance mechanism and the separation of the reward construct. Security rewards were particularly influential and significantly moderated the relationship between effort and absenteeism.Research limitations/implications– Differential effects of occupational rewards were identified in the study, indicating that there are significant opportunities for expansion of the effort-reward imbalance model along with opportunities for HRM practitioners in terms of employee recognition and remuneration programmes. This research was focused on a specific sample of operational officers, therefore should be expanded to include multiple occupational groups.Originality/value– This paper considers and expanded model of worker strain and contributes a longitudinal assessment of the association between perceived effort and reward systems and worker absenteeism.

Ventral Striatum Response During Reward and Punishment Reversal Learning in Unmedicated Major Depressive Disorder

Affective biases may underlie many of the key symptoms of major depressive disorder, from anhedonia to altered cognitive performance. Understanding the cause of these biases is therefore critical in the quest for improved treatments. Depression is associated, for example, with a negative affective bias in reversal learning. However, despite the fact that reversal learning is associated with striatal response in healthy individuals and depressed individuals exhibit attenuated striatal function on multiple tasks, studies to date have not demonstrated striatal involvement in the negative bias in reversal learning in depression. In this study, the authors sought to determine whether this may be because reversal learning tasks conventionally used to study behavior examine reversals only on the basis of unexpected punishment and therefore do not adequately separate reward- and punishment-based behavior. The authors used functional MRI to compare the hemodynamic response to a reversal learning task with mixed reward- and punishment-based reversal stages between individuals with unmedicated major depressive disorder (N=13) and healthy comparison subjects (N=14). Impaired reward (but not punishment) reversal accuracy was found alongside attenuated anteroventral striatal response to unexpected reward in depression. Attenuated neurophysiological response of the anteroventral striatum may reflect dysfunction in circuits involving afferent projections from the orbitofrontal, limbic, and/or mesostriatal dopaminergic pathways, which conceivably may, together with the ventral striatum, underlie anhedonia in depression. Learning to appreciate and enjoy positive life experiences is critical for recovery from depression. This study pinpoints a neural target for such recovery.

REINFORCEMENT LEARNING IN NON-MARKOV CONSERVATIVE ENVIRONMENT USING AN INDUCTIVE QUALITATIVE MODEL

The majority of real-world processes, such as power plants, banking and retail businesses, are non-Markov processes, being conservative systems with stochastic supply and demand. As an example, a retail process possesses long-term memory of the customer's experience and market price drift that deviates from the Markov property. Modeling the reward in this process is directed towards actions that have to be executed daily in order to support it. These actions are further severely distracted by the hidden periodicity of customer behavior on a monthly and weekly basis. Alternative solutions in the retail business are achieved using a retail potential market model and a pricing policy based on demography. The policy of non-Markov behavior has not been intensively studied, although the literature indicates the non-Markov nature of many real process models, such as bank rating migrations. A solution is proposed, based on day-to-day data collection from point-of-sale (POS) locations, synthesizing the reward function from separate sale component rewards using qualitative models, and indicating the most outstanding sale groups that form the reward model. The normalization of POS data has been used for the elimination of periodicities and of non-Markov features of the process data. Reinforcement learning has been additionally supported by artificial corrections of the normalized reward function, and thus the obtained models used for recognition of the most promising and most defective hidden retail product groups. Model data were analyzed for the statistical significance of the obtained results, comparing normalized and non-normalized sales data distributions. The method is simple and effective, being applicable to each POS separately, for a complex retail business network, as well as for other conservative environments. The obtained qualitative correlations of model and reward function lie between 0.72 and 0.95, even for the simple cases presented.

Individuals Family History Positive for Alcoholism Show Functional Magnetic Resonance Imaging Differences in Reward Sensitivity That Are Related to Impulsivity Factors

Substance-abusing individuals tend to display abnormal reward processing and a vulnerability to being impulsive. Detoxified alcoholics show differences in regional brain activation during a monetary incentive delay task. However, there is limited information on whether this uncharacteristic behavior represents a biological predisposition toward alcohol abuse, a consequence of chronic alcohol use, or both. We investigated proposed neural correlates of substance disorder risk by examining reward system activity during a monetary incentive delay task with separate reward prospect, reward anticipation, and reward outcome phases in 30 individuals with and 19 without family histories of alcoholism. All subjects were healthy, lacked DSM-IV past or current alcohol or substance abuse histories, and were free of illegal substances as verified by a urine toxicology screening at the time of scanning. Additionally, we explored specific correlations between task-related nucleus accumbens (NAcc) activation and distinct factor analysis-derived domains of behavioral impulsivity. During reward anticipation, functional magnetic resonance imaging data confirmed blunted NAcc activation in family history positive subjects. In addition, we found atypical activation in additional reward-associated brain regions during additional task phases. We further found a significant negative correlation between NAcc activation during reward anticipation and an impulsivity construct. Overall, results demonstrate that sensitivity of the reward circuit, including NAcc, is functionally different in alcoholism family history positive individuals in multiple regards.

The Role of Effort–Reward Imbalance for Reservists on a Military Deployment

The present study extended the effort–reward imbalance (ERI) model (Siegrist, 1996b), first by differentiating between separate reward components, second by including intrinsic reward, and third by additionally predicting a non-health outcome (i.e., turnover intention) in a military context. Regression analyses of 789 reservists found financial and esteem reward to moderate the impact of extrinsic effort on depressive symptoms, whereas only financial reward moderated the effort turnover intentions relationship. Results supported the usefulness of the extended ERI model in the military context. Relevant areas for future research on the military effort–reward perspective are discussed.