Reward Value Research Articles

Cross-species translational paradigms for assessing positive valence system as defined by the RDoC matrix.

Functions associated with processing reward-related information are fundamental drivers of motivation, learning, and goal-directed behavior. Such functions have been classified as the positive valence system under the Research Domain and Criteria (RDoC) criteria and are negatively impacted across a range of psychiatric disorders and mental illnesses. The positive valence system is composed of three comprehensive categories containing related but dissociable functions that are organized into either Reward Responsiveness, Reward Learning, or Reward Valuation. The presence of overlapping behavioral dysfunction across diagnostic mental disorders is in-part what motivated the RDoC initiative, which emphasized that the study of mental illness focus on investigating relevant behavior and cognitive functions and their underlying mechanisms, rather than separating efforts on diagnostic categories (i.e., transdiagnostic). Moreover, the RDoC approach is well-suited for preclinical neuroscience research, as the rise in genetic toolboxes and associated neurotechnologies enables researchers to probe specific cellular targets with high specificity. Thus, there is an opportunity to dissect whether behaviors and cognitive functions are supported by shared or distinct neural mechanisms. For preclinical research to effectively inform our understandings of human behavior however, the cognitive and behavioral paradigms should have predictive, neurobiological, and pharmacological predictive validity to the human test. Touchscreen-based testing systems provide a further advantage for this endeavor enabling tasks to be presented to animals using the same media and task design as in humans. Here, we outline the primary categories of the positive valence system and review the work that has been done cross-species to investigate the neurobiology and neurochemistry underlying reward-related functioning. Additionally, we provide clinical tasks outlined by RDoC, along with validity and/or need for further validation for analogous rodent paradigms with a focus on implementing the touchscreen-based cognitive testing systems.

Ecological momentary assessment of delay discounting, reward valuation, and craving in very light cigarette users.

Heightened delay discounting has been linked to adverse smoking cessation outcomes, including among light cigarette users. Few studies have evaluated delay discounting's proposed mechanism, preference reversal (concurrent increases in valuation of/craving for desired objects), and none have done so in naturalistic settings. We examined how person-level delay discounting moderated the within-person association between cigarette valuation and craving among very light daily cigarette users who were financially incentivized to abstain. Forty participants completed a baseline delay-discounting task and intermittent ratings of cigarette valuation and craving during the incentivized abstinence attempt. Subjects earned monetary rewards for abstinence on a descending schedule (e.g., $20 on Days 1 and 2 and $2.50 on Days 9 and 10). Consistent with preference reversals, there was a positive association between cigarette valuation and craving. This relation was moderated by delay discounting (stronger among those with low discounting rates) and by monetary reinforcement amount (stronger on days with low reinforcement). Additionally, subjects were more likely to report stronger cravings on days with high monetary reinforcement, with this effect moderated by delay discounting (stronger among those with low discounting rates). The results suggest that heightened delay discounting may not confer risk for preference reversal among very light daily cigarette users who are attempting abstinence.

Blunted sensitivity to expected value during risky decision making in individuals with problematic pornography use.

Neurobiological models of addiction posit that addiction manifests through an amplified salience towards addiction-associated stimuli and a diminished responsiveness to non-addiction-related incentives. However, existing research on reward processing in individuals with problematic pornography use (PPU) has primarily been limited to sexual cue reactivity. In this event-related potential (ERP) study, we employed a risky decision-making task involving 30 individuals with PPU and 33 healthy controls (HCs) to examine the effects of PPU on non-pornographic (money) reward valuation. Compared to HCs, individuals with PPU exhibited compromised sensitivity to monetary rewards. Specifically, while the HC group demonstrated a differential response in late positive potential (LPP) amplitude to various expected value (EV) levels, this pattern was absent in the PPU group. This impairment was associated with poorer adaptive decision-making, as evidenced by PPU participants' inability to adjust risk choices based on changes in EV, leading to a propensity for riskier decisions in disadvantageous situations. The findings of impaired monetary evaluation in individuals with PPU may potentially explain why they continually pursue pornographic rewards while showing insensitivity to other rewards in daily life. Consequently, treatment development strategies may prioritize improving sensitivity to non-pornographic rewards within this population.

Revisiting the musical reminiscence bump: insights from neurocognitive and social brain development in adolescence.

Music listening is enjoyed across the lifespan and around the world. This has spurred many theories on the evolutionary purpose of music. The Music for Social Bonding hypothesis posits that the human capacity to make music evolved for the purpose of creating and preserving relationships between one another. Considering different time periods of music use across the lifespan, adolescence is especially a period of social reorientation away from family towards peers, characterized by new social bonds and increased prosocial behavior. This shift is accompanied by notable structural and functional changes in brain networks supporting reward processing and prosocial behavior. Reviewing the extant literature on developmental cognitive neuroscience and adolescent music use, we propose that neurocognitive changes in the reward system make adolescence an ideal developmental time window for investigating interactions between prosocial behavior and reward processing, as adolescence constitutes a time of relative increase in music reward valuation. Testing this hypothesis may clarify our understanding of developmental trajectories in music reward valuation, and offer insights into why music from adults' adolescence holds a great deal of personal significance.

Development and validation of Intertemporal Decision-Making Ability Scale for early adolescents

ABSTRACT Objective Intertemporal decision-making ability refers to the ability of individuals to weigh costs and benefits at different time points and make choices, which is crucial for adolescents to resist temptation, improve self-control, and prospection. However, there are rarely studies that have been conducted to directly measure the intertemporal decision-making ability of adolescents. Method In this study, adolescents aged 9 to 15 years old were include to finish the measurement, and the intertemporal decision-making ability scale was developed based on three neural networks of intertemporal decision-making. After conducting the exploratory factor analysis, the final scale had 14 items in three dimensions. Results The results showed that the intertemporal decision-making ability scale for early adolescents contains three dimensions, namely, cognitive control, reward valuation, and prospection, with a total of 14 items. Conclusion The intertemporal decisionmaking ability scale developed in this study can effectively assess the intertemporal decisionmaking ability for early adolescents..

Can Stress Put Digital Privacy at Risk? Evidence from a Controlled Experiment Examining the Impact of Acute Stress on Privacy Decisions on a Simulated Social Network Site.

Internet and Communication Technologies (ICTs) can foster efficient communication and knowledge acquisition, but there are also tradeoffs in terms of risks to one's privacy. Previous research, including work with the privacy calculus framework, indicates that factors such as perceived risks and benefits of using ICTs, ICT trust, and general privacy concerns can influence individuals' digital privacy-related decisions. One pervasive psychological factor that may potentially alter such privacy-related behaviors is acute stress. Acute stress can promote risk-seeking behaviors and a tendency to prefer immediate rewards over delayed, greater value rewards. However, the effect of acute stress in the applied context of privacy decision making is relatively unknown. Participants (N = 143) in this study were randomly assigned to either an acute stress task (socially evaluated cold pressor task) or an active control task (lukewarm water alone). Results revealed that acute stress condition increased information disclosure, as indexed by accepting more online cookies, sharing one's location more frequently, and revealing greater willingness to self-disclose personal information. In addition, the impact of individuals' levels of perceived risk and benefits, trust, and privacy concern on privacy decision making was examined. However, none of these constructs consistently influenced privacy decisions over and above the effect of stress. Overall, our findings suggest that acute stress has robust, independent influence on privacy decision making.

Exaggerated frontoparietal control over cognitive effort-based decision-making in young women with anorexia nervosa.

Effortful tasks are generally experienced as costly, but the value of work varies greatly across individuals and populations. While most mental health conditions are characterized by amotivation and effort avoidance, individuals with anorexia nervosa (AN) persistently engage in effortful behaviors that most people find unrewarding (food restriction, excessive exercise). Current models of AN differentially attribute such extreme weight-control behavior to altered reward responding and exaggerated cognitive control. In a novel test of these theoretical accounts, we employed an established cognitive effort discounting paradigm in combination with fMRI in young acutely underweight female patients with AN (n = 48) and age-matched healthy controls (HC; n = 48). Contrary to the hypothesis that individuals with AN would experience cognitive effort (operationalized as N-back task performance) as less costly than HC participants, groups did not differ in the subjective value (SV) of discounted rewards or in SV-related activation of brain regions involved in reward valuation. Rather, all group differences in both behavior (superior N-back performance in AN and associated effort ratings) and fMRI activation (increased SV-related frontoparietal activation during decision-making in AN even for easier choices) were more indicative of increased control. These findings suggest that while effort discounting may be relatively intact in AN, effort investment is high both when performing demanding tasks and during effort-based decision-making; highlighting cognitive overcontrol as an important therapeutic target. Future research should establish whether exaggerated control during effort-based decision-making persists after weight-recovery and explore learning the value of effort in AN with tasks involving disorder-relevant effort demands and rewards.

Adolescent-Preferred financial incentives to promote type 1 diabetes Self-Care: A discrete choice experiment

AimsThis study aimed to quantify preferences for the characteristics of a financial incentives program that would motivate adolescent engagement in type 1 diabetes (T1D) self-care. MethodWe performed a discrete choice experiment with 12–18 year-olds with T1D from two pediatric hospital endocrinology clinics (n = 317). We identified key attributes of incentives: (1) monthly value of the reward, (2) payment structure, and (3) difficulty of incentivized behaviors. In twelve choice questions, adolescents chose the incentive option from a pair of profiles that was more likely to motivate them to increase adherence to recommended self-care. Options presented were tailored to adolescents’ T1D technology use and perceived difficulty of completing each behavior. We analyzed data using a conditional logit model. ResultsThe value of the reward accounted for 60.8% of preferences. Adolescents were willing to accept lower value rewards when incentive payments used positive vs. negative reinforcement (−$10.88 (95% CI: −$12.60, −9.24)) and preferred higher incentives for performing hard vs. easier behaviors (+$14.92 (95% CI: +$12.66, +$17.28)). ConclusionsStated preferences can inform intervention design. Future research will evaluate the external validity of the discrete choice experiment-informed intervention design by assessing adolescent health and behavioral outcomes in a randomized controlled trial.

Parvalbumin expression does not account for discrete electrophysiological profiles of glutamatergic ventral pallidal subpopulations

The ventral pallidum (VP) has emerged as a critical node in the mesolimbic reward system. Modulating the VP can impact the subjective valuation of rewards, reward motivation, and reward seeking under conflict, making it an attractive target for clinical neuromodulation therapies that manage substance use disorders. To understand how to rationally modulate the VP, we need a better understanding of the electrophysiological properties of VP neurons and the molecular and biophysical determinants of these properties. Here, we used patch-clamp electrophysiology to characterize the intrinsic properties of glutamatergic VP (VPGlu) neurons and observed two distinct electrophysiological profiles: VPGlu neurons that undergo depolarization block in response to progressively increasing current injection amplitudes and those that were resistant to depolarization block. To explore the mechanisms that could contribute to these distinct profiles, we used targeted ribosome affinity purification to identify ion channel subunits and regulatory proteins by isolating actively transcribed mRNA selectively from VPGlu neurons. We then used this transcriptomic information to implement a Markov Chain Monte Carlo method to parameterize a large population of biophysically distinct multicompartment models of VPGlu neurons conforming to either subpopulation. Based on prior literature suggesting parvalbumin (PV) is expressed in a subset of VPGlu neurons, and that PV expression governs the firing properties of those neurons, we tested the hypothesis that PV expression accounted for differences in subgroups, by increasing the maximal firing frequency and conferring resistance to depolarization block. In contrast, our model determined that PV expression at physiological levels had no effect on maximum firing rate. However, supraphysiological expression levels of PV appeared to induce a depolarization block in previously depolarization block-resistant neuron models, suggesting that other intracellular calcium-binding proteins could play a role in determining the firing phenotype of VPGlu neurons. We corroborated this result with single-cell patch-clamp RT-PCR, which confirmed that PV expression did not distinguish the two electrophysiologically distinct subpopulations. Together, these findings establish that VPGlu neurons are composed of biophysically distinct subpopulations that have not been appreciated in prior studies interrogating the function of this population. With the advent of novel tools for cell-type specific pharmacology and targeted neurostimulation, this understanding will be critical for developing strategies to rationally modulate VPGlu cells to treat disorders characterized by maladaptive reward seeking.

Value-Based Subgoal Discovery and Path Planning for Reaching Long-Horizon Goals.

Learning to reach long-horizon goals in spatial traversal tasks is a significant challenge for autonomous agents. Recent subgoal graph-based planning methods address this challenge by decomposing a goal into a sequence of shorter-horizon subgoals. These methods, however, use arbitrary heuristics for sampling or discovering subgoals, which may not conform to the cumulative reward distribution. Moreover, they are prone to learning erroneous connections (edges) between subgoals, especially those lying across obstacles. To address these issues, this article proposes a novel subgoal graph-based planning method called learning subgoal graph using value-based subgoal discovery and automatic pruning (LSGVP). The proposed method uses a subgoal discovery heuristic that is based on a cumulative reward (value) measure and yields sparse subgoals, including those lying on the higher cumulative reward paths. Moreover, LSGVP guides the agent to automatically prune the learned subgoal graph to remove the erroneous edges. The combination of these novel features helps the LSGVP agent to achieve higher cumulative positive rewards than other subgoal sampling or discovery heuristics, as well as higher goal-reaching success rates than other state-of-the-art subgoal graph-based planning methods.

The impact of subclinical psychotic symptoms on delay and effort discounting: Insights from behavioral, computational, and electrophysiological methods

BackgroundThe ability to value rewards is crucial for adaptive behavior and is influenced by the time and effort required to obtain them. Impairments in these computations have been observed in patients with schizophrenia and may be present in individuals with subclinical psychotic symptoms (PS). MethodsIn this study, we employed delay and effort-discounting tasks with food rewards in thirty-nine participants divided into high and low levels of PS. We investigated the underlying mechanisms of effort-discounting through computational modelling of dopamine prefrontal and subcortical circuits and the electrophysiological biomarker of both delay and effort-discounting alterations through resting-state frontal alpha asymmetry (FAA). ResultsResults revealed greater delay discounting in the High PS group compared to the Low PS group but no differences in the effort discounting task. However, in this task, the same levels of estimated dopamine release were associated with a lower willingness to exert effort for high-calorie food rewards in High PS participants compared to Low PS participants. Although there were no significant differences in FAA between the High PS and Low PS groups, FAA was significantly associated with the severity of participants' negative symptoms. ConclusionsOur study suggests that the dysfunction in temporal and effort cost computations, seen in patients with schizophrenia, may be present in individuals with subclinical PS. These findings provide valuable insight into the early vulnerability markers (behavioral, computational, and electrophysiological) for psychosis, which may aid in the development of preventive interventions. These findings are preliminary and warrant further investigation.

Functional connectivity changes in the brain of adolescents with internet addiction: A systematic literature review of imaging studies

Internet usage has seen a stark global rise over the last few decades, particularly among adolescents and young people, who have also been diagnosed increasingly with internet addiction (IA). IA impacts several neural networks that influence an adolescent’s behaviour and development. This article issued a literature review on the resting-state and task-based functional magnetic resonance imaging (fMRI) studies to inspect the consequences of IA on the functional connectivity (FC) in the adolescent brain and its subsequent effects on their behaviour and development. A systematic search was conducted from two databases, PubMed and PsycINFO, to select eligible articles according to the inclusion and exclusion criteria. Eligibility criteria was especially stringent regarding the adolescent age range (10–19) and formal diagnosis of IA. Bias and quality of individual studies were evaluated. The fMRI results from 12 articles demonstrated that the effects of IA were seen throughout multiple neural networks: a mix of increases/decreases in FC in the default mode network; an overall decrease in FC in the executive control network; and no clear increase or decrease in FC within the salience network and reward pathway. The FC changes led to addictive behaviour and tendencies in adolescents. The subsequent behavioural changes are associated with the mechanisms relating to the areas of cognitive control, reward valuation, motor coordination, and the developing adolescent brain. Our results presented the FC alterations in numerous brain regions of adolescents with IA leading to the behavioural and developmental changes. Research on this topic had a low frequency with adolescent samples and were primarily produced in Asian countries. Future research studies of comparing results from Western adolescent samples provide more insight on therapeutic intervention.

Translational research in punishment learning.

Punishment learning is learning of the causal relationship between responses and their adverse or undesirable consequences. Here, we review our translational approach for understanding whether, when, and how individuals differ in what they learn during punishment, and how these differences in learning may drive persistent poor or maladaptive decisions. We show that individual differences in punishment insensitivity can emerge from differences between individuals in what they learn about punishment (instrumental contingency knowledge), rather than differences in aversive valuation, reward valuation, general (impulsivity), or specific (habit) behavioral control. These differences in instrumental contingency knowledge are shared with and can be studied in other animals. Our approach has strong construct and predictive validity, providing a robust translational platform for studying how punishment learning and decision making may contribute to neuropsychiatric disorders. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

How subjective idea valuation energizes and guides creative idea generation.

What drives us to search for creative ideas, and why does it feel good to find one? While previous studies demonstrated the positive influence of motivation on creative abilities, how reward and subjective values play a role in creativity remains unknown. This study proposes to characterize the role of individual preferences (how people value ideas) in creative ideation via behavioral experiments and computational modeling. Using the Free Generation of Associates Task coupled with rating tasks, we demonstrate the involvement of valuation processes during idea generation: Preferred ideas are provided faster. We found that valuation depends on the adequacy and originality of ideas and guides response selection and creativity. Finally, our computational model correctly predicts the speed and quality of human creative responses, as well as interindividual differences in creative abilities. Altogether, this model introduces the mechanistic role of valuation in creativity. It paves the way for a neurocomputational account of creativity mechanisms. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Handling Long and Richly Constrained Tasks through Constrained Hierarchical Reinforcement Learning

Safety in goal directed Reinforcement Learning (RL) settings has typically been handled through constraints over trajectories and have demonstrated good performance in primarily short horizon tasks. In this paper, we are specifically interested in the problem of solving temporally extended decision making problems such as robots cleaning different areas in a house while avoiding slippery and unsafe areas (e.g., stairs) and retaining enough charge to move to a charging dock; in the presence of complex safety constraints. Our key contribution is a (safety) Constrained Search with Hierarchical Reinforcement Learning (CoSHRL) mechanism that combines an upper level constrained search agent (which computes a reward maximizing policy from a given start to a far away goal state while satisfying cost constraints) with a low-level goal conditioned RL agent (which estimates cost and reward values to move between nearby states). A major advantage of CoSHRL is that it can handle constraints on the cost value distribution (e.g., on Conditional Value at Risk, CVaR) and can adjust to flexible constraint thresholds without retraining. We perform extensive experiments with different types of safety constraints to demonstrate the utility of our approach over leading approaches in constrained and hierarchical RL.

Deep Reinforcement Learning With Reward Design for Quantum Control

Deep reinforcement learning (DRL) has been recognized as a powerful tool in quantum physics, where DRL's reward design is non-trivial but crucial for quantum control tasks. To address the problem of over-reliance on human empirical knowledge to design DRL's rewards, we propose a deep reinforcement learning approach with a novel reward paradigm designed by the learning process information (DRL-LPI), where the learning process information (LPI) comprises the state information and the experiences. In DRL-LPI, the state information after being classified by a fidelity threshold, and the experiences are first stored simultaneously in the respective sequences, and this process is repeated until a similar-segment ends. Then, the stored state information is converted to the real value and used to design the reward value by applying a self-amplitude function. Next, the designed reward values are integrated with the stored experiences to compose transitions for DRL's training. Through comparisons to five representative reward schemes, the proposed DRL-LPI is evaluated on two typical quantum control tasks, i.e., the spin-(1/2) quantum state control and many-coupled qubits state control, and the experimental results show the superior learning efficiency and control performance of the proposed approach. More results show that DRL-LPI exhibits the ability to learn the control strategy with few control actions compared to stochastic gradient descent (SGD) and genetic algorithm (GA).

Dampened motivation in schizophrenia: evidence from a novel effort-based decision-making task in social scenarios.

Apathy represents a significant manifestation of negative symptoms within individuals diagnosed with schizophrenia (SCZ) and exerts a profound impact on their social relationships. However, the specific implications of this motivational deficit in social scenarios have yet to be fully elucidated. The present study aimed to examine effort-based decision-making in social scenarios and its relation to apathy symptoms in SCZ patients. We initially recruited a group of 50 healthy participants (16 males) to assess the validity of the paradigm. Subsequently, we recruited 45 individuals diagnosed with SCZ (24 males) and 49 demographically-matched healthy controls (HC, 25 males) for the main study. The Mock Job Interview Task was developed to measure effort-based decision-making in social scenarios. The proportion of hard-task choice and a range of subjective ratings were obtained to examine potential between-group differences. SCZ patients were less likely than HC to choose the hard task with strict interviewers, and this group difference was significant when the hard-task reward value was medium and high. More severe apathy symptoms were significantly correlated with an overall reduced likelihood of making a hard-task choice. When dividing the jobs into two categories based on the levels of social engagement needed, SCZ patients were less willing to expend effort to pursue a potential offer for jobs requiring higher social engagement. Our findings indicated impaired effort-based decision-making in SCZ can be generalized from the monetary/nonsocial to a more ecologically social dimension. Our findings affirm the critical role of aberrant effort allocation on negative symptoms, and may facilitate the development of targeted clinical interventions.

Slowing of Movements in Healthy Aging as a Rational Economic Response to an Elevated Effort Landscape.

Why do we move slower as we grow older? The reward circuits of the brain, which tend to invigorate movements, decline with aging, raising the possibility that reduced vigor is due to the diminishing value that our brain assigns to movements. However, as we grow older, it also becomes more effortful to make movements. Is age-related slowing principally a consequence of increased effort costs from the muscles, or reduced valuation of reward by the brain? Here, we first quantified the cost of reaching via metabolic energy expenditure in human participants (male and female), and found that older adults consumed more energy than the young at a given speed. Thus, movements are objectively more costly for older adults. Next, we observed that when reward increased, older adults, like the young, responded by initiating their movements earlier. Yet, unlike the young, they were unwilling to increase their movement speed. Was their reluctance to reach quicker for rewards due to the increased effort costs, or because they ascribed less value to the movement? Motivated by a mathematical model, we next made the young experience a component of aging by making their movements more effortful. Now the young responded to reward by reacting faster but chose not to increase their movement speed. This suggests that slower movements in older adults are partly driven by an adaptive response to an elevated effort landscape. Moving slower may be a rational economic response the brain is making to mitigate the elevated effort costs that accompany aging.

Research on decision making of intelligent vehicle based on composite priority experience replay

To address the problems of underutilization of samples and unstable training for intelligent vehicle training in the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, a TD3 algorithm based on the Composite Prioritized Experience Replay (CPR-TD3) mechanism is proposed. It considers experience immediate reward value and Temporal Difference error (TD-error) based and respectively to construct priorities to rank the samples. Subsequently composite average ranking of the samples to recalculate the priorities for sampling, uses the collected samples to train the target network. Then introduces the minimum lane change distance and the variable headway time distance to improve the reward function. Finally, the improved algorithm is proved to be effective by comparing it with the traditional TD3 on the highway scenario, and the CPR-TD3 algorithm improves the training efficiency of intelligent vehicles.

Multi-surrogate-assisted stochastic fractal search based on scale-free network for high-dimensional expensive optimization

Surrogate-assisted meta-heuristic algorithms (SAMAs) have been increasingly popular in recent years for solving challenging optimization problems. However, the majority of recent studies concentrate on low-dimensional problems. In this paper, a scale-free network based multi-surrogate-assisted stochastic fractal search (SF-MSASFS) algorithm is proposed. Specifically, based on the stochastic fractal search (SFS) algorithm, multiple surrogate models, namely RBF and Kriging models, are used to enhance the robustness of the algorithm. The scale-free network is used to build the topology structure of the SFS algorithm, and the offspring particles are generated by means of the connection relationship between the parent particles. In addition, to further enhance adaptability, an adaptive mechanism is implemented, tailoring three distinct update mechanisms based on their corresponding reward values. Finally, the performance of the proposed algorithm is demonstrated by comparing the proposed algorithm with a number of state-of-the-art SAMAs on several well-known benchmark functions, in particular in solving high-dimensional expensive problems (HEOPs). The results underscore the SF-MSASFS algorithm’s commendable optimization performance. (The MATLAB code can be found at the authors github: https://github.com/xiaodi-Cheng/SF-MSASFS)