Short-term Rewards Research Articles

Trait Anxiety Leads to "Better" Performance? A Study on Acute Stress and Uncertain Decision-Making.

In uncertain situations, individuals seek to maximize rewards while managing risks. Yet, the effects of acute stress and anxiety on decision-making in ambiguous and risky contexts are unclear. This study aims to contribute to the exploration of how acute stress influences sensitivity to immediate vs. delayed rewards, risk management strategies, and the role of anxiety in these processes. This study used the laboratory acute stress induction paradigm to analyze the direction of influence of acute stress on ambiguity decision-making and risky decision-making in males and then used moderating effect analysis to study the impact of anxiety on this process. The results show that a combination of the Socially Evaluated Cold Pressor Test and the Sing-a-Song Stress Test can successfully induce acute stress, which reduces both the proportion of the options selected that represent long-term rewards and risk-adjustment ability. Additionally, trait anxiety had a moderating effect on the influence of stress on ambiguity decision-making. Acute stress reduces focus on long-term rewards while increasing focus on short-term rewards, leading to impulsivity and impaired risk-adjustment. Additionally, to some extent, high trait anxiety scores predict better performance in making decisions under ambiguity during stress.

Dopamine and Endorphins: A Study of Speed in Fashion as a Mechanism of Desire

This study delves into the contrasting mechanisms of fast fashion and slow fashion, examining how they operate as expressions of human desire driven by distinct neurological and philosophical underpinnings. Fast fashion, fueled by rapid production cycles and the influence of technology and capital, leverages dopamine-driven mechanisms to provide instant gratification. This model aligns with the short-term reward system, reinforcing consumer behaviors that lead to continuous purchases and cyclical consumption. However, this pursuit often results in environmental degradation, resource overuse, and psychological exhaustion among consumers. In contrast, slow fashion embodies a counter-narrative that emphasizes quality, sustainability, and long-term satisfaction, aligning with the endorphin-driven mechanisms. These endorphins, associated with deeper emotional fulfillment and resilience through challenges, promote a reflective, meaningful consumer experience. The philosophical analysis incorporates Schopenhauer’s concept of insatiable will, paralleling the endless desire perpetuated by fast fashion, and Nietzsche’s idea of the will to power, suggesting slow fashion as a creative and life-affirming response that embraces challenge and meaning. Ultimately, the article argues that while both fast and slow fashion engage human desire mechanisms, their impacts differ vastly in terms of sustainability, psychological well-being, and existential fulfillment. The study calls for a broader exploration of how technology, culture, and ethical considerations can reshape fashion consumption toward more sustainable and emotionally satisfying practices.

Evolutionary origins of temporal discounting: Modeling how time and uncertainty constrain optimal decision-making strategies across taxa.

The propensity of humans and non-human animals to discount future returns for short-term benefits is well established. This contrasts with the ability of organisms to unfold complex developmental sequences over months or years efficiently. Research has focused on various descriptive and predictive parameters of 'temporal discounting' in behavior, and researchers have proposed models to explain temporal preference in terms of fitness-maximizing outcomes. Still, the underlying ultimate cause of this phenomenon has not been deeply explored across taxa. Here, we propose an ultimate (i.e., evolutionary) causal explanation for the selection of temporal discounting largely conserved across taxa. We propose that preference for a short-term reward (e.g., heightened impulsivity) often is less than optimal and likely is the product of constraints imposed on natural selection with respect to predicting events in a temporal framework in the context of future uncertainty. Using a simple Newtonian model for time across a fitness landscape in which movement by organisms is only possible in one direction, we examine several factors that influence the ability of an organism to choose a distant reward over a more temporally proximate reward: including the temporal distance of the far reward, the relative value of the distant reward, and the effect of uncertainty about the value and presence of the distant reward. Our results indicate that an organism may choose a more distant reward, but only if it is not too far into the future and has a substantially higher-value fitness payoff relative to the short-term reward. Notably, any uncertainty about the distant reward made it extremely unlikely for an organism to choose the delayed reward strategy compared to choosing a closer reward, even if the distant reward had a much higher payoff because events that are uncertain are only partially visible to natural selection pressures. The results help explain why natural selection is constrained to promote more optimal behavioral strategies and why it has difficulty selecting a distant reward over a lower-value short-term reward. The degree of uncertainty is an especially salient ecological variable in promoting and preferencing short-term behavioral strategies across taxa. These results further help illustrate why, from an ultimate causal perspective, human and non-human taxa have difficulty making more optimal long-term decisions.

The effects of threat on complex decision-making: evidence from a virtual environment.

Individuals living and working in dangerous settings (e.g., first responders and military personnel) make complex decisions amidst serious threats. However, controlled studies on decision-making under threat are limited given obvious ethical concerns. Here, we embed a complex decision-making task within a threatening, immersive virtual environment. Based on the Iowa Gambling Task (IGT), a paradigm widely used to study complex decision-making, the task requires participants to make a series of choices to escape a collapsing building. In Study 1 we demonstrate that, as with the traditional IGT, participants learn to make advantageous decisions over time and that their behavioural data can be described by reinforcement-learning based computational models. In Study 2 we created threatening and neutral versions of the environment. In the threat condition, participants performed worse, taking longer to improve from baseline and scoring lower through the final trials. Computational modelling further revealed that participants in the threat condition were more responsive to short term rewards and less likely to perseverate on a given choice. These findings suggest that when threat is integral to decision-making, individuals make more erratic choices and focus on short term gains. They furthermore demonstrate the utility of virtual environments for making threat integral to cognitive tasks.

Short-Term Memory Capacity Predicts Willingness to Expend Cognitive Effort for Reward.

We must often decide whether the effort required for a task is worth the reward. Past rodent work suggests that willingness to deploy cognitive effort can be driven by individual differences in perceived reward value, depression, or chronic stress. However, many factors driving cognitive effort deployment-such as short-term memory ability-cannot easily be captured in rodents. Furthermore, we do not fully understand how individual differences in short-term memory ability, depression, chronic stress, and reward anticipation impact cognitive effort deployment for reward. Here, we examined whether these factors predict cognitive effort deployment for higher reward in an online visual short-term memory task. Undergraduate participants were grouped into high and low effort groups (n HighEffort = 348, n LowEffort = 81; n Female = 332, n Male = 92, M Age = 20.37, Range Age = 16-42) based on decisions in this task. After completing a monetary incentive task to measure reward anticipation, participants completed short-term memory task trials where they could choose to encode either fewer (low effort/reward) or more (high effort/reward) squares before reporting whether or not the color of a target square matched the square previously in that location. We found that only greater short-term memory ability predicted whether participants chose a much higher proportion of high versus low effort trials. Drift diffusion modeling showed that high effort group participants were more biased than low effort group participants toward selecting high effort trials. Our findings highlight the role of individual differences in cognitive effort ability in explaining cognitive effort deployment choices.

Reduced sensitivity to future consequences underlies gambling decision in cerebellar ataxia

IntroductionPrevious research has identified that people with cerebellar ataxia (CA) showed impaired reward-related decision-making in the Iowa Gambling Task (IGT). To investigate the mechanisms underlying this impairment, we examined CA participants' combination of performance in the IGT, which predominantly tests reward seeking, and the modified IGT (mIGT), which mainly assesses punishment avoidance. MethodsFifty participants with CA and one hundred controls completed the IGT and mIGT. Task performance in each of the five twenty-trial blocks was compared between groups and the learning rates were assessed with simple linear regressions. Each participant's IGT score and mIGT score were compared. ResultsCA participants performed worse than controls in both the IGT and the mIGT, especially in the last block (IGT: −0.24 ± 10.05 vs. 3.88 ± 10.31, p = 0.041; mIGT: 2.72 ± 7.62 vs. 8.65 ± 8.64, p < 0.001). In contrast to the controls, those with CA did not significantly improve their scores over time in either task. Controls performed better in the mIGT than the IGT, while CA participants' scores in the two tasks showed no significant difference. IGT and mIGT performance did not correlate with ataxia severity or depressive symptoms. ConclusionIndividuals with CA showed impaired performance in both the IGT and mIGT, which indicates disruption in both short-term reward seeking and short-term punishment avoidance. Therefore, these results suggest that reduced sensitivity to long-term consequences drives the risky decision-making in CA.

Teaching transposon classification as a means to crowd source the curation of repeat annotation – a tardigrade perspective

BackgroundThe advancement of sequencing technologies results in the rapid release of hundreds of new genome assemblies a year providing unprecedented resources for the study of genome evolution. Within this context, the significance of in-depth analyses of repetitive elements, transposable elements (TEs) in particular, is increasingly recognized in understanding genome evolution. Despite the plethora of available bioinformatic tools for identifying and annotating TEs, the phylogenetic distance of the target species from a curated and classified database of repetitive element sequences constrains any automated annotation effort. Moreover, manual curation of raw repeat libraries is deemed essential due to the frequent incompleteness of automatically generated consensus sequences.ResultsHere, we present an example of a crowd-sourcing effort aimed at curating and annotating TE libraries of two non-model species built around a collaborative, peer-reviewed teaching process. Manual curation and classification are time-consuming processes that offer limited short-term academic rewards and are typically confined to a few research groups where methods are taught through hands-on experience. Crowd-sourcing efforts could therefore offer a significant opportunity to bridge the gap between learning the methods of curation effectively and empowering the scientific community with high-quality, reusable repeat libraries.ConclusionsThe collaborative manual curation of TEs from two tardigrade species, for which there were no TE libraries available, resulted in the successful characterization of hundreds of new and diverse TEs in a reasonable time frame. Our crowd-sourcing setting can be used as a teaching reference guide for similar projects: A hidden treasure awaits discovery within non-model organisms.

Reinforcement learning in building controls: A comparative study of algorithms considering model availability and policy representation

Reinforcement Learning (RL) has presented considerable potential as an advanced control technique in building controls to enable buildings operating more energy-efficient. As various types of RL algorithms have been studied on their performance of building controls, benchmarking these algorithms across the entire spectrum of features is essential to provide an overview and deepen the understanding of RL applications. Therefore, this study aims to compare and analyze the effectiveness of various RL algorithms, encompassing the entire RL categories featured by value-based, policy gradient, actor-critic and model-based RL considering model availability and policy representation. To provide a comprehensive analysis, in addition to the control performance quantified by the cumulative rewards based on the cost function of RL, data demand and robustness of hyperparameter tuning were investigated. The open-source Gym-Eplus framework was selected as the virtual environment to train and test different RL agents. The results showed that both model-free and model-based RL agents outperformed the baseline rule-based control in terms of energy consumption and thermal comfort, and RL agents were capable of evaluating both short-term and long-term rewards to achieve adaptive control optimization continuously along with the online control process. Model-based RL agent improved the data sampling efficiency but presented a relatively sacrificed control performance during the tested summer days.

Individual quality, insecure organizational attachment, and formalistic task completion: Social cognitive perspective.

Formalistic tasks are widely utilized in modern companies due to their ability to increase productivity and contribute to the achievement of corporate goals at a lower cost. However, these tasks are often meet with resistance from individuals because they do not provide direct short-term rewards for their efforts. Drawing on social cognitive theory, this study examined the influence of individual quality and organizational attachment on the completion of formalistic tasks. To address this, the study conducted a questionnaire survey to collect data from 602 Chinese respondents and built a structural equation model for data analysis. Through empirical research, the study confirmed the positive role of individual quality, including knowledge and personality, in the completion of formalistic tasks. Furthermore, the study proved that avoidant attachment could significantly weaken the effect of some components of individual quality on formalistic task completion. This paper is the first to reveal the influence of individual and environmental factors on individuals' completion of formalistic tasks, progressing from bottom to top. The implications of these results are discussed.

Modeling andcontrolling the body inmaladaptive ways: an active inference perspective onnon-suicidal self-injury behaviors.

A significant number of persons engage in paradoxical behaviors, such as extreme food restriction (up to starvation) and non-suicidal self-injuries, especially during periods of rapid changes, such as adolescence. Here, we contextualize these and related paradoxical behavior within an active inference view of brain functions, which assumes that the brain forms predictive models of bodily variables, emotional experiences, and the embodied self and continuously strives to reduce the uncertainty of such models. We propose that not only in conditions of excessive or prolonged uncertainty, such as in clinical conditions, but also during pivotal periods of developmental transition, paradoxical behaviors might emerge as maladaptive strategies to reduce uncertainty-by "acting on the body"- soliciting salient perceptual and interoceptive sensations, such as pain or excessive levels of hunger. Although such strategies are maladaptive and run against our basic homeostatic imperatives, they might be functional not only to provide some short-term reward (e.g. relief from emotional distress)-as previously proposed-but also to reduce uncertainty and possibly to restore a coherent model of one's bodily experience and the self, affording greater confidence in who we are and what course of actions we should pursue.

When Smaller Sooner Depletes a Pool of Resources Faster.

Behavior has short-term (proximal) and long-term (distal) consequences, and these consequences often involve different commodities. In particular, a commonly encountered distal consequence involves running out of resources - energy to respond, available food, ammunition, or money in the bank - that must be replenished before continuing a rewarding task. The current project examines proximal behavioral consequences in a video game (the amount of damage done to a clicked-on target as a function of waiting) and distal behavioral consequences (running out of the resources that allow the player to click on a target). When depleted, the resource replenished after a fixed amount of time. Thus, participants sometimes faced a tradeoff between behaviors that maximized their short-term reward rate and those that maximized their long-term reward rate. When the proximal contingency did not affect the short-term reward rate, the mere presence of limitations resulted in the slower use of resources, but the slowdown did not evidence strong sensitivity to the size of the resource pool nor the delay to its replenishment (Experiment 1). However, when the proximal contingency rewarded faster use of resources, participants did show sensitivity to the duration of the replenishment delay and the size of the resource pool (Experiment 2).

GANTEE: Generative Adversarial Network for Taxonomy Enterance Evaluation

Taxonomy is formulated as directed acyclic graphs or trees of concepts that support many downstream tasks. Many new coming concepts need to be added to an existing taxonomy. The traditional taxonomy expansion task aims only at finding the best position for new coming concepts in the existing taxonomy. However, they have two drawbacks when being applied to the real-scenarios. The previous methods suffer from low-efficiency since they waste much time when most of the new coming concepts are indeed noisy concepts. They also suffer from low-effectiveness since they collect training samples only from the existing taxonomy, which limits the ability of the model to mine more hypernym-hyponym relationships among real concepts. This paper proposes a pluggable framework called Generative Adversarial Network for Taxonomy Entering Evaluation (GANTEE) to alleviate these drawbacks. A generative adversarial network is designed in this framework by discriminative models to alleviate the first drawback and the generative model to alleviate the second drawback. Two discriminators are used in GANTEE to provide long-term and short-term rewards, respectively. Moreover, to further improve the efficiency, pre-trained language models are used to retrieve the representation of the concepts quickly. The experiments on three real-world large-scale datasets with two different languages show that GANTEE improves the performance of the existing taxonomy expansion methods in both effectiveness and efficiency.

The influence of temporal focus on individual intertemporal decision-making in life history strategy framework

Temporal factors have an important role in intertemporal decision-making. Temporal focus is the extent to which individuals characteristically direct their attention to the past, present, and/or future. Temporal focus can influence resource allocation. It was not known, however, whether temporal focus might affect intertemporal decision-making. The purpose of this study is to investigate how life history strategies and temporal focus together influence intertemporal decision-making. Study 1 used a questionnaire to preliminarily explore the relationship between temporal focus and intertemporal decision-making. Results showed that temporal focus was positively associated with individuals' intertemporal decision preferences, and the more individuals preferred future focus, the higher their rate of temporal discounting. Study 2 examined the joint effect of the temporal focus and life history strategy on intertemporal decision-making by manipulating the temporal focus of individuals with different life history strategies. The manipulation of temporal focus influenced intertemporal decision preferences of fast strategy individuals, that is, the short-term small reward in the future focus and the long-term large reward in the past focus, but has no effect on the intertemporal decision preference of slow strategy individuals. The entire study confirms life history theory and validates the role of perceptual time-based models in intertemporal decision-making.

Dopamine D2 receptors in nucleus accumbens cholinergic interneurons increase impulsive choice.

Impulsive choice, often characterized by excessive preference for small, short-term rewards over larger, long-term rewards, is a prominent feature of substance use and other neuropsychiatric disorders. The neural mechanisms underlying impulsive choice are not well understood, but growing evidence implicates nucleus accumbens (NAc) dopamine and its actions on dopamine D2 receptors (D2Rs). Because several NAc cell types and afferents express D2Rs, it has been difficult to determine the specific neural mechanisms linking NAc D2Rs to impulsive choice. Of these cell types, cholinergic interneurons (CINs) of the NAc, which express D2Rs, have emerged as key regulators of striatal output and local dopamine release. Despite these relevant functions, whether D2Rs expressed specifically in these neurons contribute to impulsive choice behavior is unknown. Here, we show that D2R upregulation in CINs of the mouse NAc increases impulsive choice as measured in a delay discounting task without affecting reward magnitude sensitivity or interval timing. Conversely, mice lacking D2Rs in CINs showed decreased delay discounting. Furthermore, CIN D2R manipulations did not affect probabilistic discounting, which measures a different form of impulsive choice. Together, these findings suggest that CIN D2Rs regulate impulsive decision-making involving delay costs, providing new insight into the mechanisms by which NAc dopamine influences impulsive behavior.

Associations between eating in the absence of hunger and executive functions in adolescents with binge-eating disorder: An experimental study

Eating in the absence of hunger (EAH) is one of the key behavioral features of binge-eating disorder (BED) in youth. Although preliminary evidence revealed that adolescent BED co-occurs with deficits in executive functions (EFs), it is unclear whether EFs are related to EAH. Thus, this study experimentally examined whether deficits in EFs predict EAH in adolescents with and without BED. Adolescents (12–20 years) with BED (n = 28) and age-, sex-, and weight-matched controls (n = 28) underwent an EAH paradigm in the laboratory, where they were offered snacks ad libitum after having established satiety during a lunch meal. Cognitive interference, cognitive flexibility, decision making, and EFs in daily life were assessed by neuropsychological tests and self- and parent-report. The BED group showed a significantly higher food intake in gram during the EAH trial than controls with medium effect, but no significant group differences in EFs emerged. Dysfunctional decision making in terms of risky decision making, but no other EFs, predicted increased EAH (g, kcal) in the total sample. Although increases in risky decision making over adolescence are well known, this study uniquely revealed that general decision-making abilities driven by short-term reward may account for disinhibited eating behavior. Interventions targeting decision making with focus on reward sensitivity should be evaluated for their efficacy in preventing and reducing disinhibited eating behavior in adolescents.

Adversarial Decision-Making for Moving Target Defense: A Multi-Agent Markov Game and Reinforcement Learning Approach

Reinforcement learning has shown a great ability and has defeated human beings in the field of real-time strategy games. In recent years, reinforcement learning has been used in cyberspace to carry out automated and intelligent attacks. Traditional defense methods are not enough to deal with this problem, so it is necessary to design defense agents to counter intelligent attacks. The interaction between the attack agent and the defense agent can be modeled as a multi-agent Markov game. In this paper, an adversarial decision-making approach that combines the Bayesian Strong Stackelberg and the WoLF algorithms was proposed to obtain the equilibrium point of multi-agent Markov games. With this method, the defense agent can obtain the adversarial decision-making strategy as well as continuously adjust the strategy in cyberspace. As verified in experiments, the defense agent should attach importance to short-term rewards in the process of a real-time game between the attack agent and the defense agent. The proposed approach can obtain the largest rewards for defense agent compared with the classic Nash-Q and URS-Q algorithms. In addition, the proposed approach adjusts the action selection probability dynamically, so that the decision entropy of optimal action gradually decreases.

Disrupting future discounting: a commentary on an underutilised psychological approach for improving adherence to diet and physical activity interventions.

Non-communicable diseases (NCD) such as CVD and type 2 diabetes mellitus are major contributors to the burden of disease. NCD are largely driven by modifiable lifestyle factors including poor diet and insufficient physical activity, and consequently, prevention is a public health priority. Although diet and physical activity levels can be improved via lifestyle interventions, long-term adherence to such interventions remains low, which limits their effectiveness. Thus, it is critical to identify the underlying mechanisms that challenge uptake and adherence to such interventions. The current commentary discusses an important, but underexplored, psychological driver of poor adherence to lifestyle interventions, namely, future discounting, which describes the tendency to prefer smaller, short-term rewards over larger, long-term rewards. For example, in the nutrition domain, future discounting refers to valuing the immediate reward of excessive intake of energy-dense, nutrient-poor, discretionary foods high in salt, sugar, and saturated fat, and insufficient intake of low-energy, nutrient-dense, whole foods such as vegetables. Prominent theoretical models propose that excessive future discounting is a major contributor to the development of unhealthy lifestyle behaviours. Furthermore, a vast body of evidence suggests that future discounting plays a key role in risk of NCD. Thus, the evidence to date supports the idea that future discounting is an important multi-behaviour target for supporting lifestyle behaviour change; however, this approach has been largely neglected in preventive health efforts. Furthermore, this commentary discusses promising techniques (e.g. Episodic Future Thinking) for disrupting future discounting to promote improved adherence to lifestyle interventions aimed at reducing NCD risk.

Metacognitive deficits are associated with lower sensitivity to preference reversals in nicotine dependence

Deficits in impulse control belong to the core profile of nicotine dependence. Smokers might thus benefit from voluntarily self-restricting their access to the immediate temptation of nicotine products (precommitment) in order to avoid impulse control failures. However, little is known about how smokers’ willingness to engage in voluntary self-restrictions is determined by metacognitive insight into their general preferences for immediate over delayed rewards. Here, with a series of monetary intertemporal choice tasks, we provide empirical evidence for reduced metacognitive accuracy in smokers relative to non-smokers and show that smokers overestimate the subjective value of delayed rewards relative to their revealed preferences. In line with the metacognitive deficits, smokers were also less sensitive to the risk of preference reversals when deciding whether or not to restrict their access to short-term financial rewards. Taken together, the current findings suggest that deficits not only in impulse control but also in metacognition may hamper smokers’ resistance to immediate rewards and capacity to pursue long-term goals.

Self-Supervised Reinforcement Learning with dual-reward for knowledge-aware recommendation

To improve the recommendation accuracy and offer explanations for recommendations, Reinforcement Learning (RL) has been applied to path reasoning over knowledge graphs. However, in recommendation tasks, most existing RL methods learn the path-finding policy using only a short-term or single reward, leading to a local optimum and losing some potential paths. To address these issues, we propose a Self-Supervised Reinforcement Learning (SSRL) framework combined with dual-reward for knowledge-aware recommendation reasoning over knowledge graphs. Then, we improve Actor–Critic algorithm by using a dual-reward driven strategy, which combines short-term reward with long-term incremental evaluation. The improved algorithm helps the policy guide path reasoning in an overall situation. In addition, to find the most potential paths, in the improved Actor–Critic algorithm, a loss constraint of each sample is used as a reinforced signal to update the gradients. With some improvements against baselines, experimental results demonstrate the effectiveness of our framework.

Deconfounding Actor-Critic Network with Policy Adaptation for Dynamic Treatment Regimes.

Despite intense efforts in basic and clinical research, an individualized ventilation strategy for critically ill patients remains a major challenge. Recently, dynamic treatment regime (DTR) with reinforcement learning (RL) on electronic health records (EHR) has attracted interest from both the healthcare industry and machine learning research community. However, most learned DTR policies might be biased due to the existence of confounders. Although some treatment actions non-survivors received may be helpful, if confounders cause the mortality, the training of RL models guided by long-term outcomes (e.g., 90-day mortality) would punish those treatment actions causing the learned DTR policies to be suboptimal. In this study, we develop a new deconfounding actor-critic network (DAC) to learn optimal DTR policies for patients. To alleviate confounding issues, we incorporate a patient resampling module and a confounding balance module into our actor-critic framework. To avoid punishing the effective treatment actions non-survivors received, we design a short-term reward to capture patients' immediate health state changes. Combining short-term with long-term rewards could further improve the model performance. Moreover, we introduce a policy adaptation method to successfully transfer the learned model to new-source small-scale datasets. The experimental results on one semi-synthetic and two different real-world datasets show the proposed model outperforms the state-of-the-art models. The proposed model provides individualized treatment decisions for mechanical ventilation that could improve patient outcomes.