Reward Value Research Articles

Training-induced circuit-specific excitatory synaptogenesis in mice is required for effort control

Synaptogenesis is essential for circuit development; however, it is unknown whether it is critical for the establishment and performance of goal-directed voluntary behaviors. Here, we show that operant conditioning via lever-press for food reward training in mice induces excitatory synapse formation onto a subset of anterior cingulate cortex neurons projecting to the dorsomedial striatum (ACC→DMS). Training-induced synaptogenesis is controlled by the Gabapentin/Thrombospondin receptor α2δ−1, which is an essential neuronal protein for proper intracortical excitatory synaptogenesis. Using germline and conditional knockout mice, we found that deletion of α2δ−1 in the adult ACC→DMS circuit diminishes training-induced excitatory synaptogenesis. Surprisingly, this manipulation does not impact learning but results in a significant increase in effort exertion without affecting sensitivity to reward value or changing contingencies. Bidirectional optogenetic manipulation of ACC→DMS neurons rescues or phenocopies the behaviors of the α2δ−1 cKO mice, highlighting the importance of synaptogenesis within this cortico-striatal circuit in regulating effort exertion.

Pengaruh Implementasi Kegiatan keagamaan (Pembinaan, Pembiasaan, Pendidikan), Terhadap Peningkatan Semangat Ibadah Siswa

This article discusses and analyzes the Effects of the Implementation of Religious Activities (Coaching, Habituation, Education), on Increasing Student Worship Enthusiasm, as additional insight for teachers and prospective teachers so they can understand what affects Increasing Student Worship Enthusiasm. Writing this Literature Review article uses library research methods sourced from online media such as Google Scholar, Mendeley, and other academic online media. The results of this article are: 1) Guidance has an effect on Increasing the Spirit of Worship of Students; 2) Habits Affect the Improvement of Students' Worship Enthusiasm; and 3) Education has an effect on increasing the spirit of student worship. In addition to these 3 exogenous variables that affect endogenous variables in increasing the spirit of worship, there are many other factors including training, reward values, and the school environment.

Adaptor protein complex 2 in the orbitofrontal cortex predicts alcohol use disorder.

Alcohol use disorder (AUD) is a life-threatening disease characterized by compulsive drinking, cognitive deficits, and social impairment that continue despite negative consequences. The inability of individuals with AUD to regulate drinking may involve functional deficits in cortical areas that normally balance actions that have aspects of both reward and risk. Among these, the orbitofrontal cortex (OFC) is critically involved in goal-directed behavior and is thought to maintain a representation of reward value that guides decision making. In the present study, we analyzed post-mortem OFC brain samples collected from age- and sex-matched control subjects and those with AUD using proteomics, bioinformatics, machine learning, and reverse genetics approaches. Of the 4,500+ total unique proteins identified in the proteomics screen, there were 47 proteins that differed significantly by sex that were enriched in processes regulating extracellular matrix and axonal structure. Gene ontology enrichment analysis revealed that proteins differentially expressed in AUD cases were involved in synaptic and mitochondrial function, as well as transmembrane transporter activity. Alcohol-sensitive OFC proteins also mapped to abnormal social behaviors and social interactions. Machine learning analysis of the post-mortem OFC proteome revealed dysregulation of presynaptic (e.g., AP2A1) and mitochondrial proteins that predicted the occurrence and severity of AUD. Using a reverse genetics approach to validate a target protein, we found that prefrontal Ap2a1 expression significantly correlated with voluntary alcohol drinking in male and female genetically diverse mouse strains. Moreover, recombinant inbred strains that inherited the C57BL/6J allele at the Ap2a1 interval consumed higher amounts of alcohol than those that inherited the DBA/2J allele. Together, these findings highlight the impact of excessive alcohol consumption on the human OFC proteome and identify important cross-species cortical mechanisms and proteins that control drinking in individuals with AUD.

Reliable population code for subjective economic value from heterogeneous neuronal signals in primate orbitofrontal cortex

Behavior-related neuronal signals often vary between neurons, which might reflect the unreliability of individual neurons or a truly heterogeneous code. This notion may also apply to economic ("value-based") choices and the underlying reward signals. Reward value is subjective and can be described by a nonlinearly weighted magnitude (utility) and probability. Defining subjective values relies on the continuity axiom, whose testing involves structured variations of a wide range of reward magnitudes and probabilities. Axiom compliance demonstrates understanding of the stimuli and the meaningful character of choices. Using these tests, we investigated the encoding of subjective economic value by neurons in a key economic-decision structure of the monkey brain, the orbitofrontal cortex (OFC). We found that individual neurons carry heterogeneous neuronal value signals that largely fail to match the animal's choices. However, neuronal population signals matched the animal's choices well, suggesting accurate subjective economic value encoding by a heterogeneous population of unreliable neurons.

Parallel processing of value-related information during multi-attribute decisions.

When choosing between options with multiple attributes, do we integrate all attributes into a unified measure for comparison, or does the comparison also occur at the level of each attribute, involving parallel processes that can dynamically influence each other? What happens when independent sensory features all carry information about the same decision factor, such as reward value? To investigate these questions, we asked human participants to perform a two-alternative forced choice reaching task in which the reward value of a target was indicated by two visual attributes-its brightness ("bottom-up," BU feature) and its orientation ("top-down," TD feature). If decisions always occur after the integration of both features, there should be no difference in the reaction time (RT) regardless of the attribute combinations that drove the choice. Counter to that prediction, RT distributions depended on the attribute combinations of given targets and the choices made by participants. RTs were shortest when both attributes were congruent or when the choice was based on the bottom-up feature, and longer when the attributes were in conflict (favoring opposite options). In conflict trials, nearly two-thirds of participants made faster decisions when choosing the option favored by the bottom-up feature than when choosing the top-down-favored option. We also observed mid-reach changes-of-mind in a subset of conflict trials, mostly changing from the bottom-up to the top-down-favored target. These data suggest that multi-attribute value-based decisions are better explained by a distributed process including competition among different features than by a competition based on a single, integrated estimate of value.NEW & NOTEWORTHY We show that during value-based decisions, humans do not always use all reward-related information to make their choice, but instead can "jump the gun" using partial information. In particular, when different sources of information were in conflict, early decisions were mostly based on fast bottom-up information, and sometimes followed by corrective changes-of-mind based on slower top-down information. This supports parallel decision processes among different information sources, as opposed to a single integrated "common currency."

The role of sex on sign-tracking acquisition and outcome devaluation sensitivity in Long Evans rats

Cues that predict rewards can trigger reward-seeking behaviors but also can, in some cases, become targets of motivation themselves. One behavioral phenomenon that captures this idea is sign-tracking in which animals, including humans, interact with reward-predictive cues even though it is not necessary to do so. Sign-tracking in rats has been studied in the domain of motivation and in how motivated behaviors can or cannot become excessive and habit-like over time. Many prior studies look at sign-tracking examine this behavior in male subjects, but there are few papers that look at this behavior in female subjects. Moreover, it is unknown where there might be sex-related variation in how flexible sign-tracking is when faced with changing reward values. Therefore, we asked if there were sex differences in the acquisition of sign-tracking behavior and if there were any sex differences in how sensitive animals were in their sign-tracking following reward devaluation. In contrast to previous reports, we found that males and females show no differences in how they acquire sign-tracking and in ultimate sign-tracking levels following training. Additionally, we found no difference in how quickly males and females learned to devalue the food reward, and we found no differences in sign-tracking levels by sex following outcome devaluation. We believe that this is primarily due to our experiment being performed in the Long Evans strain but also believe that there are many other factors contributing to differences between our study and previous work.

Reliability of gamified reinforcement learning in densely sampled longitudinal assessments.

Reinforcement learning is a core facet of motivation and alterations have been associated with various mental disorders. To build better models of individual learning, repeated measurement of value-based decision-making is crucial. However, the focus on lab-based assessment of reward learning has limited the number of measurements and the test-retest reliability of many decision-related parameters is therefore unknown. In this paper, we present an open-source cross-platform application Influenca that provides a novel reward learning task complemented by ecological momentary assessment (EMA) of current mental and physiological states for repeated assessment over weeks. In this task, players have to identify the most effective medication by integrating reward values with changing probabilities to win (according to random Gaussian walks). Participants can complete up to 31 runs with 150 trials each. To encourage replay, in-game screens provide feedback on the progress. Using an initial validation sample of 384 players (9729 runs), we found that reinforcement learning parameters such as the learning rate and reward sensitivity show poor to fair intra-class correlations (ICC: 0.22-0.53), indicating substantial within- and between-subject variance. Notably, items assessing the psychological state showed comparable ICCs as reinforcement learning parameters. To conclude, our innovative and openly customizable app framework provides a gamified task that optimizes repeated assessments of reward learning to better quantify intra- and inter-individual differences in value-based decision-making over time.

Representation of rewards differing in their hedonic valence in the caudate nucleus correlates with the performance in a problem-solving task in dogs (Canis familiaris)

We have investigated dogs’ (Canis familiaris) abilities in associating different sounds with appetitive stimuli of different incentive values. The association’s establishment was first tested on family dogs (n = 20) in a problem-solving behavioural paradigm (experiment 1), then in a problem-solving behavioural paradigm as well as an fMRI study on specially trained family dogs (n = 20) (experiment 2). The aim was to show behavioural and parallel neural effects of the association formed between the two sounds and two different associated appetitive stimuli. The latency of solving the problem was considered an indicator of the motivational state. In our first experiment, where only behaviour was studied, we found that dogs were quicker in solving a problem upon hearing the sound associated with food higher in reward value, suggesting that they have successfully associated the sounds with the corresponding food value. In our second experiment, this behaviour difference was not significant. In the fMRI study, the cerebral response to the two sounds was compared both before and after the associative training. Two bilateral regions of interest were explored: the caudate nucleus and the amygdala. After the associative training, the response in the caudate nucleus was higher to the sound related to a higher reward value food than to the sound related to a lower reward value food, which difference was not present before the associative training. We found an increase in the amygdala response to both sounds after the training. In a whole-brain representational similarity analysis, we found that cerebral patterns in the caudate nucleus to the two sounds were different only after the training. Moreover, we found a positive correlation between the dissimilarity index in the caudate nucleus for activation responses to the two sounds and the difference in latencies (i.e. high reward value associated sound condition latency—low reward value associated sound condition latency) to solve the behavioural task: the bigger the difference between the conditions in latency to solve the task, the greater the difference in the neural representation of the two sounds was. In summary, family dogs’ brain activation patterns reflected their expectations based on what they learned about the relationship between two sounds and their associated appetitive stimuli.

Cue-induced reinstatement of seeking behavior in male rats is independent from the rewarding value of the primary reinforcer: Effect of mGluR5 blockade

Environmental conditioning factors have a profound impact on alcohol-seeking behavior and the maintenance of alcohol use in individuals with alcohol dependence. Cues associated with alcohol, depending on the perceived value of the primary reinforcer, gain salience and can trigger relapse.This study investigates the correlation between the reward magnitude of the primary reinforcer and the reinstatement evoked by cues predictive of their availability in male rats. Rat self-administration procedures were used to test reinstatement, with reinforcers consisting of 10% alcohol, 10% sucrose, or 2% sodium chloride (NaCl) experienced under need-state conditions. The effect of MTEP ([(2-methyl-1,3-thiazol-4-yl) ethynyl] pyridine), a selective metabotropic glutamate receptor 5 (mGluR5) antagonist, on motivation and reinstatement behaviors was also evaluated. Resultsdemonstrate that under Fixed Ratio 1 (FR1) schedule, the three reinforcers maintain operant responding with the following order of magnitude 10% sucrose >2% NaCl >10% alcohol > water. Under a progressive ratio (PR) schedule of reinforcement, rats exhibit a significantly higher breakpoint for 2% NaCl (under Na-depletion), followed by 10% sucrose and 10% alcohol. After extinction, a significant reinstatement is observed with the magnitude order of 10% sucrose >10% alcohol >2% NaCl. However, only re-exposure to alcohol-paired cues induced significant reinstatement of alcohol-seeking after 4 and 8 months. Treatment with MTEP significantly reduces reinstatement of responding across all reinforcers, with the strongest effect observed on alcohol-seeking. These findings suggest that mGluR5 plays a general role in controlling cue-reactivity, but the effect is prominent in the case of alcohol compared to natural rewards.In conclusion, the results demonstrate a remarkable dissociation between the rewarding magnitude of the primary reinforcer and its ability to trigger relapse upon presentation of a cue previously associated with it. Importantly, alcohol, despite having lower intrinsic motivational value compared to a natural reward (sucrose) or a consummatory stimulus experienced under need state conditions (NaCl), can elicit more robust and longer-term reinstatement of seeking responses. Finally, our data demonstrate a significant involvement of the mGluR5 system in the regulation of seeking behavior.

Evolving the MCTS Upper Confidence Bounds for Trees Using a Semantic-Inspired Evolutionary Algorithm in the Game of Carcassonne

Monte Carlo Tree Search (MCTS) is a sampling best-first method to search for optimal decisions. The success of MCTS depends heavily on how the tree is built and the selection process plays a fundamental role in this. One particular selection mechanism that has proved to be reliable is based on the Upper Confidence Bounds for Trees (UCT). The UCT attempts to balance exploration and exploitation by considering the values stored in the statistical tree of the MCTS. However, some tuning of the MCTS UCT is necessary for this to work well. In this work, we use Evolutionary Algorithms (EAs) to evolve mathematical expressions with the goal to substitute the UCT formula and use the evolved expressions in MCTS. More specifically, we evolve expressions by means of our proposed Semantic-inspired Evolutionary Algorithm in MCTS approach (SIEA-MCTS). This is inspired by semantics in Genetic Programming (GP), where the use of fitness cases is seen as a requirement to be adopted in GP. Fitness cases are normally used to determine the fitness of individuals and can be used to compute the semantic similarity (or dissimilarity) of individuals. However, fitness cases are not available in MCTS. We extend this notion by using multiple reward values from MCTS that allow us to determine both the fitness of an individual and its semantics. By doing so, we show how SIEA-MCTS is able to successfully evolve mathematical expressions that yield better or competitive results compared to UCT without the need of tuning these evolved expressions. We compare the performance of the proposed SIEA-MCTS against MCTS algorithms, MCTS Rapid Action Value Estimation algorithms, three variants of the *-minimax family of algorithms, a random controller and two more EA approaches. We consistently show how SIEA-MCTS outperforms most of these intelligent controllers in the challenging game of Carcassonne, whose state-space complexity is, approx., 10 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{40}$</tex-math></inline-formula> .

Exposure effects to unfamiliar food advertisements on YouTube: A randomized controlled trial among adolescents

Despite the societal relevance of the topic, experimental evidence on the effects of digital food marketing on adolescents' food attitudes and choice behavior remains relatively scarce. Drawing on the logic model explaining the influence of marketing on health-related responses, the current between-subjects experiment sought to (i) evaluate the effects of exposure to unfamiliar food advertisements on YouTube with respect to adolescents' advertisement and brand recall, brand recognition, attitudes, and food choice in a hypothetical scenario; and (ii) compare the effect of exposure to advertisements promoting healthy (vs. unhealthy) food. Uruguayan adolescents (N = 433; 51% female) were randomly assigned to one of four experimental groups: a control group not exposed to any advertisement, a non-food group exposed to a clothing advertisement, an unhealthy food group exposed to an advertisement promoting a burger, and a healthy food group exposed to an advertisement promoting a salad. Following exposure to the advertisement, participants provided responses linked to advertisement and brand recall, brand recognition, attitudes, and food choice. The findings revealed that the largest effects of exposure to food advertisements emerged for advertisement recall and brand recognition (i.e., the most proximal steps in the logic model). Further, the burger advertisement was more easily recalled than the salad and the non-food advertisements, presumably due to its higher reward value. The effects of the food advertisements on brand attitudes and food choices were more inconclusive, although participants’ sex and frequency of burger consumption consistently moderated the impact of ad exposure on these outcomes, through mainly influencing males and adolescents who consumed burgers more frequently. Together, these results contribute to the literature on the effects of digital marketing by providing causal evidence for the role of advertisement exposure on adolescents’ advertisement and brand recall, brand recognition, brand attitudes, and choice behavior.

Reinforcement learning for traffic signal control: Incorporating a virtual mesoscopic model for depicting oversaturated traffic conditions

Recently, with increasing urban traffic congestion, there has been an upsurge in studies on reinforcement learning for traffic signal control (RL-TSC), which enables efficient traffic management. However, most existing RL-TSC research relies on simulation, and instances of real-world application are relatively scarce. Furthermore, existing RL-TSC methods employ a step-based approach, controlling traffic signals every short step. This approach can be inefficient in oversaturated traffic conditions, where large deviations in traffic demand between movements prevent traffic signals from being effectively managed based on the overall traffic situation. In this study, we aim to transform simulation-based RL-TSC into a more practical and applicable model. We have developed a RL-TSC method capable of responding to oversaturated traffic conditions, designing an action space that enables the agent to derive an optimal signal set for every cycle length, thereby understanding the traffic situation across all movements. During each cycle length, our proposed model conducts traffic signal optimization and identifies the optimal signal through an iterative exploration. To facilitate a fast and accurate strategy search process, we developed a kinematic wave-based mesoscopic model. This model estimates the density across the entire link, based on collected traffic data and derives the state and reward values. We have validated the field applicability of the proposed RL-TSC method through a real-world demonstration at a congested intersection in Seoul, Korea. The results indicated a significant improvement in traffic congestion, with the average queue length at the intersection reduced by up to 11.4%.

A Novel, Expert-Endorsed, Neurocognitive Digital Assessment Tool for Addictive Disorders: Development and Validation Study.

Many people with harmful addictive behaviors may not meet formal diagnostic thresholds for a disorder. A dimensional approach, by contrast, including clinical and community samples, is potentially key to early detection, prevention, and intervention. Importantly, while neurocognitive dysfunction underpins addictive behaviors, established assessment tools for neurocognitive assessment are lengthy and unengaging, difficult to administer at scale, and not suited to clinical or community needs. The BrainPark Assessment of Cognition (BrainPAC) Project sought to develop and validate an engaging and user-friendly digital assessment tool purpose-built to comprehensively assess the main consensus-driven constructs underpinning addictive behaviors. The purpose of this study was to psychometrically validate a gamified battery of consensus-based neurocognitive tasks against standard laboratory paradigms, ascertain test-retest reliability, and determine their sensitivity to addictive behaviors (eg, alcohol use) and other risk factors (eg, trait impulsivity). Gold standard laboratory paradigms were selected to measure key neurocognitive constructs (Balloon Analogue Risk Task [BART], Stop Signal Task [SST], Delay Discounting Task [DDT], Value-Modulated Attentional Capture [VMAC] Task, and Sequential Decision-Making Task [SDT]), as endorsed by an international panel of addiction experts; namely, response selection and inhibition, reward valuation, action selection, reward learning, expectancy and reward prediction error, habit, and compulsivity. Working with game developers, BrainPAC tasks were developed and validated in 3 successive cohorts (total N=600) and a separate test-retest cohort (N=50) via Mechanical Turk using a cross-sectional design. BrainPAC tasks were significantly correlated with the original laboratory paradigms on most metrics (r=0.18-0.63, P<.05). With the exception of the DDT k function and VMAC total points, all other task metrics across the 5 tasks did not differ between the gamified and nongamified versions (P>.05). Out of 5 tasks, 4 demonstrated adequate to excellent test-retest reliability (intraclass correlation coefficient 0.72-0.91, P<.001; except SDT). Gamified metrics were significantly associated with addictive behaviors on behavioral inventories, though largely independent of trait-based scales known to predict addiction risk. A purpose-built battery of digitally gamified tasks is sufficiently valid for the scalable assessment of key neurocognitive processes underpinning addictive behaviors. This validation provides evidence that a novel approach, purported to enhance task engagement, in the assessment of addiction-related neurocognition is feasible and empirically defensible. These findings have significant implications for risk detection and the successful deployment of next-generation assessment tools for substance use or misuse and other mental disorders characterized by neurocognitive anomalies related to motivation and self-regulation. Future development and validation of the BrainPAC tool should consider further enhancing convergence with established measures as well as collecting population-representative data to use clinically as normative comparisons.

Worth the Work? Monkeys Discount Rewards by a Subjective Adapting Effort Cost.

All life must solve how to allocate limited energy resources to maximize benefits from scarce opportunities. Economic theory posits decision makers optimize choice by maximizing the subjective benefit (utility) of reward minus the subjective cost (disutility) of the required effort. While successful in many settings, this model does not fully account for how experience can alter reward-effort trade-offs. Here, we test how well the subtractive model of effort disutility explains the behavior of two male nonhuman primates (Macaca mulatta) in a binary choice task in which reward quantity and physical effort to obtain were varied. Applying random utility modeling to independently estimate reward utility and effort disutility, we show the subtractive effort model better explains out-of-sample choice behavior when compared with parabolic and exponential effort discounting. Furthermore, we demonstrate that effort disutility depends on previous experience of effort: in analogy to work from behavioral labor economics, we develop a model of reference-dependent effort disutility to explain the increased willingness to expend effort following previous experience of effortful options in a session. The result of this analysis suggests that monkeys discount reward by an effort cost that is measured relative to an expected effort learned from previous trials. When this subjective cost of effort, a function of context and experience, is accounted for, trial-by-trial choices can be explained by the subtractive cost model of effort. Therefore, in searching for net utility signals that may underpin effort-based decision-making in the brain, careful measurement of subjective effort costs is an essential first step.SIGNIFICANCE STATEMENT All decision-makers need to consider how much effort they need to expend when evaluating potential options. Economic theories suggest that the optimal way to choose is by cost-benefit analysis of reward against effort. To be able to do this efficiently over many decision contexts, this needs to be done flexibly, with appropriate adaptation to context and experience. Therefore, in aiming to understand how this might be achieved in the brain, it is important to first carefully measure the subjective cost of effort. Here, we show monkeys make reward-effort cost-benefit decisions, subtracting the subjective cost of effort from the subjective value of rewards. Moreover, the subjective cost of effort is dependent on the monkeys' experience of effort in previous trials.

Measuring utility with diffusion models.

The drift diffusion model (DDM) is a prominent account of how people make decisions. Many of these decisions involve comparing two alternatives based on differences of perceived stimulus magnitudes, such as economic values. Here, we propose a consistent estimator for the parameters of a DDM in such cases. This estimator allows us to derive decision thresholds, drift rates, and subjective percepts (i.e., utilities in economic choice) directly from the experimental data. This eliminates the need to measure these values separately or to assume specific functional forms for them. Our method also allows one to predict drift rates for comparisons that did not occur in the dataset. We apply the method to two datasets, one comparing probabilities of earning a fixed reward and one comparing objects of variable reward value. Our analysis indicates that both datasets conform well to the DDM. We find that utilities are linear in probability and slightly convex in reward.

An Improved CSMA/CA Protocol Anti-Jamming Method Based on Reinforcement Learning

The CSMA/CA algorithm uses the binary backoff mechanism to solve the multi-user channel access problem, but this mechanism is vulnerable to jamming attacks. Existing research uses channel-hopping to avoid jamming, but this method fails when the channel is limited or hard to hop. To address this problem, we first propose a Markov decision process (MDP) model with contention window (CW) as the state, throughput as the reward value, and backoff action as the control variable. Based on this, we design an intelligent CSMA/CA protocol based on distributed reinforcement learning. Specifically, each node adopts distributed learning decision-making, which needs to query and update information from a central status collection equipment (SCE). It improves its anti-jamming ability by learning from different environments and adapting to them. Simulation results show that the proposed algorithm is significantly better than CSMA/CA and SETL algorithms in both jamming and non-jamming environments. And it has little performance difference with the increase in the number of nodes, effectively improving the anti-jamming performance. When the communication node is 10, the normalized throughputs of the proposed algorithm in non-jamming, intermittent jamming, and random jamming are increased by 28.45%, 21.20%, and 17.07%, respectively, and the collision rates are decreased by 83.93%, 95.71%, and 81.58% respectively.

Contrasting frontal cortical responses to food versus money rewards across BMI groups

ABSTRACT Previous research has linked obesity with an altered perception of rewards. This study aimed to contrast frontal cortical activities across body mass index (BMI) groups, in responding to differential rewards (monetary versus food). A total of 60 male participants (27.43 ± 6.07 years of age; 21 normal weight [BMI: 18.5–24.9 kg·m−2]; 20 overweight [BMI: 25.0–29.9 kg·m−2]; and 19 individuals with obesity [BMI ≥ 30 kg·m−2]) were tested for their response bias towards food and money rewards using the Probabilistic Reward Task (PRT), while their frontal cortical responses were recorded using electroencephalography (EEG). The feedback-related negativity (FRN), a reliable measure of reward valuation and learning, was calculated for food (FRN(Food)) and money (FRN(Money)). Results indicate a left-lateralised frontal cortical activity associated with the food reward condition, in the group of overweight and obesity. In contrast, a right-lateralisation was observed in the money reward condition only in the group with obesity. More specifically, FRN(Food) was shown to significantly differ between left and right frontal cortical areas among individuals with obesity (p = 0.035) and overweight (p = 0.012), but not in normal-weight individuals (p = 0.153). Additionally, results revealed that FRN(Food) and FRN(Money) were significantly different for individuals with obesity (p = 0.019), but such a significant difference was not evident in the overweight and normal-weight individuals (p ≥ 0.05). These findings offer intriguing new insights into neuropsychological differentiation across BMI groups, adding to the understanding of obesity-related behaviour.

Neural Representation of Donating Time and Money.

Volunteering and charitable donations are two common forms of prosocial behavior, yet it is unclear whether these other-benefitting behaviors are supported by the same or different neurobiological mechanisms. During an fMRI task, 40 participants (20 female-identifying; age: mean = 18.92 years, range = 18.32-19.92 years) contributed their time (in minutes) and money (in dollars) to a variety of local charities. With the maximum amount of time and money that participants could spend on these charities, they did not differentially donate their time and money. At the neural level, donating time and money both showed activations in brain regions involved in cognitive control (e.g., dorsolateral PFC) and affective processing (e.g., dorsal anterior cingulate cortex), but donating time recruited regions involved in reward valuation (e.g., ventral striatum) and mentalizing (e.g., temporal pole) to a greater extent than donating money. Further, the precuneus, which is also a region involved in mentalizing, more strongly tracked the varying amount of money than time donated, suggesting that the precuneus may be more sensitive to the increasing magnitude of a nonsocial exchange (e.g., donating money is a financial exchange) than a social exchange (e.g., donating time is an interpersonal exchange). Our findings elucidate shared as well as distinct neurobiological properties of two prosocial behaviors, which have implications for how humans share different resources to positively impact their community.SIGNIFICANCE STATEMENT Prosocial behaviors broadly characterize how humans act to benefit others. Various prosocial behaviors, such as volunteering and charitable donations, share the goal of positively contributing to community. Our study identifies brain regions that may serve as ubiquitous neurobiological markers of community-based prosocial behaviors. Despite this shared goal, our study also shows that the human brain responds to donating time and money in diverging ways, such that brain regions associated with processing emotional reward and thinking about others are more strongly recruited for donating time than for money. Therefore, our study sheds light on how different personal resources, such as one's time and money, within a prosocial context are represented in the brain.

Pricing curb parking: Differentiated parking fees or cash rewards?

Parking price adjustment is an effective method for influencing curb parking choices, playing a crucial role in alleviating urban parking shortages and balancing uneven distribution. The government has extensively explored the implementation of differentiated parking fees and cash incentives for curb parking; however, few studies have investigated the effects of these policies on parking balance management. This study utilized a questionnaire survey to construct two multinomial logit models and examine the mechanisms behind drivers' choice behaviors. Taking the curb parking facilities in Yinzhou District, Ningbo as the case study area, the optimal dynamic pricing schemes for the two strategies were established, and their operational effectiveness was evaluated. The results show that parking cost, value of cash rewards, parking occupancy, and surrounding road conditions have a significant impact on curb parking choice behavior. Both strategies can effectively balance the utilization rate of curb parking space-time resources. The cash rewards strategy yields higher earnings compared to the differentiated parking fee strategy, but the latter is more conducive for reducing the occupancy during peak hours. The research findings of this study can provide insights and a theoretical foundation for real-time and proactive management of roadside parking.

Multiple intersections traffic signal control based on cooperative multi-agent reinforcement learning

For the multi-agent traffic signal controls, the traffic signal at each intersection is controlled by an independent agent. Since the control policy for each agent is dynamic, when the traffic scale is large, the adjustment of the agent's policy brings non-stationary effects over surrounding intersections, leading to the instability of the overall system. Therefore, there is the necessity to eliminate this non-stationarity effect to stabilize the multi-agent system. A collaborative multi-agent reinforcement learning method is proposed in this work to enable the system to overcome the instability problem through a collaborative mechanism. Decentralized learning with limited communication is used to reduce the communication latency between agents. The Shapley value reward function is applied to comprehensively calculate the contribution of each agent to avoid the influence of reward function coefficient variation, thereby reducing unstable factors. The Kullback-Leibler divergence is then used to distinguish the current and historical policies, and the loss function is optimized to eliminate the environmental non-stationarity. Experimental results demonstrate that the average travel time and its standard deviation are reduced by using the Shapley value reward function and optimized loss function, respectively, and this work provides an alternative for traffic signal controls on multiple intersections.