Value-based Decision Research Articles

Correction to Biderman et al. (2023).

Reports an error in "The role of memory in counterfactual valuation" by Natalie Biderman, Samuel J. Gershman and Daphna Shohamy (Journal of Experimental Psychology: General, 2023[Jun], Vol 152[6], 1754-1767). In this article, several corrections have been made to two equations, the text, and Figure 3. First, there was an error in two equations of the policy gradient model depicted in the Model Description section. The correction did not alter the main conclusion of the model, but it did change slightly the comparison between experimental conditions of each model depicted in the Results section. The correct Equation 3 and correct Equation 4 are present in the erratum. Additionally, the last four sentences in the first paragraph of the A Policy Gradient Model Captures the Memory-Based Inverse Decision Bias have been revised. Finally, the model predictions shown in the gray bars of Figure 3 were slightly modified. The online version of this article has been corrected. (The following abstract of the original article appeared in record 2023-72914-001.) Value-based decisions are often guided by past experience. If a choice led to a good outcome, we are more likely to repeat it. This basic idea is well-captured by reinforcement-learning models. However, open questions remain about how we assign value to options we did not choose and which we therefore never had the chance to learn about directly. One solution to this problem is proposed by policy gradient reinforcement-learning models; these do not require direct learning of value, instead optimizing choices according to a behavioral policy. For example, a logistic policy predicts that if a chosen option was rewarded, the unchosen option would be deemed less desirable. Here, we test the relevance of these models to human behavior and explore the role of memory in this phenomenon. We hypothesize that a policy may emerge from an associative memory trace formed during deliberation between choice options. In a preregistered study (n = 315) we show that people tend to invert the value of unchosen options relative to the outcome of chosen options, a phenomenon we term inverse decision bias. The inverse decision bias is correlated with memory for the association between choice options; moreover, it is reduced when memory formation is experimentally interfered with. Finally, we present a new memory-based policy gradient model that predicts both the inverse decision bias and its dependence on memory. Our findings point to a significant role of associative memory in valuation of unchosen options and introduce a new perspective on the interaction between decision-making, memory, and counterfactual reasoning. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Monkeys exhibit human-like gaze biases in economic decisions.

In economic decision-making individuals choose between items based on their perceived value. For both humans and nonhuman primates, these decisions are often carried out while shifting gaze between the available options. Recent studies in humans suggest that these shifts in gaze actively influence choice, manifesting as a bias in favor of the items that are viewed first, viewed last, or viewed for the overall longest duration in a given trial. This suggests a mechanism that links gaze behavior to the neural computations underlying value-based choices. In order to identify this mechanism, it is first necessary to develop and validate a suitable animal model of this behavior. To this end, we have created a novel value-based choice task for macaque monkeys that captures the essential features of the human paradigms in which gaze biases have been observed. Using this task, we identified gaze biases in the monkeys that were both qualitatively and quantitatively similar to those in humans. In addition, the monkeys' gaze biases were well-explained using a sequential sampling model framework previously used to describe gaze biases in humans-the first time this framework has been used to assess value-based decision mechanisms in nonhuman primates. Together, these findings suggest a common mechanism that can explain gaze-related choice biases across species, and open the way for mechanistic studies to identify the neural origins of this behavior.

Reduced loss aversion in value-based decision-making and edge-centric functional connectivity in patients with internet gaming disorder.

Impaired value-based decision-making is a feature of substance and behavioral addictions. Loss aversion is a core of value-based decision-making and its alteration plays an important role in addiction. However, few studies explored it in internet gaming disorder patients (IGD). In this study, IGD patients (PIGD) and healthy controls (Con-PIGD) performed the Iowa gambling task (IGT), under functional magnetic resonance imaging (fMRI). We investigated group differences in loss aversion, brain functional networks of node-centric functional connectivity (nFC) and the overlapping community features of edge-centric functional connectivity (eFC) in IGT. PIGD performed worse with lower average net score in IGT. The computational model results showed that PIGD significantly reduced loss aversion. There was no group difference in nFC. However, there were significant group differences in the overlapping community features of eFC1. Furthermore, in Con-PIGD, loss aversion was positively correlated with the edge community profile similarity of the edge2 between left IFG and right hippocampus at right caudate. This relationship was suppressed by response consistency3 in PIGD. In addition, reduced loss aversion was negatively correlated with the promoted bottom-to-up neuromodulation from the right hippocampus to the left IFG in PIGD. The reduced loss aversion in value-based decision making and their related edge-centric functional connectivity support that the IGD showed the same value-based decision-making deficit as the substance use and other behavioral addictive disorders. These findings may have important significance for understanding the definition and mechanism of IGD in the future.

Prosocial learning: Model-based or model-free?

Prosocial learning involves the acquisition of knowledge and skills necessary for making decisions that benefit others. We asked if, in the context of value-based decision-making, there is any difference between learning strategies for oneself vs. for others. We implemented a 2-step reinforcement learning paradigm in which participants learned, in separate blocks, to make decisions for themselves or for a present other confederate who evaluated their performance. We replicated the canonical features of the model-based and model-free reinforcement learning in our results. The behaviour of the majority of participants was best explained by a mixture of the model-based and model-free control, while most participants relied more heavily on MB control, and this strategy enhanced their learning success. Regarding our key self-other hypothesis, we did not find any significant difference between the behavioural performances nor in the model-based parameters of learning when comparing self and other conditions.

Functional Connectivity and Microstructural Network Correlates of Interindividual Variability in Distinct Executive Functions of Healthy Older Adults

Executive functions, essential for daily life, are known to be impaired in older age. Some executive functions, including working memory updating and value-based decision-making, are specifically sensitive to age-related deterioration. While their neural correlates in young adults are well-described, a comprehensive delineation of the underlying brain substrates in older populations, relevant to identify targets for modulation against cognitive decline, is missing. Here, we assessed letter updating and Markov decision-making task performance to operationalize these trainable functions in 48 older adults. Resting-state functional magnetic resonance imaging was acquired to quantify functional connectivity (FC) in task-relevant frontoparietal and default mode networks. Microstructure in white matter pathways mediating executive functions was assessed with diffusion tensor imaging and quantified by tract-based fractional anisotropy (FA). Superior letter updating performance correlated with higher FC between dorsolateral prefrontal cortex and left frontoparietal and hippocampal areas, while superior Markov decision-making performance correlated with decreased FC between basal ganglia and right angular gyrus. Furthermore, better working memory updating performance was related to higher FA in the cingulum bundle and the superior longitudinal fasciculus. Stepwise linear regression showed that cingulum bundle FA added significant incremental contribution to the variance explained by fronto-angular FC alone. Our findings provide a characterization of distinct functional and structural connectivity correlates associated with performance of specific executive functions. Thereby, this study contributes to the understanding of the neural correlates of updating and decision-making functions in older adults, paving the way for targeted modulation of specific networks by modulatory techniques such as behavioral interventions and non-invasive brain stimulation.

The Nexus of Policy Legitimacy and Crisismanship Performance: Examining the Harmonizing Role of Value-Based Decision Making

The Nexus of Policy Legitimacy and Crisismanship Performance: Examining the Harmonizing Role of Value-Based Decision Making

EE214 Unravelling the Elements of Value of Health Care and Assessing Their Importance Using Evidence from Two Discrete-Choice Experiments in England

Health systems worldwide are moving towards value-based decision-making by implementing care models that coordinate services around patients’ needs. Nevertheless, little is known about the elements of value of health care and their relative importance. In addition, there is an ongoing debate about whether healthcare costs should constitute part of the value metric. This study aimed to define the elements of value of health care and assess their relative importance to the public in England.

The role of memory in counterfactual valuation.

Value-based decisions are often guided by past experience. If a choice led to a good outcome, we are more likely to repeat it. This basic idea is well-captured by reinforcement-learning models. However, open questions remain about how we assign value to options we did not choose and which we therefore never had the chance to learn about directly. One solution to this problem is proposed by policy gradient reinforcement-learning models; these do not require direct learning of value, instead optimizing choices according to a behavioral policy. For example, a logistic policy predicts that if a chosen option was rewarded, the unchosen option would be deemed less desirable. Here, we test the relevance of these models to human behavior and explore the role of memory in this phenomenon. We hypothesize that a policy may emerge from an associative memory trace formed during deliberation between choice options. In a preregistered study (n = 315) we show that people tend to invert the value of unchosen options relative to the outcome of chosen options, a phenomenon we term inverse decision bias. The inverse decision bias is correlated with memory for the association between choice options; moreover, it is reduced when memory formation is experimentally interfered with. Finally, we present a new memory-based policy gradient model that predicts both the inverse decision bias and its dependence on memory. Our findings point to a significant role of associative memory in valuation of unchosen options and introduce a new perspective on the interaction between decision-making, memory, and counterfactual reasoning. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

RWD141 AI-Powered Pre-Competitive Disease and Care Modeling: Building a Reference Disease-Centric Framework for Value-Based Decision Making

Drugs evaluation has been undergoing a transformational shift from the classical sequence of randomized control trials (RCT) and Real-World (RW) studies towards a more agile process capitalizing better and faster from retrospective data. The purpose of this work is to propose a regulatory-grade, data-integrative, real-world-value-based and disease-centric approach to accelerate and de-risk drugs development and launch. This approach relies on disease and care modeling on the pre-competitive space and federated data sourcing.

GABAergic and glutamatergic control of perceptual versus value-based decisions

GABAergic and glutamatergic control of perceptual versus value-based decisions

Muscarinic acetylcholine receptor antagonism attenuates sensitivity to learned values in value-based decision making

Muscarinic acetylcholine receptor antagonism attenuates sensitivity to learned values in value-based decision making

Contextual valence influences the neural dynamics of time and magnitude representation during feedback evaluation.

Contextual valence is an important dimension during value-based decision-making. Previous research has revealed behavioral and neural asymmetries between the gain context and the loss context. The present event-related potential study investigated the effects of contextual valence on neural dynamics underlying magnitude and time, two important reward dimensions, during feedback evaluation. Forty-two participants performed a simple guessing task in which they experienced both a gain context wherein high or low rewards were delivered immediately or six months later, and a loss context wherein high or low losses were delivered in the same way. Results showed that in the gain context, time and magnitude information were processed in a parallel way during the time windows of the reward positivity (RewP) and the P3. In the loss context, however, time and magnitude information were processed in a serial way such that time information was encoded during the RewP and P3 periods, whereas magnitude information was not tracked until the time window of the late positive potential. Our findings suggest that the neural dynamics underlying time and magnitude information are distinct between the gain and loss contexts, thus providing a novel perspective for the well-known gain-loss asymmetry.

Genetic Counseling in Kidney Disease: A Perspective

As genetic testing is increasingly integrated into nephrology practice there is a growing need for partnership with genetic experts. Genetic counselors are ideally suited to fill this role. The value of genetic counseling is born out of the clinical value of genetic test results against the backdrop of the complexity of genetic testing. Genetic counselors who specialize in nephrology are trained to understand and explain the potential effects of genes on kidney disease, which can enable patients to make informed decisions about proceeding with genetic testing, navigating variants of uncertain significance, educating on extrarenal features of hereditary kidney disease, facilitating cascade testing, providing post-test education about testing results, and assisting with family planning. Genetic counselors can partner with the nephrologist and provide the knowledge needed to maximize the use of genetic testing for patients for nephrology consultation. Genetic counseling is more than an element or extension of genetic testing; it is a dynamic, shared conversation between the patient and the genetic counselor where concerns, sentiments, information, and education are exchanged, and value-based decision making is facilitated.

Systematic review of shared decision-making interventions for people living with chronic respiratory diseases

ObjectiveShared decision-making (SDM) supports patients to make informed and value-based decisions about their care. We are developing an intervention to enable healthcare professionals to support patients’ pulmonary rehabilitation (PR) decision-making....

The nematode worm C. elegans chooses between bacterial foods as if maximizing economic utility.

In value-based decision making, options are selected according to subjective values assigned by the individual to available goods and actions. Despite the importance of this faculty of the mind, the neural mechanisms of value assignments, and how choices are directed by them, remain obscure. To investigate this problem, we used a classic measure of utility maximization, the Generalized Axiom of Revealed Preference, to quantify internal consistency of food preferences in Caenorhabditis elegans, a nematode worm with a nervous system of only 302 neurons. Using a novel combination of microfluidics and electrophysiology, we found that C. elegans food choices fulfill the necessary and sufficient conditions for utility maximization, indicating that nematodes behave as if they maintain, and attempt to maximize, an underlying representation of subjective value. Food choices are well-fit by a utility function widely used to model human consumers. Moreover, as in many other animals, subjective values in C. elegans are learned, a process we find requires intact dopamine signaling. Differential responses of identified chemosensory neurons to foods with distinct growth potentials are amplified by prior consumption of these foods, suggesting that these neurons may be part of a value-assignment system. The demonstration of utility maximization in an organism with a very small nervous system sets a new lower bound on the computational requirements for utility maximization and offers the prospect of an essentially complete explanation of value-based decision making at single neuron resolution in this organism.

To escape or to pursue: opposite decision making concerning a single moving object is influenced by starvation and sex.

Decision-making processes in the context of prey-predator interactions are studied from the side of the prey or the predator. Thus, prey capture and escape behaviours are researched separately, using different stimuli in different species. The crab Neohelice preys upon individuals of its own species; hence, it behaves as prey and as predator. These two innate opposite behaviours can be elicited by the same object moving on the ground. Here, we studied how the decision to perform avoidance, predatory or freezing responses to a moving dummy depends on sex and starvation level. In the first experiment, we assessed the probability of each response type in unfed crabs for 22 days. Males showed a higher predatory response probability than females. When starvation increased, the predatory response increased, while avoidance and freezing declined, but this only occurred in males. In the second experiment, we compared regularly fed and unfed males for 17 days. While fed crabs did not change their behaviour throughout the experiment, unfed crabs significantly intensified their predatory responses, displayed different exploratory activities and pursued earlier than fed crabs. Our results show the unusual situation of an animal that, to deal with a single stimulus, has to choose between opposite innate behaviours. This is a value-based decision as it is affected by factors other than the stimulus itself.

Developing Criteria for Health Economic Quality Evaluation Tool

ObjectivesBecause existing publication guidelines and checklists have limitations when used to assess the quality of cost-effectiveness analysis, we developed a novel quality assessment tool for cost-effectiveness analyses, differentiating methods and reporting quality and incorporating the relative importance of different quality attributes. MethodsWe defined 15 quality domains from a scoping review and identified 72 methods and reporting quality attributes (36 each). After designing a best-worst scaling survey, we fielded an online survey to researchers and practitioners to estimate the relative importance of the attributes in February 2021. We analyzed the survey data using a sequential conditional logit model. The final tool included 48 quality attributes deemed most important for assessing methods and reporting quality (24 each), accompanied by a free and web-based scoring system. ResultsA total of 524 participants completed the methodology section, and 372 completed both methodology and reporting sections. Quality attributes pertaining to the “modeling” and “data inputs and evidence synthesis” domains were deemed most important for methods quality, including “structure of the model reflects the underlying condition and intervention’s impact” and “model validation is conducted.” Quality attributes pertaining to “modeling” and “Intervention/comparator(s)” domains were considered most important for reporting quality, including “model descriptions are detailed enough for replication.” Despite its growing prominence, “equity considerations” were not deemed as important as other quality attributes. ConclusionsThe Criteria for Health Economic Quality Evaluation tool allows users to differentiate methods and reporting as well as quantifies the relative importance of quality attributes. Alongside other considerations, it could help assess and improve the quality of cost-effectiveness evidence to inform value-based decisions.

Episodic Memory Retrieval Affects the Onset and Dynamics of Evidence Accumulation during Value-based Decisions.

In neuroeconomics, there is much interest in understanding simple value-based choices where agents choose between visually presented goods, comparable to a shopping scenario in a supermarket. However, many everyday decisions are made in the physical absence of the considered goods, requiring agents to recall information about the goods from memory. Here, we asked whether and how this reliance on an internal memory representation affects the temporal dynamics of decision making on a cognitive and neural level. Participants performed a remember-and-decide task in which they made simple purchasing decisions between money offers and snack items while undergoing EEG. Snack identity was presented either visually (value trials) or had to be recalled from memory (memory trials). Behavioral data indicated comparable choice consistency across both trial types, but considerably longer RTs in memory trials. Drift-diffusion modeling suggested that this RT difference was because of longer nondecision time of decision processes as well as altered evidence accumulation dynamics (lower accumulation rate and higher decision threshold). The nondecision time effect was supported by a delayed onset of the lateralized readiness potential. These results show that both, decision and nondecision, processes are prolonged when participants need to resort to internal memory representations during value-based decisions.

Adaptive tuning of human learning and choice variability to unexpected uncertainty.

Human value-based decisions are notably variable under uncertainty. This variability is known to arise from two distinct sources: variable choices aimed at exploring available options and imprecise learning of option values due to limited cognitive resources. However, whether these two sources of decision variability are tuned to their specific costs and benefits remains unclear. To address this question, we compared the effects of expected and unexpected uncertainty on decision-making in the same reinforcement learning task. Across two large behavioral datasets, we found that humans choose more variably between options but simultaneously learn less imprecisely their values in response to unexpected uncertainty. Using simulations of learning agents, we demonstrate that these opposite adjustments reflect adaptive tuning of exploration and learning precision to the structure of uncertainty. Together, these findings indicate that humans regulate not only how much they explore uncertain options but also how precisely they learn the values of these options.

Neurons in human pre-supplementary motor area encode key computations for value-based choice.

Adaptive behaviour in real-world environments requires that choices integrate several variables, including the novelty of the options under consideration, their expected value and uncertainty in value estimation. Here, to probe how integration over decision variables occurs during decision-making, we recorded neurons from the human pre-supplementary motor area (preSMA), ventromedial prefrontal cortex and dorsal anterior cingulate. Unlike the other areas, preSMA neurons not only represented separate pre-decision variables for each choice option but also encoded an integrated utility signal for each choice option and, subsequently, the decision itself. Post-decision encoding of variables for the chosen option was more widely distributed and especially prominent in the ventromedial prefrontal cortex. Our findings position the human preSMA as central to the implementation of value-based decisions.