Value-based Decision Research Articles

Risk and aversion coding in human habenula high gamma activity.

Neurons in the primate lateral habenula fire in response to punishments and are inhibited by rewards. Through its modulation of midbrain monoaminergic activity, the habenula is believed to play an important role in adaptive behavioural responses to punishment and underlie depressive symptoms and their alleviation with ketamine. However, its role in value-based decision-making in humans is poorly understood due to limitations with non-invasive imaging methods which measure metabolic, not neural, activity with poor temporal resolution. Here, we overcome these limitations to more closely bridge the gap between species by recording local field potentials directly from the habenula in 12 human patients receiving deep brain stimulation treatment for bipolar disorder (n = 4), chronic pain (n = 3), depression (n = 3) and schizophrenia (n = 2). This allowed us to record neural activity during value-based decision-making tasks involving monetary rewards and losses. High-frequency gamma (60-240 Hz) activity, a proxy for population-level spiking involved in cognitive computations, increased during the receipt of loss and decreased during receipt of reward. Furthermore, habenula high gamma also encoded risk during decision-making, being larger in amplitude for high compared to low risk. For both risk and aversion, differences between conditions peaked approximately between 400 and 750 ms after stimulus onset. The findings not only demonstrate homologies with the primate habenula but also extend its role to human decision-making, showing its temporal dynamics and suggesting revisions to current models. The findings suggest that habenula high gamma could be used to optimize real-time closed-loop deep brain stimulation treatment for mood disturbances and impulsivity in psychiatric disorders.

The role of reinforcement learning and value-based decision-making frameworks in understanding food choice and eating behaviors.

The obesogenic food environment includes easy access to highly-palatable, energy-dense, "ultra-processed" foods that are heavily marketed to consumers; therefore, it is critical to understand the neurocognitive processes the underlie overeating in response to environmental food-cues (e.g., food images, food branding/advertisements). Eating habits are learned through reinforcement, which is the process through which environmental food cues become valued and influence behavior. This process is supported by multiple behavioral control systems (e.g., Pavlovian, Habitual, Goal-Directed). Therefore, using neurocognitive frameworks for reinforcement learning and value-based decision-making can improve our understanding of food-choice and eating behaviors. Specifically, the role of reinforcement learning in eating behaviors was considered using the frameworks of (1) Sign-versus Goal-Tracking Phenotypes; (2) Model-Free versus Model-Based; and (3) the Utility or Value-Based Model. The sign-and goal-tracking phenotypes may contribute a mechanistic insight on the role of food-cue incentive salience in two prevailing models of overconsumption-the Extended Behavioral Susceptibility Theory and the Reactivity to Embedded Food Cues in Advertising Model. Similarly, the model-free versus model-based framework may contribute insight to the Extended Behavioral Susceptibility Theory and the Healthy Food Promotion Model. Finally, the value-based model provides a framework for understanding how all three learning systems are integrated to influence food choice. Together, these frameworks can provide mechanistic insight to existing models of food choice and overconsumption and may contribute to the development of future prevention and treatment efforts.

Distinct neural dynamics underlying risk and ambiguity during valued-based decision making.

Uncertainty can be fractioned into risk and ambiguity psychologically and neurobiologically. However, whether and how risk and ambiguity are dissociated in terms of neural dynamics during value-based decision making remain elusive. The present event-related potential (ERP) study addressed theseissues by asking participants to perform a wheel-of-fortune task either during a risky context (Experiment 1; N=30) where outcome probability was known or during an ambiguous context (Experiment 2; N=30) where outcome probability was unknown. Results revealed that the cue-P3 was more enhanced for risk versus ambiguity during the anticipatory phase, whereas the RewP was more increased for ambiguity than risk during the consummatory phase. Moreover, the SPN and the fb-P3 components were further modulated by the levels of risk and ambiguity, respectively. These findings demonstrate a neural dissociation between risk and ambiguity, which unfolds from the anticipatory phase to the consummatory phase.

Behavioral and neural representation of expected reward and risk

When faced with uncertainty, individuals’ value-based decisions are influenced by the expected rewards and risks. Understanding how reward and risk are processed and integrated at the behavioral and neural levels is essential for building up utility theories. Using a modified monetary incentive delay task in which the mean of two possible outcomes (expected reward) and the standard deviation (SD) of the possible outcomes (risk) were parametrically manipulated and orthogonalized, we measured eye movements, response times (RTs), and brain activity when participants seek to secure a reward. We found that RTs varied as a function of the mean but not the SD of the potential reward, suggesting that expected rewards are the main driver of RTs. Moreover, the difference between gazes focused on high vs. low value rewards became smaller when the magnitude of the potential reward (mean of possible outcomes) was larger and when risk (SD of possible outcomes) became smaller, highlighting that reward and risk have different effects on attention deployment. Processing the mean reward activated the striatum. The positive striatal connectivity to the amygdala and negative striatal connectivity to the superior frontal gyrus were correlated with individuals’ sensitivity to the expected reward. In contrast, processing risk activated the anterior insula. Its positive connectivity to the ventromedial prefrontal cortex and negative connectivity to the anterior midcingulate cortex were correlated with individual differences in risk sensitivity, further suggesting the functional dissociation of reward and risk at the neural level. Our findings, based on several different measures, delineate the distinct representations of reward and risk in non-decision contexts and provide insight into how these utility parameters modulate attention, motivation, and brain networks.

Reward salience but not spatial attention dominates the value representation in the orbitofrontal cortex

The orbitofrontal cortex (OFC) encodes value and plays a key role in value-based decision-making. However, the attentional modulation of the OFC’s value encoding is poorly understood. We trained two monkeys to detect a luminance change at a cued location between a pair of visual stimuli, which were over-trained pictures associated with different amounts of juice reward and, thus, different reward salience. Both the monkeys’ behavior and the dorsolateral prefrontal cortex neuronal activities indicated that the monkeys actively directed their spatial attention toward the cued stimulus during the task. However, the OFC’s neuronal responses were dominated by the stimulus with higher reward salience and encoded its value. The value of the less salient stimulus was only weakly represented regardless of spatial attention. The results demonstrate that reward and spatial attention are distinctly represented in the prefrontal cortex and the OFC maintains a stable representation of reward salience minimally affected by attention.

Temporal scaling of human scalp-recorded potentials

Much of human behavior is governed by common processes that unfold over varying timescales. Standard event-related potential analysis assumes fixed-duration responses relative to experimental events. However, recent single-unit recordings in animals have revealed neural activity scales to span different durations during behaviors demanding flexible timing. Here, we employed a general linear modeling approach using a combination of fixed-duration and variable-duration regressors to unmix fixed-time and scaled-time components in human magneto-/electroencephalography (M/EEG) data. We use this to reveal consistent temporal scaling of human scalp-recorded potentials across four independent electroencephalogram (EEG) datasets, including interval perception, production, prediction, and value-based decision making. Between-trial variation in the temporally scaled response predicts between-trial variation in subject reaction times, demonstrating the relevance of this temporally scaled signal for temporal variation in behavior. Our results provide a general approach for studying flexibly timed behavior in the human brain.

Embodied decisions during walking.

Research on embodied decision-making only recently started to examine whether and how concurrent actions influence value-based decisions. For instance, during walking humans preferably make decisions that align with a turn toward the side of their current swing leg, sometimes resulting in unfavorable choices (e.g., less reward). It is suggested that concurrent movements influence decision-making by coincidental changes in motor costs. If this is true, systematic manipulations of motor costs should bias decisions. To test this, participants had to accumulate rewards (i.e., points) by walking and turning toward left and right targets displaying rewards across three experiments. In experiments 1a and 1b, we manipulated the turning cost based on the current swing leg by applying different symmetric turning magnitudes (i.e., same angles for left and right targets). In experiment 2, we manipulated the turning cost by administering asymmetric turning magnitudes (i.e., different angles for left and right targets). Finally, in experiment 3, we increased the cost of walking by adding ankle weights. Altogether, the experiments support the claim that differences in motor costs influenced participants' decisions: experiments 1a and 1b revealed that the swing leg effect and stepping behavior were moderated by turning magnitude. In experiment 2, participants showed a preference for less costly, smaller turning magnitudes. Experiment 3 replicated the swing leg effect when motor costs were increased by means of ankle weights. In conclusion, these findings provide further evidence that value-based decisions during ongoing actions seem to be influenced by dynamically changing motor costs, thereby supporting the concept of "embodied decision-making."NEW & NOTEWORTHY Motor processes of concurrent movements have been shown to influence embodied decision-making. It is hypothesized that this is driven by coincidental changes in motor costs. We tested this claim by systematically manipulating motor costs of choice options during walking. In three experiments we show how variations in motor cost (e.g., turning angle or stepping constraints) bias decision-making, thereby supporting the concept of "embodied decision-making."

Predicting Product Preferences on Retailers' Web Shops through Measurement of Gaze and Pupil Size Dynamics.

Previous studies used gaze behavior to predict product preference in value-based decision-making, based on gaze angle variables such as dwell time, fixation duration and the first fixated product. While the application for online retail seems obvious, research with realistic web shop stimuli has been lacking so far. Here, we studied the decision process for 60 Dutch web shops of a variety of retailers, by measuring eye movements and pupil size during the viewing of web shop images. The outcomes of an ordinal linear regression model showed that a combination of gaze angle variables accurately predicted product choice, with the total dwell time being the most predictive gaze dynamic. Although pupillometric analysis showed a positive relationship between pupil dilation and product preference, adding pupil size to the model only slightly improved the prediction accuracy. The current study holds the potential to substantially improve retargeting mechanisms in online marketing based on consumers’ gaze information. Also, gaze-based product preference proves to be a valuable metric in pre-testing product introductions for market research and prevent product launches from failure.

The value of confidence: Confidence prediction errors drive value-based learning in the absence of external feedback.

Reinforcement learning algorithms have a long-standing success story in explaining the dynamics of instrumental conditioning in humans and other species. While normative reinforcement learning models are critically dependent on external feedback, recent findings in the field of perceptual learning point to a crucial role of internally generated reinforcement signals based on subjective confidence, when external feedback is not available. Here, we investigated the existence of such confidence-based learning signals in a key domain of reinforcement-based learning: instrumental conditioning. We conducted a value-based decision making experiment which included phases with and without external feedback and in which participants reported their confidence in addition to choices. Behaviorally, we found signatures of self-reinforcement in phases without feedback, reflected in an increase of subjective confidence and choice consistency. To clarify the mechanistic role of confidence in value-based learning, we compared a family of confidence-based learning models with more standard models predicting either no change in value estimates or a devaluation over time when no external reward is provided. We found that confidence-based models indeed outperformed these reference models, whereby the learning signal of the winning model was based on the prediction error between current confidence and a stimulus-unspecific average of previous confidence levels. Interestingly, individuals with more volatile reward-based value updates in the presence of feedback also showed more volatile confidence-based value updates when feedback was not available. Together, our results provide evidence that confidence-based learning signals affect instrumentally learned subjective values in the absence of external feedback.

Magnitude-sensitive reaction times reveal non-linear time costs in multi-alternative decision-making.

Optimality analysis of value-based decisions in binary and multi-alternative choice settings predicts that reaction times should be sensitive only to differences in stimulus magnitudes, but not to overall absolute stimulus magnitude. Yet experimental work in the binary case has shown magnitude sensitive reaction times, and theory shows that this can be explained by switching from linear to multiplicative time costs, but also by nonlinear subjective utility. Thus disentangling explanations for observed magnitude sensitive reaction times is difficult. Here for the first time we extend the theoretical analysis of geometric time-discounting to ternary choices, and present novel experimental evidence for magnitude-sensitivity in such decisions, in both humans and slime moulds. We consider the optimal policies for all possible combinations of linear and geometric time costs, and linear and nonlinear utility; interestingly, geometric discounting emerges as the predominant explanation for magnitude sensitivity.

Shared Decision-Making as an Imperative of Modern Medicine?

Shared Decision-Making as an Imperative of Modern Medicine? Abstract. Shared Decision-Making (SDM) is a normative model of decision-making in the clinical context, in which physicians and patients assess alternative treatment options and jointly take value-based decisions based on the preferences of patients. It is important that the exchange of information is free and uncoerced and that the advantages and disadvantages of different treatment options are carefully considered. The aim of this article is to reconstruct SDM historically and conceptually and to embed the model in a normative theory of patient autonomy. Furthermore, we define the scope of SDM in clinical practice by pointing to the limits of the SDM model. Our aim is to determine under what conditions patients and doctors take decisions together, what role doctors shall play, and when patients potentially participate too much or too little in the decision-making process.

Electrophysiological measures of conflict and reward processing are associated with decisions to engage in physical effort.

Anterior cingulate cortex (ACC), a key brain region involved in cognitive control and decision making, is suggested to mediate effort- and value-based decision making, but the specific role of ACC in this process remains debated. Here we used frontal midline theta (FMT) and the reward positivity (RewP) to examine ACC function in a value-based decision making task requiring physical effort. We investigated whether (1) FMT power is sensitive to the difficulty of the decision or to selecting effortful actions, and (2) RewP is sensitive to the subjective value of reward outcomes as a function of effort investment. On each trial, participants chose to execute a low-effort or a high-effort behavior (that required squeezing a hand-dynamometer) to obtain smaller or larger rewards, respectively, while their brainwaves were recorded. We replicated prior findings that tonic FMT increased over the course of the hour-long task, which suggests increased application of control in the face of growing fatigue. RewP amplitude also increased following execution of high-effort compared to low-effort behavior, consistent with increased valuation of reward outcomes by ACC. Although neither phasic nor tonic FMT were associated with decision difficulty or effort selection per se, an exploratory analysis revealed that the interaction of phasic FMT and expected value of choice predicted effort choice. This interaction suggests that phasic FMT increases specifically under situations of decision difficulty when participants ultimately select a high-effort choice. These results point to a unique role for ACC in motivating and persisting at effortful behavior when decision conflict is high.

Effort Reinforces Learning.

Humans routinely learn the value of actions by updating their expectations based on past outcomes - a process driven by reward prediction errors (RPEs). Importantly, however, implementing a course of action also requires the investment of effort. Recent work has revealed a close link between the neural signals involved in effort exertion and those underpinning reward-based learning, but the behavioral relationship between these two functions remains unclear. Across two experiments, we tested healthy male and female human participants (N = 140) on a reinforcement learning task in which they registered their responses by applying physical force to a pair of hand-held dynamometers. We examined the effect of effort on learning by systematically manipulating the amount of force required to register a response during the task. Our key finding, replicated across both experiments, was that greater effort increased learning rates following positive outcomes and decreased them following negative outcomes, which corresponded to a differential effect of effort in boosting positive RPEs and blunting negative RPEs. Interestingly, this effect was most pronounced in individuals who were more averse to effort in the first place, raising the possibility that the investment of effort may have an adaptive effect on learning in those less motivated to exert it. By integrating principles of reinforcement learning with neuroeconomic approaches to value-based decision-making, we show that the very act of investing effort modulates one's capacity to learn, and demonstrate how these functions may operate within a common computational framework.SIGNIFICANCE STATEMENT Recent work suggests that learning and effort may share common neurophysiological substrates. This raises the possibility that the very act of investing effort influences learning. Here, we tested whether effort modulates teaching signals in a reinforcement learning paradigm. Our results showed that effort resulted in more efficient learning from positive outcomes and less efficient learning from negative outcomes. Interestingly, this effect varied across individuals, and was more pronounced in those who were more averse to investing effort in the first place. These data highlight the importance of motivational factors in a common framework of reward-based learning, which integrates the computational principles of reinforcement learning with those of value-based decision-making.

Aging shifts the relative contributions of episodic and semantic memory to decision-making.

Healthy aging is accompanied by well-characterized shifts in memory systems: episodic memory tends to decline with age while semantic memory remains relatively intact, with some knowledge domains strengthening. Beyond reflecting on the past, these distinct memory systems often guide decisions about the future. Yet how such age-related memory shifts influence simple value-based choices remain understudied. Here, younger (18-24 years) and older (61-75 years) adults completed a card game in which they could use task-relevant episodic memories to maximize the number of points they earned. Critically, they could also use task-irrelevant semantic memories to guide their choices. Both younger and older adults successfully used episodic memory to make decisions, but older adults did so less reliably than younger adults. Further, while younger adults strategically suppressed task-irrelevant semantic memories when a relevant episodic memory could be used, older adults used semantic memory to guide their decisions regardless of the relevance of episodic memory. We provide evidence that declining inhibitory control may play a role in how older adults arbitrate between competing memory sources when making decisions. These effects are consistent with the literature on age-related shifts in memory and cognitive control systems and add to a growing body of work on how episodic memories inform reinforcement learning and value-based decision-making. Our findings highlight how patterns of age-related memory differences can have consequences for value-based choices, which has implications for other types of decision-making, from the economic to the mundane. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

TU-174. A causal role of non-invasive vmPFC stimulation in value-based decisions in humans

TU-174. A causal role of non-invasive vmPFC stimulation in value-based decisions in humans

Association of Direct-to-Consumer Advertising of Prescription Drugs With Consumer Health-Related Intentions and Beliefs Among Individuals at Risk of Cardiovascular Disease

Consumers in the US are exposed to unprecedented high levels of direct-to-consumer advertising (DTCA) for prescription drugs, yet there is limited evidence regarding their effect on health-related intentions and beliefs. To provide evidence on the association of DTCA for prescription drugs with consumer health-related intentions and beliefs. This cross-sectional study recruited participants from a nationally representative sample of individuals at high risk of cardiovascular disease. Participants were randomly assigned into 1 of 3 study arms: (1) exposure to DTCA for heart disease medications (treatment 1 [n = 926]), (2) exposure to DTCA for heart disease medications with price disclosure (treatment 2 [n = 921]), (3) and exposure to nonpharmaceutical advertising (control group [n = 902]). Each study arm viewed five 1-minute video advertisements, totaling 5 minutes of advertising exposure. The 2 treatment arms viewed pharmaceutical advertising videos for 4 heart disease medications, and the control arm viewed nonpharmaceutical advertising videos. Participants then completed a survey questionnaire to measure medication- and lifestyle-related intentions and health-related beliefs and perceptions. Direct-to-consumer advertising for heart disease medications (treatment 1), DTCA for heart disease medications with price disclosure (treatment 2), and nonpharmaceutical advertising (control group). The primary outcomes included ordinal measures of medication- and lifestyle-related intentions, health-related beliefs, and brand perceptions. Among the 2874 included participants (mean [SD] age, 53.8 [7.1] years; 1549 [54%] male) χ2 tests confirmed that there were no statistically significant differences in baseline demographic characteristics across study arms. There was a positive association between DTCA and medication-related behavioral intentions, including intention to switch medication (marginal effect [ME] = 0.004; P = .002) and engage in information-seeking behaviors (ME = 0.02; P = .01). There was no evidence that pharmaceutical DTCA discouraged use of nonpharmacological lifestyle interventions that can help manage heart disease (eg, diet and exercise), and DTCA exposure also had a positive association with consumers' favorable perceptions of pharmaceutical manufacturers (competence: ME = 0.03; P = .01; innovative: ME = 0.03; P = .008). There was no evidence for differential associations of price disclosures in DTCA. In this cross-sectional study, results showed that brief exposure to pharmaceutical DTCA had a large and positive association with medication-related demand intentions with no offsetting negative spillovers on lifestyle-related intentions. Lack of associations with price disclosure in DTCA suggests that policy makers should consider alternative strategies to promote value-based decision-making for prescription drugs.

Episodic memory integration shapes value-based decision-making in spatial navigation.

Valued-based decision-making has been studied for decades in myriad topics such as consumer spending and gambling, but very rarely in spatial navigation despite the link between the two being highly relevant to survival. Furthermore, how people integrate episodic memories, and what factors are related to the extent of memory integration in value-based decision-making, remain largely unknown. In the current study, participants learned locations of various objects in a virtual environment and then decided whether to reach goal objects from familiar starting locations or unpredictable ones, with different penalties associated with each option. We developed computational models to test whether, when given an object to find, participants' starting location decisions reflected their past performance specific to that goal (Target-specific model) or integrated memory from performance with all goals in the environment (Target-common model). Because participants' wayfinding performance improved throughout the experiment, we were able to examine what factors related to the generalization of past experience. We found that most participants' decisions were better fit by the Target-common model, and for the people whose decisions were better fit by the Target-common model this integrative tendency may be tied to their concurrently greater performance variability with individual targets. Moreover, greater success on our task was predicted by an interaction between the ability to estimate probabilities relevant to decision-making and self-report general task ability. Collectively, our results show how related navigational episodic memories can be reflected in decision-making, and uncover individual differences contributing to such processes. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Supplemental Material for Behavioral Economic and Value-Based Decision-Making Constructs That Discriminate Current Heavy Drinkers Versus People Who Reduced Their Drinking Without Treatment

Supplemental Material for Behavioral Economic and Value-Based Decision-Making Constructs That Discriminate Current Heavy Drinkers Versus People Who Reduced Their Drinking Without Treatment

Data-driven conservation actions of heritage places curated with HBIM

Digital surveying tools provide a highly accurate geometric representation of cultural heritage sites in the form of point cloud data. With the recent advances in interoperability between point cloud data and Building Information Modelling (BIM), digital heritage researchers have introduced the Heritage/Historic Information Modelling (HBIM) notion to the field. As heritage data require safeguarding strategies to ensure their sustainability, the process is closely tied to conservation actions in the architectural conservation field. Focusing on the intersection of the ongoing trends in HBIM research and the global needs for heritage conservation actions, this paper tackles methodological pipelines for the data-driven management of archaeological heritage places. It illustrates how HBIM discourse could be beneficial for easing value-based decision-making in the conservation process. It introduces digital data-driven conservation actions by implementing a novel methodology for ancient building remains in Erythrae archaeological site (Turkey). The research ranges from a) surveying the in-situ remains and surrounding stones of the Heroon remains with digital photogrammetry and terrestrial laser scanning to b) designing a database system for building archaeology. The workflow offers high geometric fidelity and management of non-geometric heritage data by testing out the suitability and feasibility for the study of material culture and the physical assessment of archaeological building remains. This methodology is a fully data-enriched NURBS-based (non-uniform rational basis spline) three-dimensional (3D) model—which is integrated and operational in the BIM environment— for the holistic conservation process. Using a state-of-the-art digital heritage approach can be applied from raw data (initial stages) to decision-making about an archaeological heritage site (final stages). In conclusion, the paper offers a method for data-driven conservation actions, and given its methodological framework, it lends itself particularly well to HBIM-related solutions for building archaeology.

Supplemental Material for Evidence Accumulates for Individual Attributes During Value-Based Decisions

Supplemental Material for Evidence Accumulates for Individual Attributes During Value-Based Decisions