Uncertain Choices Research Articles

Solving multi-armed bandit problems using a chaotic microresonator comb

The Multi-Armed Bandit (MAB) problem, foundational to reinforcement learning-based decision-making, addresses the challenge of maximizing rewards amid multiple uncertain choices. While algorithmic solutions are effective, their computational efficiency diminishes with increasing problem complexity. Photonic accelerators, leveraging temporal and spatial-temporal chaos, have emerged as promising alternatives. However, despite these advancements, current approaches either compromise computation speed or amplify system complexity. In this paper, we introduce a chaotic microresonator frequency comb (chaotic comb) to tackle the MAB problem, where each comb mode is assigned to a slot machine. Through a proof-of-concept experiment, we employ 44 comb modes to address an MAB with 44 slot machines, demonstrating performance competitive with both conventional software algorithms and other photonic methods. Furthermore, the scalability of decision making is explored with up to 512 slot machines using experimentally obtained temporal chaos in different time slots. Power-law scalability is achieved with an exponent of 0.96, outperforming conventional software-based algorithms. Moreover, we find that a numerically calculated chaotic comb accurately reproduces experimental results, paving the way for discussions on strategies to increase the number of slot machines.

Framing Effect in Medical Decision Making

In the century that the medical system is becoming more and more mature, there are more options interfering patients’ eyes. Framing effect, known to be effective in one’s decision-making process, can lead patients to certain options, misunderstandings and subsequent arbitrary decisions may take place. Therefore, in this study, four main factors, or effects, are discussed based on framing effect. People behave preferences according to words they hear. This ‘words’ include descriptive adjectives attached to one event or object, numerical descriptions like survival probability and whether it is described as an uncertainty or a risk. As a result of loss aversion, a negative description will lead to risk-taking behaviours, while a positive description will lead to risk aversion ones. At the same time, if the given probability level is considerably low, patients will turn to risk aversion and choose conservative treatments, weighing life quality more than quantity. Aside from numeric, people show consistent disbelief upon uncertainty to certain risk no matter the real probability level and will set a subjective probability for uncertain choices. Health state utility can also be another indicator when judging the patients’ state because of the prospect theory. In conclusion, people with different health states put different anticipation on medical results. Ages and prior disease experience are the two main factors influencing the patients’ own measurement of their health state utilities. The older the patients are, the more utility they will assign to. Those who have historical disease assign more utility than those who do not.

Relationship between BMI and alcohol consumption levels in decision making

BackgroundDecision-making deficits in obesity and alcohol use disorder (AUD) may contribute to the choice of immediate rewards despite their long-term deleterious consequences.MethodsGambling task functional MRI in Human connectome project (HCP) dataset was used to investigate neural activation differences associated with reward or punishment (a key component of decision-making behavior) in 418 individuals with obesity (high BMI) and without obesity (lean BMI) and either at high (HR) or low (LR) risk of AUD based on their alcohol drinking levels.ResultsInteraction between BMI and alcohol drinking was seen in regions of the default mode network (DMN) and those implicated in self-related processing, memory, and salience attribution. ObesityHR relative to obesityLR also recruited DMN along with primary motor and regions implicated in inattention, negative perception, and uncertain choices, which might facilitate impulsive choices in obesityHR. Furthermore, obesityHR compared to leanHR/leanLR also demonstrated heightened activation in DMN and regions implicated in uncertain decisions.ConclusionsThese results suggest that BMI is an independent variable from that of alcohol drinking levels in neural processing of gambling tasks. Moreover, leanLR relative to leanHR, showed increased activation in motor regions [precentral and superior frontal gyrus] suggestive of worse executive function from excessive alcohol use. Delayed discounting measures failed to distinguish between obesity and high alcohol drinking levels, which as for gambling task results suggests independent negative effects of obesity and chronic alcohol drinking on decision-making. These findings highlight distinct associations of obesity and high-risk alcohol drinking with two key constituents of decision-making behavior.

How chimpanzees decide in the face of social and nonsocial uncertainty

Uncertainty can arise in interactions with both social partners and nonliving objects. Previous research has shown that humans display higher aversion to uncertainty arising from social interactions than to uncertainty caused by interactions with objects such as gambling machines, and that this difference may be mediated by betrayal aversion. We investigated whether chimpanzees, Pan troglodytes, differentiate between social and nonsocial forms of uncertainty. Subjects participated in two experiments, each involving a social and a nonsocial condition. In both experiments, choosing the safe option resulted in immediate access to low-value food. Choosing the uncertain option could result in access to high-value food, but only if the partner (social condition) or a machine (nonsocial condition) proved trustworthy. In experiment 1, where chimpanzees had no prior information on reciprocation rates (i.e. decided under uncertainty), chimpanzees were less likely to choose the uncertain option when they interacted with a partner than with a machine. When they did choose the uncertain option, chimpanzees also hesitated longer in the social condition. In experiment 2, where chimpanzees had learned the statistical probabilities on reciprocation rates (i.e. decided under risk), they did not distinguish between social and nonsocial situations and were generally risk averse. These results suggest that chimpanzees are more averse to engaging in uncertain choices when the source of uncertainty is a conspecific than when it is a machine; when confronted with risky choices, chimpanzees show no such tendency.

Shifting uncertainty intolerance: methylphenidate and attention-deficit hyperactivity disorder

Risk evaluation is a critical component of decision making. Risk tolerance is relevant in both daily decisions and pathological disorders such as attention-deficit hyperactivity disorder (ADHD), where impulsivity is a cardinal symptom. Methylphenidate, a commonly prescribed drug in ADHD, improves attention but has mixed reports on risk-based decision making. Using a double-blinded placebo protocol, we studied the risk attitudes of ADHD patients and age-matched healthy volunteers while performing the 2-step sequential learning task and examined the effect of methylphenidate on their choices. We then applied a novel computational analysis using the hierarchical drift–diffusion model to extract parameters such as threshold (‘a’—amount of evidence accumulated before making a decision), drift rate (‘v’—information processing speed) and response bias (‘z’ apriori bias towards a specific choice) focusing specifically on risky choice preference. Critically, we show that ADHD patients on placebo have an apriori bias towards risky choices compared to controls. Furthermore, methylphenidate enhanced preference towards risky choices (higher apriori bias) in both groups but had a significantly greater effect in the patient population independent of clinical scores. Thus, methylphenidate appears to shift tolerance towards risky uncertain choices possibly mediated by prefrontal dopaminergic and noradrenergic modulation. We emphasise the utility of computational models in detecting underlying processes. Our findings have implications for subtle yet differential effects of methylphenidate on ADHD compared to healthy population.

Derivation and Application of the Subjective–Objective Probability Relationship from Entropy: The Entropy Decision Risk Model (EDRM)

The uncertainty, or entropy, of an atom of an ideal gas being in a certain energy state mirrors the way people perceive uncertainty in the making of decisions, uncertainty that is related to unmeasurable subjective probability. It is well established that subjects evaluate risk decisions involving uncertain choices using subjective probability rather than objective, which is usually calculated using empirically derived decision weights, such as those described in Prospect Theory; however, an exact objective–subjective probability relationship can be derived from statistical mechanics and information theory using Kullback–Leibler entropy divergence. The resulting Entropy Decision Risk Model (EDRM) is based upon proximity or nearness to a state and is predictive rather than descriptive. A priori EDRM, without factors or corrections, accurately aligns with the results of prior decision making under uncertainty (DMUU) studies, including Prospect Theory and others. This research is a first step towards the broader effort of quantifying financial, programmatic, and safety risk decisions in fungible terms, which applies proximity (i.e., subjective probability) with power utility to evaluate choice preference of gains, losses, and mixtures of the two in terms of a new parameter referred to as Prospect. To facilitate evaluation of the EDRM against prior studies reported in terms of the percentage of subjects selecting a choice, the Percentage Evaluation Model (PEM) is introduced to convert choice value results into subject response percentages, thereby permitting direct comparison of a utility model for the first time.

Neural mechanisms underlying the effects of physical fatigue on effort-based choice

Physical fatigue crucially influences our decisions to partake in effortful action. However, there is a limited understanding of how fatigue impacts effort-based decision-making at the level of brain and behavior. We use functional magnetic resonance imaging to record markers of brain activity while human participants engage in uncertain choices for prospective physical effort, before and after bouts of exertion. Using computational modeling of choice behavior we find that fatiguing exertions cause participants to increase their subjective cost of effort, compared to a baseline/rested state. We describe a mechanism by which signals related to motor cortical state in premotor cortex influence effort value computations, instantiated by insula, thereby increasing an individual’s subjective valuation of prospective physical effort while fatigued. Our findings provide a neurobiological account of how information about bodily state modulates decisions to engage in physical activity.

MULTIPLE PERSPECTIVES ON WHAT (IF ANY) IS AN OPTIMAL TIME FOR PEOPLE WITH DEMENTIA TO MOVE TO A CARE HOME

Deciding an ‘optimal’ time for a person with dementia to move to a care home may be difficult for people with dementia, family carers, and professionals who support them; but there is currently limited evidence to help make this decision. Using phenomenology, we carried out qualitative interviews with 20 family carers, 5 people with dementia, 20 care home managers and 20 social workers, about their experiences, views and attitudes regarding timing of a move to a care home. Social workers indicated that managing risks and safety of person with dementia living in their own home were paramount when considering where person with dementia should live. These concerns included mishandling gas and electrical equipment at home, wandering and getting lost outside, and breakdown of family care. They and care home managers valued wishes of the person with dementia, and minimising any emotional distress to them when a move did come about. Family carers reported feeling stressed, and guilty around decision-making and ultimate move of their relative to a care home. Many described weighing up various risks when reaching ‘tipping point’ and making trade-offs between available options or uncertain future choices. Participants with dementia recognised they had struggled to cope at home and needed more support; however, many found the move difficult as they relocated nearer to family, away from their home and friends, and resigned themselves to less independence. Most people with dementia reported that their carers initiated discussions about timing of move, and that family discussions about this were common.

Roles of Ventromedial Prefrontal Cortex and Anterior Cingulate in Subjective Valuation of Prospective Effort.

The perceived effort level of an action shapes everyday decisions. Despite the importance of these perceptions for decision-making, the behavioral and neural representations of the subjective cost of effort are not well understood. While a number of studies have implicated anterior cingulate cortex (ACC) in decisions about effort/reward trade-offs, none have experimentally isolated effort valuation from reward and choice difficulty, a function that is commonly ascribed to this region. We used functional magnetic resonance imaging to monitor brain activity while human participants engaged in uncertain choices for prospective physical effort. Our task was designed to examine effort-based decision-making in the absence of reward and separated from choice difficulty-allowing us to investigate the brain's role in effort valuation, independent of these other factors. Participants exhibited subjectivity in their decision-making, displaying increased sensitivity to changes in subjective effort as objective effort levels increased. Analysis of blood-oxygenation-level dependent activity revealed that the ventromedial prefrontal cortex (vmPFC) encoded the subjective valuation of prospective effort, and ACC activity was best described by choice difficulty. These results provide insight into the processes responsible for decision-making regarding effort, partly dissociating the roles of vmPFC and ACC in prospective valuation of effort and choice difficulty.

Incorporating Certainty and Attribute Non-Attendance in Choice Experiments: An Application to Valuation of Coastal Habitat

Due to their non-market characteristics, coastal habitats are often valued using stated preference methods, such as discrete choice experiments. An extensive literature cautions the confounding influence of both choice complexity and hypothetical bias in these types of welfare analyses. In response, researchers have developed a variety of ex post controls to reduce the influence of potential biases. This study utilizes a discrete choice experiment with the goal of measuring residents and visitors’ willingness-to-pay for estuarine habitat and access in the Sarasota Bay Estuary watershed. Our application examines model calibration to address two important issues in the empirical literature: 1) reported choice certainty and 2) stated attribute non-attendance. We examine choice certainty by focusing on recalibrating data to account for uncertain choices by jointly modeling choice and certainty. We also account for stated attribute attendance and non-attendance and compare results with more standard models. Our results indicate that accounting for attribute non-attendance and choice certainty have minor effects on coefficient estimates, generally leading to a small reduction in the marginal utilities for coastal habitat and a small increase in the marginal utility of income. The more apparent influence of these approaches comes via smaller standard errors. These approaches result in slightly lower estimates of marginal-willingness-to-pay (WTP), suggesting that accounting for uncertainty may ameliorate hypothetical bias but they also provide more precise estimates. Results from our preferred model indicate respondents are willing to pay between $1.50 to $2.06 for an additional acre of wetlands, $0.54 to $4.55 for an acre of oyster habitat, $0.57 to $2.28 for an acre of seagrass, $0.09 to $ 0.32 for an additional artificial reef dome, and $ 2.87 to $ 7.18 for an additional ecological park. Uncertain respondents exhibit WTP that are not statistically different than $0.

A tractable approach to compare the hedonic and discrete choice frameworks

The two primary approaches to estimate marginal willingness-to-pay (MWTP) are hedonic (Rosen, 1974) and discrete choice (McFadden, 1974). While both approaches rely on revealed preference methods to estimate MWTP, the primitives underlying both models are different, making it difficult to compare them. This paper establishes the assumptions needed to develop a tractable framework to compare both approaches. I begin with a discrete choice model and show how to derive the gradient of the equilibrium price function implicitly. I then incorporate Rosen’s insight that the price gradient is equal to the MWTP of the marginal individual whose indifference curve is tangent to the price function in equilibrium. However, with discrete choices, some individuals may be inframarginal and their indifference curves will not be tangent to the price function. The analytical mapping I derive formalizes this intuition and shows that the price gradient depends on weighted averages of marginal utilities where higher weights are assigned to individuals whose choice probabilities indicate more uncertain choices (marginal individuals). As this choice becomes more certain, the weights start to decrease. This result shows how choice probabilities and other moments of choice data can be used to distinguish marginal versus inframarginal individuals.

Amphetamine primes enhanced motivation toward uncertain choices in rats with genetic alcohol preference.

Comorbidity with gambling disorder (GD) and alcohol use disorder (AUD) is well documented. The purpose of our study was to examine the influence of genetic alcohol drinking tendency on reward-guided decision making behavior of rats and the impact of dopamine releaser D-amphetamine on this behavior. In this study, Alko alcohol (AA) and Wistar rats went through long periods of operant lever pressing training where the task was to choose the profitable of two options. The lever choices were guided by different-sized sucrose rewards (one or three pellets), and the probability of gaining the larger reward was slowly changed to a level where choosing the smaller reward would be the most profitable in the long run. After training, rats were injected (s.c.) with dopamine releaser D-amphetamine (0.3, 1.0mg/kg) to study the impact of rapid dopamine release on this learned decision making behavior. Administration of D-amphetamine promoted unprofitable decision making of AA rats more robustly when compared to Wistar rats. At the same time, D-amphetamine reduced lever pressing responses. Interestingly, we found that this reduction in lever pressing was significantly greater in Wistar rats than in AA rats and it was not linked to motivation to consume sucrose. Our results indicate that conditioning to the lever pressing in uncertain environments is more pronounced in AA than in Wistar rats and indicate that the reinforcing effects of a gambling-like environment act as a stronger conditioning factor for rats that exhibit a genetic tendency for high alcohol drinking.

Decision-Making Process Regarding Fertility Among Reproductive-Age Women With Cancer in Taiwan.

Previous studies indicate that women with cancer experience infertility after cancer-related treatment. With the rapid progress in fertility science, women face diverse and uncertain choices regarding pregnancy. The aim of this study is to understand the decision-making process regarding fertility choices among reproductive-age women with cancer in Taiwan. Grounded theory methodology guided data collection using in-depth interviews with 18 women diagnosed and treated for cancer. Verbatim transcriptions were analyzed using constant comparative analysis and open, axial, and selective coding. The core category that describes the decision-making process regarding fertility among reproductive-age women with cancer is "searching for balance in life and creating value in life." The decision process was divided into 3 phases: needing to have children before treatment, struggling with self-living during cancer treatment, and returning to life after treatment. The style of cancer participants' decision making in pregnancy was divided into 3 patterns: action taking, hesitation, and persistence. Decision making regarding fertility among women with cancer was affected by the need for children before treatment and their experience during treatment. Health providers should be aware of and understand the needs of women with cancer to balance their need for children with their perception of their cancer prognosis and its effects on fertility, and help them with pregnancy planning if desired.

A lagrangian-based special purpose procedure to solve a two-stage single constrained linear fractional programming problem with bounded variables

This paper presents a general two-stage single constrained linear fractional programming problem with bounded variables. The objective function is a fraction of the linear functions and the decision variables are finitely bounded. This model consists of a constraint under right hand side uncertainties. The problem is considerably complex and difficult as the number of uncertain choices is high. A graphical-based method was used sequentially along with interpolations of the corresponding Lagrange Multiplier to solve the problem. Moreover, the special purpose procedure coded as an Excel - macro can increase the efficiency of large size problem-solving whereas the Excel Solver Gurobi cannot handle such a problem.

Uncertain expected utility function and its risk premium

Other than traditional decision theory, this paper employs uncertainty theory to handle indeterminacy. Uncertain variables are used to represent uncertain choices. Uncertain expected utility function is defined as an increasing function of uncertain choices. Several mathematical properties of the uncertain expected utility functions are derived using inverse uncertainty distributions. In order to compare two different choices, the first order dominance and second order dominance via uncertain expected utility functions are introduced. We also investigate risk aversion attitude and risk premium. Finally, the relationship between risk premium and risk averse attitude is investigated.

The effects of the previous outcome on probabilistic choice in rats.

This study examined the effects of previous outcomes on subsequent choices in a probabilistic-choice task. Twenty-four rats were trained to choose between a certain outcome (1 or 3 pellets) versus an uncertain outcome (3 or 9 pellets), delivered with a probability of .1, .33, .67, and .9 in different phases. Uncertain outcome choices increased with the probability of uncertain food. Additionally, uncertain choices increased with the probability of uncertain food following both certain-choice outcomes and unrewarded uncertain choices. However, following uncertain-choice food outcomes, there was a tendency to choose the uncertain outcome in all cases, indicating that the rats continued to "gamble" after successful uncertain choices, regardless of the overall probability or magnitude of food. A subsequent manipulation, in which the probability of uncertain food varied within each session as a function of the previous uncertain outcome, examined how the previous outcome and probability of uncertain food affected choice in a dynamic environment. Uncertain-choice behavior increased with the probability of uncertain food. The rats exhibited increased sensitivity to probability changes and a greater degree of win-stay/lose-shift behavior than in the static phase. Simulations of two sequential choice models were performed to explore the possible mechanisms of reward value computations. The simulation results supported an exponentially decaying value function that updated as a function of trial (rather than time). These results emphasize the importance of analyzing global and local factors in choice behavior and suggest avenues for the future development of sequential-choice models.

Choice of Action: Using Data to Make Instructional Decisions in Kindergarten

This article focuses on the different—and at times uncertain—choices a kindergarten teacher makes when selecting and interpreting assessment data and designing instruction intended to help students develop the skills and dispositions for later reading. By examining assessment data and engaging in reflective practice, the teacher is able to anticipate students' literacy needs and design instruction that builds upon their strengths.A close‐up look at a shared writing lesson is used to illustrate the connections the teacher makes between her interpretation of data and the instruction she delivers. The article concludes with a brief examination of the complexities inherent in data analysis.

Subjective Probability and Information: A Constructive Perspective

SYNOPTIC ABSTRACTSubjective probability was developed for representing uncertainty in real world situations. Traditionally, it is viewed either as an intuitive notion that can be measured directly, or as a by-product of an agent's preferences among uncertain choices. This paper argues that subjective probability is often not a primitive notion, but a construct that must be built using the available information. In this view, probability represents the extent to which available information supports the occurrence of an event. Approaches to probability construction are discussed, and one of the models is illustrated with examples. The discussion also sheds some light on the sources of difficulties in the measurement of subjective probability.

Reformulating decision theory using fuzzy set theory and Shafer's theory of evidence

Utilities and probabilities in decision theory are usually assessed by asking individuals to indicate their preferences between various uncertain choices. In this paper, we argue that (1) The utility of a consequence can be assessed as the membership function of the consequence in the fuzzy set ‘ satisfactory’. (2) The probability of an event, instead of being directly assessed, should be inferred from the evidence associated with that event. The degree of evidence is quantified using Shaferian basic probability assignments. In addition, we use the Heisenberg Uncertainty Principle to argue for a change in one of the technical assumptions underlying decision theory. As a result of this change, some kinds of evidence will be observable in certain experiments but unobservable in others. Since probabilities are defined over the potential outcomes of an experiment, they will only be defined over some, but not all, the evidence. As a result, the probabilities associated with different experiments could be inconsistent. This formulation emphasizes the importance of new distinctions (and not just new information) in updating probabilities. We argue that this formulation addresses many of the observed empirical deviations between decision theory and experiment. It also addresses the anomalies of quantum physics. We close with a brief discussion of directions for further research.

A rational framework for decision making by the National Institute For Clinical Excellence (NICE)

Regulatory and reimbursement authorities face uncertain choices when considering the adoption of health-care technologies. In this Viewpoint, we present an analytic framework that separates the issue of whether a technology should be adopted on the basis of existing evidence from whether more research should be demanded to support future decisions. We show the application of this framework to the assessment of heath-care technologies using a published analysis of a new drug treatment for Alzheimer's disease. The results of the analysis show that the amount and type of evidence required to support the adoption of a health technology will differ substantially between technologies with different characteristics. Additionally, the analysis can be used to aid the efficient design of research. We discuss the implications of adoption of this new framework for regulatory and reimbursement decisions.