Human Choice Behavior Research Articles

Investigating day-to-day route choices based on multi-scenario laboratory experiments, Part II: Route-dependent attraction-based stochastic process model

Laboratory experiments are one of the important means used to investigate travel choice behavior under strategic uncertainty. Many experiment-based studies have shown that the Nash equilibrium can predict aggregated route choices, while the fluctuations, whose mechanisms are still unclear, continue to exist until the end. To understand the fluctuations, this paper proposes a route-dependent attraction-based stochastic process model, which shares exactly the same behavioral foundation introduced in Part I of the study (Qi et al., 2023), i.e., route-dependent inertia and route-dependent preference. The model predictions are carefully compared with the experimental observations obtained from the congestible parallel-route laboratory experiments containing 312 subjects and eight decision-making scenarios (Qi et al., 2023). The results show that the proposed stochastic process model can precisely reproduce the random oscillations both in terms of flow switching and route flow evolution. Subsequently, an approximated model is developed to enhance the efficiency in evaluating the equilibrium distribution, providing a practical tool to evaluate the impacts of transportation policies in both long- and short-term runs. To the best of our knowledge, this paper is the first attempt to model and explain experimental phenomena by introducing stochastic process theories, as well as a successful example of applying experimental economics methodology to improve our understanding of human travel choice behavior.

Using recurrent neural network to estimate irreducible stochasticity in human choice behavior.

Theoretical computational models are widely used to describe latent cognitive processes. However, these models do not equally explain data across participants, with some individuals showing a bigger predictive gap than others. In the current study, we examined the use of theory-independent models, specifically recurrent neural networks (RNNs), to classify the source of a predictive gap in the observed data of a single individual. This approach aims to identify whether the low predictability of behavioral data is mainly due to noisy decision-making or misspecification of the theoretical model. First, we used computer simulation in the context of reinforcement learning to demonstrate that RNNs can be used to identify model misspecification in simulated agents with varying degrees of behavioral noise. Specifically, both prediction performance and the number of RNN training epochs (i.e., the point of early stopping) can be used to estimate the amount of stochasticity in the data. Second, we applied our approach to an empirical dataset where the actions of low IQ participants, compared with high IQ participants, showed lower predictability by a well-known theoretical model (i.e., Daw's hybrid model for the two-step task). Both the predictive gap and the point of early stopping of the RNN suggested that model misspecification is similar across individuals. This led us to a provisional conclusion that low IQ subjects are mostly noisier compared to their high IQ peers, rather than being more misspecified by the theoretical model. We discuss the implications and limitations of this approach, considering the growing literature in both theoretical and data-driven computational modeling in decision-making science.

Human Behavior in Suboptimal Choice Tasks: Defining Optimality.

Suboptimal choice behavior, or behavior that leads to a loss of resources over time, has been observed in a laboratory setting from multiple species. A procedure commonly used to capture this effect involves presenting two alternatives during choice trials, one of which is optimal whereas the other is suboptimal. The optimal alternative yields reinforcement more often than the suboptimal alternative, but often does not produce signals that indicate whether reinforcement will occur. The suboptimal alternative produces less reinforcement than the optimal alternative but may include reinforcement-predictive stimuli that indicate to the organism whether reinforcement will occur. This procedural framework has consistently produced a preference for the suboptimal alternative in pigeons and, to a lesser extent, rats. However, human participants have demonstrated preference for the optimal alternative. Following a review of past suboptimal choice research, we applied the reinforcement-derived definition of optimality to two sets of our previously published human data. We found that under multiple conditions, human choice behavior was consistent with what was predicted by the proportion of obtained reinforcement, thus supporting that the behavior was optimal. However, we found that participants in two conditions chose the suboptimal alternative more than expected. This finding could be considered as a demonstration of suboptimal choice in humans. We propose that comparing choice behavior to what past obtained reinforcement outcomes would predict might be a more accurate view of whether patterns of choice are within the parameters of this task.

Surprise-minimization as a solution to the structural credit assignment problem.

The structural credit assignment problem arises when the causal structure between actions and subsequent outcomes is hidden from direct observation. To solve this problem and enable goal-directed behavior, an agent has to infer structure and form a representation thereof. In the scope of this study, we investigate a possible solution in the human brain. We recorded behavioral and electrophysiological data from human participants in a novel variant of the bandit task, where multiple actions lead to multiple outcomes. Crucially, the mapping between actions and outcomes was hidden and not instructed to the participants. Human choice behavior revealed clear hallmarks of credit assignment and learning. Moreover, a computational model which formalizes action selection as the competition between multiple representations of the hidden structure was fit to account for participants data. Starting in a state of uncertainty about the correct representation, the central mechanism of this model is the arbitration of action control towards the representation which minimizes surprise about outcomes. Crucially, single-trial latent-variable analysis reveals that the neural patterns clearly support central quantitative predictions of this surprise minimization model. The results suggest that neural activity is not only related to reinforcement learning under correct as well as incorrect task representations but also reflects central mechanisms of credit assignment and behavioral arbitration.

Utility in Time Description in Priority Best–Worst Discrete Choice Models: An Empirical Evaluation Using Flynn’s Data

Discrete choice models (DCMs) are applied in many fields and in the statistical modelling of consumer behavior. This paper focuses on a form of choice experiment, best–worst scaling in discrete choice experiments (DCEs), and the transition probability of a choice of a consumer over time. The analysis was conducted by using simulated data (choice pairs) based on data from Flynn’s (2007) ‘Quality of Life Experiment’. Most of the traditional approaches assume the choice alternatives are mutually exclusive over time, which is a questionable assumption. We introduced a new copula-based model (CO-CUB) for the transition probability, which can handle the dependent structure of best–worst choices while applying a very practical constraint. We used a conditional logit model to calculate the utility at consecutive time points and spread it to future time points under dynamic programming. We suggest that the CO-CUB transition probability algorithm is a novel way to analyze and predict choices in future time points by expressing human choice behavior. The numerical results inform decision making, help formulate strategy and learning algorithms under dynamic utility in time for best–worst DCEs.

Harnessing the flexibility of neural networks to predict dynamic theoretical parameters underlying human choice behavior.

Reinforcement learning (RL) models are used extensively to study human behavior. These rely on normative models of behavior and stress interpretability over predictive capabilities. More recently, neural network models have emerged as a descriptive modeling paradigm that is capable of high predictive power yet with limited interpretability. Here, we seek to augment the expressiveness of theoretical RL models with the high flexibility and predictive power of neural networks. We introduce a novel framework, which we term theoretical-RNN (t-RNN), whereby a recurrent neural network is trained to predict trial-by-trial behavior and to infer theoretical RL parameters using artificial data of RL agents performing a two-armed bandit task. In three studies, we then examined the use of our approach to dynamically predict unseen behavior along with time-varying theoretical RL parameters. We first validate our approach using synthetic data with known RL parameters. Next, as a proof-of-concept, we applied our framework to two independent datasets of humans performing the same task. In the first dataset, we describe differences in theoretical RL parameters dynamic among clinical psychiatric vs. healthy controls. In the second dataset, we show that the exploration strategies of humans varied dynamically in response to task phase and difficulty. For all analyses, we found better performance in the prediction of actions for t-RNN compared to the stationary maximum-likelihood RL method. We discuss the use of neural networks to facilitate the estimation of latent RL parameters underlying choice behavior.

Can competing demands affect pro-environmental behaviour: a study of the impact of exposure to partly related sequential experiments

The study of human behaviour is central to the development of appropriate policies for sustainability. We argue that mathematical models of human choice behaviour may produce biased results if they fail to account for the possibility of spillover effects, in particular the possibility that individual behaviour may change as a result of interventions along with competing demands (multiple demands), such as in the sequential exposure to partly related choice contexts. Using a sample of 751 individuals and a carefully constructed experiment, we develop mathematical models that jointly explain the choice between different pro-environmental actions and the willingness to donate money for environmental causes, and at the same time, allow us to test the indirect effect of exposure to multiple demands. We find that the strength of preferences for behavioural changes leading to greater CO2 reductions is (causally) shaped by participants previously considering other similar behavioural changes. The kind of spillover effects we find are relatively complex and often subtle, and thus warrant further replication studies. Care was taken to account for variation of tastes, formatting, order and learning effects, thus reducing the risk of the spillover-related results being influenced by differences across individuals in environmental preferences. Our study demonstrates the existence of a specific type of spillover effects, namely how prior exposure to related choice contexts may affect behaviour in subsequent settings and showcased the effectiveness of discrete choice models to test for it. Given our results, we believe that spillover effects need to be taken into account in the broader choice modelling literature, and at the same time we showcase a useful experimental framework for environmental psychologists and economists.

Route Choice-Based Socio-Technical Macroscopic Traffic Model

Human route choice is undeniably one of the key contributing factors towards traffic dynamics. However, most existing macroscopic traffic models are typically concerned with driving behavior and do not incorporate human route choice behavior models in their formulation. In this paper, we propose a socio-technical macroscopic traffic model that characterizes the traffic states using human route choice attributes. Essentially, such model provides a framework for capturing the Cyber-Physical-Social coupling in smart transportation systems. To derive this model, we first use Cumulative Prospect Theory (CPT) to model the human passengers’ route choice under bounded rationality. These choices are assumed to be influenced by traffic alerts and other incomplete traffic information. Next, we assume that the vehicles are operating under a non-cooperative cruise control scenario. Accordingly, human route choice segregates the traffic into multiple classes where each class corresponds to a specific route between an origin-destination pair. Thereafter, we derive a Mean Field Game (MFG) limit of this non-cooperative game to obtain a macroscopic model which embeds the human route choice attribute. Finally, we analyze the mathematical characteristics of the proposed model, present simulation studies to illustrate the model behavior and show comparison with existing traffic model.

Narrative framing may increase human suboptimal choice behavior.

Under certain conditions, multiple nonhuman species have been observed engaging in choice behavior that resulted in less food earned when compared to the amount of food that was available to be earned over the course of a session. This phenomenon is particularly strong in pigeons, but has also been observed in rats and nonhuman primates. Conversely, human participants have demonstrated a propensity to choose more optimally. However, human participants do not exclusively choose the alternative associated with more reinforcement. Framing a task in a real-world narrative has been effective in improving problem-solving on other tasks such as the Wason Four-Card problem. The present study gave human participants a choice task with either abstract stimuli or with a real-world narrative. In addition, participants were given terminal stimuli that were either predictive or unpredictive of reinforcement. Thus, participants were assigned to one of four conditions: Abstract Predictive, Abstract Unpredictive, Narrative Predictive, or Narrative Unpredictive. In contrast to the improved performance on the Wason Four-Card task, the current study found no evidence that the addition of a real-world narrative improved optimal choice performance. Rather, it may have interfered with optimal choice selection in that participants who received the narrative and unpredictive terminal stimuli were at chance performance at the end of the experimental session. Conversely, participants in the Abstract Unpredictive, Abstract Predictive, and Narrative Predictive conditions all demonstrated a preference for the optimal alternative. Possible mechanisms for these findings and future directions are discussed.

Timing and delay discounting in attention-deficit/hyperactivity disorder: A translational approach.

Attention-deficit/hyperactivity disorder (ADHD) is a common neurodevelopmental disorder that often presents with abnormal time perception and increased impulsive choice behavior. The spontaneously hypertensive rat (SHR) is the most widely used preclinical model of the ADHD-Combined and ADHD-Hyperactive/Impulsive subtypes of the disorder. However, when testing the spontaneously hypertensive rat from Charles River (SHR/NCrl) on timing and impulsive choice tasks, the appropriate control strain is not clear, and it is possible that one of the possible control strains, the Wistar Kyoto from Charles River (WKY/NCrl), is an appropriate model for ADHD-Predominately Inattentive. Our goals were to test the SHR/NCrl, WKY/NCrl, and Wistar (WI; the progenitor strain for the SHR/NCrl and WKY/NCrl) strains on time perception and impulsive choice tasks to assess the validity of SHR/NCrl and WKY/NCrl as models of ADHD, and the validity of the WI strain as a control. We also sought to assess impulsive choice behavior in humans diagnosed with the three subtypes of ADHD and compare them with our findings from the preclinical models. We found SHR/NCrl rats timed faster and were more impulsive than WKY/NCrl and WI rats, and human participants diagnosed with ADHD were more impulsive compared to controls, but there were no differences between the three ADHD subtypes.

A naive justification of hyperbolic discounting from mental algebraic operations and functional analysis

<abstract><sec><title>Background</title><p>Intertemporal decision-making, which involves making choices between outcomes at different time points, is a fundamental aspect of human behavior. Understanding the underlying mental processes is vital for comprehending the complexities of human decision-making and choice behavior.</p> </sec> <sec><title>Objective</title><p>The main objective of this study is to investigate the interplay of mental processes, specifically cognitive evaluation, subjective valuation, and comparison, in the context of intertemporal decision-making, with a specific focus on understanding the discounting process.</p> </sec> <sec><title>Methodology</title><p>Development of a mathematical representation of the discounting process that incorporates the mental processes associated with intertemporal decision-making.</p> </sec> <sec><title>Result</title><p>Our findings indicate that hyperbolic discounting aligns well with the cognitive processes underlying intertemporal decision-making. Subsequent research will employ qualitative questionnaires to establish the discount function relevant to specific groups, thereby enhancing our comprehension of the discounting process within intertemporal decision-making.</p> </sec></abstract>

Disentangling the roles of dopamine and noradrenaline in the exploration-exploitation tradeoff during human decision-making

Balancing the exploration of new options and the exploitation of known options is a fundamental challenge in decision-making, yet the mechanisms involved in this balance are not fully understood. Here, we aimed to elucidate the distinct roles of dopamine and noradrenaline in the exploration-exploitation tradeoff during human choice. To this end, we used a double-blind, placebo-controlled design in which participants received either a placebo, 400 mg of the D2/D3 receptor antagonist amisulpride, or 40 mg of the β-adrenergic receptor antagonist propranolol before they completed a virtual patch-foraging task probing exploration and exploitation. We systematically varied the rewards associated with choice options, the rate by which rewards decreased over time, and the opportunity costs it took to switch to the next option to disentangle the contributions of dopamine and noradrenaline to specific choice aspects. Our data show that amisulpride increased the sensitivity to all of these three critical choice features, whereas propranolol was associated with a reduced tendency to use value information. Our findings provide novel insights into the specific roles of dopamine and noradrenaline in the regulation of human choice behavior, suggesting a critical involvement of dopamine in directed exploration and a role of noradrenaline in more random exploration.

From Bayes-optimal to heuristic decision-making in a two-alternative forced choice task with an information-theoretic bounded rationality model.

Bayes optimal and heuristic decision-making schemes are often considered fundamentally opposed to each other as a framework for studying human choice behavior, although recently it has been proposed that bounded rationality may provide a natural bridge between the two when varying information-processing resources. Here, we investigate a two-alternative forced choice task with varying time constraints, where subjects have to assign multi-component symbolic patterns to one of two stimulus classes. As expected, we find that subjects' response behavior becomes more imprecise with more time pressure. However, we also see that their response behavior changes qualitatively. By regressing subjects' decision weights, we find that decisions allowing for plenty of decision time rely on weighing multiple stimulus features, whereas decisions under high time pressure are made mostly based on a single feature. While the first response pattern is in line with a Bayes-optimal decision strategy, the latter could be considered as an instantiation of heuristic decision-making with cue discounting. When fitting a bounded rational decision model with multiple feature channels and varying information-processing capacity to subjects' responses, we find that the model is able to capture subjects' behavioral change. The model successfully reflects the simplicity of heuristics as well as the efficiency of optimal decision making, thus acting as a bridge between the two approaches.

Efficiently irrational: deciphering the riddle of human choice.

For the past half-century, cognitive and social scientists have struggled with the irrationalities of human choice behavior; people consistently make choices that are logically inconsistent. Is human choice behavior evolutionarily adaptive or is it an inefficient patchwork of competing mechanisms? In this review, I present an interdisciplinary synthesis arguing for a novel interpretation: choice is efficiently irrational. Connecting findings across disciplines suggests that observed choice behavior reflects a precise optimization of the trade-off between the costs of increasing the precision of the choice mechanism and the declining benefits that come as precision increases. Under these constraints, a rationally imprecise strategy emerges that works toward optimal efficiency rather than toward optimal rationality. This approach rationalizes many of the puzzling inconsistencies of human choice behavior, explaining why these inconsistencies arise as an optimizing solution in biological choosers.

Intracerebral mechanisms explaining the impact of incidental feedback on mood state and risky choice.

Identifying factors whose fluctuations are associated with choice inconsistency is a major issue for rational decision theory. Here, we investigated the neuro-computational mechanisms through which mood fluctuations may bias human choice behavior. Intracerebral EEG data were collected in a large group of subjects (n=30) while they were performing interleaved quiz and choice tasks that were designed to examine how a series of unrelated feedbacks affect decisions between safe and risky options. Neural baseline activity preceding choice onset was confronted first to mood level, estimated by a computational model integrating the feedbacks received in the quiz task, and then to the weighting of option attributes, in a computational model predicting risk attitude in the choice task. Results showed that (1) elevated broadband gamma activity (BGA) in the ventromedial prefrontal cortex (vmPFC) and dorsal anterior insula (daIns) was respectively signaling periods of high and low mood, (2) increased vmPFC and daIns BGA respectively promoted and tempered risk taking by overweighting gain vs. loss prospects. Thus, incidental feedbacks induce brain states that correspond to different moods and bias the evaluation of risky options. More generally, these findings might explain why people experiencing positive (or negative) outcome in some part of their life tend to expect success (or failure) in any other.

Value, Confidence, Deliberation: A Functional Partition of the Medial Prefrontal Cortex Demonstrated across Rating and Choice Tasks.

Deciding about courses of action involves minimizing costs and maximizing benefits. Decision neuroscience studies have implicated both the ventral and dorsal medial PFC (vmPFC and dmPFC) in signaling goal value and action cost, but the precise functional role of these regions is still a matter of debate. Here, we suggest a more general functional partition that applies not only to decisions but also to judgments about goal value (expected reward) and action cost (expected effort). In this conceptual framework, cognitive representations related to options (reward value and effort cost) are dissociated from metacognitive representations (confidence and deliberation) related to solving the task (providing a judgment or making a choice). We used an original approach aimed at identifying consistencies across several preference tasks, from likeability ratings to binary decisions involving both attribute integration and option comparison. fMRI results in human male and female participants confirmed the vmPFC as a generic valuation system, its activity increasing with reward value and decreasing with effort cost. In contrast, more dorsal regions were not concerned with the valuation of options but with metacognitive variables, confidence being reflected in mPFC activity and deliberation time in dmPFC activity. Thus, there was a dissociation between the effort attached to choice options (represented in the vmPFC) and the effort invested in deliberation (represented in the dmPFC), the latter being expressed in pupil dilation. More generally, assessing commonalities across preference tasks might help in reaching a unified view of the neural mechanisms underlying the cost/benefit tradeoffs that drive human behavior.SIGNIFICANCE STATEMENT Decision neuroscience studies have implicated the medial PFC in forming the cognitive representations that drive human choice behavior. However, different studies using different tasks have suggested somewhat inconsistent links between precise computational variables and specific brain regions. Here, we use fMRI to demonstrate a robust functional partition of the medial PFC that generalizes across tasks involving an estimation of goal value and/or action cost to provide a judgment or make a choice. This general functional partition makes a critical dissociation between neural representations of decisional factors (the expected costs and benefits attached to a given option) and metacognitive estimates (confidence in the judgment or choice, and effort invested in the deliberation process).

Explicit knowledge of task structure is a primary determinant of human model-based action.

Explicit information obtained through instruction profoundly shapes human choice behaviour. However, this has been studied in computationally simple tasks, and it is unknown how model-based and model-free systems, respectively generating goal-directed and habitual actions, are affected by the absence or presence of instructions. We assessed behaviour in a variant of a computationally more complex decision-making task, before and after providing information about task structure, both in healthy volunteers and in individuals suffering from obsessive-compulsive or other disorders. Initial behaviour was model-free, with rewards directly reinforcing preceding actions. Model-based control, employing predictions of states resulting from each action, emerged with experience in a minority of participants, and less in those with obsessive-compulsive disorder. Providing task structure information strongly increased model-based control, similarly across all groups. Thus, in humans, explicit task structural knowledge is a primary determinant of model-based reinforcement learning and is most readily acquired from instruction rather than experience.

Dynamic modulation of inequality aversion in human interpersonal negotiations

Negotiating with others about how finite resources should be distributed is an important aspect of human social life. However, little is known about mechanisms underlying human social-interactive decision-making in gradually evolving environments. Here, we report results from an iterative Ultimatum Game (UG), in which the proposer’s facial emotions and offer amounts were sampled probabilistically based on the participant’s decisions. Our model-free results confirm the prediction that both the proposer’s facial emotions and the offer amount should influence acceptance rates. Model-based analyses extend these findings, indicating that participants’ decisions in the UG are guided by aversion to inequality. We highlight that the proposer’s facial affective reactions to participant decisions dynamically modulate how human decision-makers perceive self–other inequality, relaxing its otherwise negative influence on decision values. This cognitive model underlies how offers initially rejected can gradually become more acceptable under increasing affective load (predictive accuracy ~86%). Furthermore, modelling human choice behaviour isolated the role of the central arousal systems, assessed by measuring pupil size. We demonstrate that pupil-linked central arousal systems selectively encode a key component of subjective decision values: the magnitude of self–other inequality. Taken together, our results demonstrate that, under affective influence, aversion to inequality is a malleable cognitive process.

An ontology of decision models.

Decision models are essential theoretical tools in the study of choice behavior, but there is little consensus about the best model for describing choice, with different fields and different research programs favoring their own idiosyncratic sets of models. Even within a given field, decision models are seldom studied alongside each other, and insights obtained using 1 model are not typically generalized to others. We present the results of a large-scale computational analysis that uses landscaping techniques to generate a representational structure for describing decision models. Our analysis includes 89 prominent models of risky and intertemporal choice, and results in an ontology of decision models, interpretable in terms of model spaces, clusters, hierarchies, and graphs. We use this ontology to measure the properties of individual models and quantify the relationships between different models. Our results show how decades of quantitative research on human choice behavior can be synthesized within a single representational framework. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Goal Side Selection of Penalty Shots in Soccer: A Laboratory Study and Analyses of Men's World Cup Shoot-Outs.

Penalty kicks in soccer provide a unique scenario in which to examine human choice behavior under competitive conditions. Here, we report two studies examining the tendency for soccer kickers to select the goal side with the largest area to the left or right of the goalkeeper’s veridical midline, when the goalkeeper stands marginally off-center. In Study I participants viewed realistic images of a soccer goal and goalkeeper with instructions to choose the left or right side of the goalmouth to best score a goal. We systematically displaced the goalkeeper’s position along the goal line; and, to simulate changes in the kicker’s viewing position, we systematically displaced the lateral position of the goalmouth in each image. While, overall, participants tended to choose the left over the right goal side, this preference was modulated by the goalkeeper’s position relative to the center of the goal and jointly on the lateral position of the goalmouth relative to the participants’ body midline. In Study II we analyzed 100 penalty shots from men’s world cup shoot-outs between the years 1982 to 2018. Again, we found a small tendency for kickers to aim the ball to the left goal side, but with barely any modulating effect of changes in the goalkeeper’s position and no effect of changes in the kicker’s position. In contrast to earlier claims that a goalkeeper may benefit by standing marginally to the left or right of the center of the goal to influence the direction of the kicker’s shot, our findings suggest that this is probably not a good strategy in elite football competitions.