Stochastic Choice Research Articles

Competing Bandits: The Perils of Exploration Under Competition

Most online platforms learn from interactions with users, and engage in exploration : making potentially suboptimal choices in order to acquire new information. We study the interplay between exploration and competition : how such platforms balance the exploration for learning and competition for users. We consider a stylized duopoly in which two firms face the same multi-armed bandit problem. Users arrive one by one and choose between the two firms, so that each firm makes progress on its bandit problem only if it is chosen. We study whether competition incentivizes the adoption of better algorithms. We find that stark competition disincentivizes exploration, leading to low welfare. However, weaker competition incentivizes better exploration algorithms and increases welfare. We investigate two channels for weakening the competition: stochastic user choice models and a first-mover advantage. Our findings speak to the competition-innovation relationship and the first-mover advantage in the digital economy.

Stochastic choice and imperfect judgments of line lengths: What is hiding in the noise?

Stochastic choice and imperfect judgments of line lengths: What is hiding in the noise?

Social preferences and the variability of conditional cooperation

AbstractWe experimentally examine how incentives affect conditional cooperation (i.e., cooperating in response to cooperation and defecting in response to defection) in social dilemmas. In our first study, subjects play eight Sequential Prisoner’s Dilemma games with varying payoffs. We elicit second mover strategies and find that most second movers conditionally cooperate in some games and free ride in others. The rate of conditional cooperation is higher when the own gain from defecting is lower and when the loss imposed on the first mover by defecting is higher. This pattern is consistent with both social preference models and stochastic choice models. In a second study subjects play 64 social dilemma games, and we jointly estimate noise and social preference parameters at the individual level. Most of our subjects place significantly positive weight on others’ payoffs, supporting the underlying role of social preferences in conditional cooperation. Our results suggest that conditional cooperation is not a fixed trait but rather a symptom of the interaction between game incentives and underlying social preferences.

Consistent Depth of Reasoning in Level-k Models

Level-k models often assume that individuals employ a fixed depth of reasoning across different games. We study this assumption by having subjects make choices in five classes of games chosen to identify inconsistent depth of reasoning. We demonstrate that depth of reasoning is pervasively inconsistent, changing both within and between classes of games. We show that this cannot easily be explained by factors such as subject confusion, changing beliefs about others’ depth of reasoning, stochastic choice, model misspecification, changing incentives, or low cognitive ability. We develop a simple model incorporating ambiguity aversion that predicts inconsistent depth of reasoning. (JEL C57, C90, D81, D83, D91)

Revisiting Risky Money

Risk was first incorporated into monetary aggregation over thirty-five years ago,using a stochastic version of the workhorse money-in-the-utility-function model.Nevertheless, the mathematical foundations of this stochastic model remain shaky.To firm the foundations, this paper employs a slightly richer probability conceptthan standard Borel-measurability, which enables me to prove the existence of awell-behaved solution and to derive stochastic Euler equations. This measurabilityapproach is long-established albeit less common in economics, possibly because the derivation of stochastic Euler equations is new. Importantly, the problem’s economics are not restricted by the approach. Consequently, the results provide firm footing for the growing monetary aggregation under risk literature, which integrates monetary and finance theory. As crypto-currencies and stable coins garner more attention, solidifying the foundations of risky money becomes more critical. The method also supports deriving stochastic Euler equations for any dynamic economics problem that features contemporaneous uncertainty about prices, including asset pricing models like CAPM and stochastic consumer choice models.

Understanding explore-exploit dynamics in child development: current insights and future directions

Examining children's decisions to explore or exploit the environment provides a window into their developing metacognition and reflection capacities. Reinforcement learning, characterized by the balance between exploring new options (exploration) and utilizing known ones (exploitation), is central to this discussion. Children initially exhibit broad and intensive exploration, which gradually shifts toward exploitation as they grow. We review major theories and empirical findings, highlighting two main exploration strategies: random and directed. The former involves stochastic choices without considering information or rewards, while the latter is driven by reducing uncertainty for information gain. Behavioral tasks such as n-armed bandit, horizon, and patch foraging tasks are used to study these strategies. Findings on the n-armed bandit and horizon tasks showed mixed results on whether random exploration decreases over time. Directed exploration consistently decreases with age, but its emergence depends on task difficulty. In patch-foraging tasks, adults tend to overexploit (staying too long in one patch) and children overexplore (leaving too early), whereas adolescents display the most optimal balance. The paper also addresses open questions regarding the mechanisms supporting early exploration and the application of these strategies in real-life contexts like persistence. Future research should further investigate the relation between cognitive control, such as executive function and metacognition, and explore-exploit strategies, and examine their practical implications for adaptive learning and decision-making in children.

Loading conditions for self‐organization in the BML model with stochastic direction choice

A dynamical system is considered such that, in this system, particles move on a toroidal lattice of the dimension according to a version of the rule of particle movement in Biham–Middleton–Levine traffic model. We introduce a stochastic case with direction choice for particles. Particles of the first type move along rows, and the particles of the second type move along columns. The goal is to find conditions of self‐organization system for any lattice dimension. We have proved that the BML model as a dynamical system is a special case of Buslaev nets. This equivalence allows us to use of Buslaev net analysis techniques to investigate the BML model. In Buslaev nets conception, the self‐organization property of the system corresponds to the existence of velocity single point spectrum equal to 1. In the paper, we consider the model version when one notable aspect is that a particle may change its type. Exactly, we assume a constant probability that a particle changes type at each step. In the case where , the system corresponds to the classical version of the BML model. We define a state of the system where all particles continue to move indefinitely, in both the present and the future, as a state of free movement. A sufficient condition for the system to result in a state of free movement from any initial state (condition for self‐organization) has been found. This condition is that the number of particles be not greater than half the greatest common divisor of the numbers . It has been proved that, if , and whether or and there are both at least one particle of the first type and at least particle of the second type, then a necessary condition for a state of free movement to exist is the greatest common divisor of and be not less than 3. The theorems are formulated in terms of the algebraic structures and terms of the dynamical system. The spectrum of particle velocities has been found for the net . This approach allows us to hope that the spectrum of dynamical system can be studied for an arbitrary dimension of the net.

Revealed preference domains from random choice

Ordinal random utility models (RUMs) are based on the presumption that fluctuating preferences drive stochastic choices. We study a novel property of RUM subclasses called exclusiveness, satisfied whenever the supports of all RUM representations of stochastic choice data, rationalizable by a RUM over preferences within a specific domain, also belong to that domain. We demonstrate that well-known preference domains such as the single-peaked, single-dipped, triple-wise value-restricted and peak-monotone are RUM-exclusive, alongside a novel domain we term peak-pit on a line. Building on existing characterization results, we show how these preference domains can be directly revealed from stochastic choice data, without the need to compute all RUM representations.

Methods for testing the random utility model

The Random Utility Model, central in stochastic choice theory, is equivalent to assume that a probability vector belongs to a convex cone. We investigate its underlying geometry, introduce two new testing procedures, and compare them by simulation.

Stochastic heuristics for decisions under risk and uncertainty.

Models of heuristics are often predicated on the desideratum that they should possess no free parameters. As a result, heuristic implementations are usually deterministic and do not allow for any choice errors, as the latter would require a parameter to regulate the magnitude of errors. We discuss the implications of this in light of research that highlights the evidence supporting stochastic choice and its dependence on preferential strength. We argue that, in principle, the existing models of deterministic heuristics should, and can, be quite easily modified to stochastic counterparts through the addition of an error mechanism. This requires a single free parameter in the error mechanism, whilst otherwise retaining the parameter-free cognitive processes in the deterministic component of existing heuristics. We present various types of error mechanisms applicable to heuristics and discuss their comparative virtues and drawbacks, paying particular attention to their impact on model comparisons between heuristics and parameter-rich models.

Does correlation matter in probability matching? A laboratory investigation

Probability Matching, a classical violation of expected utility maximization, refers to people's tendency to randomize, or even match their choice frequency to the outcome probability, when choosing over binary lotteries that differ only in their probabilities. Why? I present an experiment designed to distinguish between several broad classes of explanations: (1) models of Correlation-Invariant Stochastic Choice — randomizing due to factors orthogonal to the correlation between lotteries, such as non-standard preferences or errors, and (2) models of Correlation-Sensitive Stochastic Choice — deliberately randomizing due to misperceived hedging opportunities, especially when lotteries are negatively correlated. My experimental design differentiates between their testable predictions by varying the correlation between lottery outcomes. The findings indicate that the first class, despite being home to most existing theories, has limited explanatory power. Using additional treatment, I rule out Similarity Heuristics as a competing explanation with the second class. The results indicate that a vast majority of individuals deliberately randomize due to misperceived hedging opportunities.

Using traffic assignment models to assist Bayesian inference for origin–destination matrices

Estimation of traffic volumes between each origin and destination of travel is a standard practice in transport engineering. Commonly the available data constitute traffic counts at various locations on the network, supplemented by a survey-based prior estimate of mean origin–destination traffic volumes. Statistical inference in this type of network tomography problem is known to be challenging. Moreover, the difficulties are increased in practice by the presence of a large number of nuisance parameters corresponding to route choice probabilities, for which we have no direct prior information. Working in a Bayesian framework, we determine these parameters using a stochastic user equilibrium route choice model. We develop an MCMC algorithm for model fitting. This requires repeated computation of stochastic user equilibrium flows, and so we develop a computationally cheap emulator. Our methods are tested on numerical examples based on a section of the road network in the English city of Leicester.

Prestige bias in cultural evolutionary dynamics.

If the traits of more successful individuals are more likely to be adopted, the resulting cultural transmission is described as success biased. In contrast, if the traits of 'prestigious' individuals-those who have already been copied many times-are more likely to be adopted, this is described as prestige-biased cultural transmission. In this case, prestige can be a convenient proxy for success. However, it is unclear how success and prestige biases interact to determine the outcome of cultural evolutionary dynamics. Here, we aim to clarify this using mathematical analysis and stochastic simulations. We find analytic approximations to the stochastic role-model choice process that facilitate the mathematical analysis and reduce the computational complexity of simulations. Approximations are given to the fixation probability and the fixation time of an invading cultural trait in different environments. Our results show that success bias effectively plays the role of natural selection, whereas prestige bias effectively plays the role of genetic drift. Prestige bias, which may be strong in highly social communities, also accelerates the evolutionary dynamics, as expected in a rich-get-richer process. These results signify a step forward in understanding how different cultural transmission biases interact.

On the multiplicative inequality

The multiplicative inequality (MI) introduced by Sattath and Tversky (1976) is a rare example of a simple and intuitively appealing condition relating choice probabilities across choice sets of different sizes. It is also a testable implication of two models of stochastic discrete choice: the Elimination by Aspects model of Tversky (1972b) and the independent random utility model. We prove several results on the multiplicative inequality and its relationship to the regularity condition. One major result illustrates how little the MI constrains binary choice probabilities: it implies that every system of binary choice probabilities on a universe of choice objects can be extended to a complete system of choice probabilities satisfying the MI. In this sense, the MI is complementary to axioms for binary choice probabilities, of which many have been proposed. We also discuss choice environments where the multiplicative inequality is implausible.

Noisy kinetic-exchange opinion model with aging.

We study the critical behavior of a noisy kinetic opinion model subject to resilience to change depending on aging, defined as the number of interactions before a change of opinion state. In this model, the opinion of each agent can take three discrete values, the extreme ones ±1, and also the intermediate value 0, and it can evolve through kinetic exchange when interacting with another agent, or independently, by stochastic choice (noise). The probability of change by pairwise interactions depends on the age that the agent has remained in the same state, according to a given kernel. We particularly develop the cases where the probability decays either algebraically or exponentially with the age, and we also consider the anti-aging scenario where the probability increases with the age, meaning that it is more likely to change mind the longer the persistence in the current state. For the opinion dynamics in a complete graph, we obtain analytical predictions for the critical curves of the order parameters, in perfect agreement with agent-based simulations. We observe that sufficiently weak aging (slow change of the kernel with the age) favors partial consensus with respect to the aging-insensitive scenario, while for sufficiently strong aging, as well as for anti-aging, the opposite trend is observed.

Consciously stochastic in preference reversals

Stochastic choice, the act of choosing differently in repeated decisions, can be a conscious decision made by individuals who are aware of their inability to make a definitive choice. To examine the prevalence and implications of conscious stochastic choice, we developed a novel method and implemented it in a preference reversal experiment: In each valuation choice between the bet and a varying reference option, subjects could either pay a small cost to select a specific option or opt for a free randomization choice where a computer randomly selects an option. Our findings revealed that the majority of subjects exhibited conscious stochastic choice, and further that their choices were significantly affected by the elicitation procedures.

Security Nanobot for Industrial Safety and Health in Sports using a Nanobot-Assisted Stochastic Discrete Choice Model

Security Nanobot for Industrial Safety and Health in Sports using a Nanobot-Assisted Stochastic Discrete Choice Model

Epigenetic programming of stochastic olfactory receptor choice.

The mammalian sense of smell relies upon a vast array of receptor proteins to detect odorant compounds present in the environment. The proper deployment of these receptor proteins in olfactory sensory neurons is orchestrated by a suite of epigenetic processes that remodel the olfactory genes in differentiating neuronal progenitors. The goal of this review is to elucidate the central role of gene regulatory processes acting in neuronal progenitors of olfactory sensory neurons that lead to a singular expression of an odorant receptor in mature olfactory sensory neurons. We begin by describing the principal features of odorant receptor gene expression in mature olfactory sensory neurons. Next, we delineate our current understanding of how these features emerge from multiple gene regulatory mechanisms acting in neuronal progenitors. Finally, we close by discussing the key gaps in our understanding of how these regulatory mechanisms work and how they interact with each other over the course of differentiation.

A Random Consideration Set Model for Demand Estimation, Assortment Optimization, and Pricing

Random Consideration Set Model We operationalize a microfounded consumer choice model—the random consideration set (RCS) choice model of Manzini and Mariotti [Manzini P, Mariotti M (2014) Stochastic choice and consideration sets. Econometrica 82(3):1153–1176]—that captures the limited attention of consumers, assuming that purchases are based on fixed preference orderings with consideration sets formed from independent attentions. We provide a condition for uniquely identifying model parameters and design an efficient algorithm for model parameters estimation. We offer a greedy-like algorithm for assortment optimization, adaptable for optimal assortment subject to cardinality constraint or discovering efficient sets. We extend the model to consider random product preferences, with a 1/2 performance-guaranteed approximation algorithm. Using data from a major U.S. airline, we find that the RCS model outperforms the mixed multinomial logit model in approximately half of the markets, particularly with smaller, less varied data sets.

Regret, responsibility, and randomization: A theory of stochastic choice

I provide a model of deliberate stochastic choice over acts (state-contingent payoffs). In this model, the decision maker finds randomization desirable since it reduces responsibility and regret after the state uncertainty resolves. I provide an axiomatic characterization of stochastic choices compatible with this model. I also discuss the comparative statics and some behavioral implications of the model and outline potential avenues for extending and generalizing the model.