Subjective Probability Distributions Research Articles

Overconfidence in Probability Distributions: People Know They Don’t Know, but They Don’t Know What to Do About It

Overconfidence is pervasive in subjective probability distributions (SPDs). We develop new methods to analyze judgments that entail both a distribution of possible outcomes in a population (aleatory uncertainty) and imperfect knowledge about that distribution (epistemic uncertainty). In four experiments, we examine the extent to which subjective probability mass is concentrated in a small portion of the distribution versus spread across all possible outcomes. We find that although SPDs roughly match the concentration of the empirical, aleatory distributions, people’s judgments are consistently overconfident because they fail to spread out probability mass to account for their own epistemic uncertainty about the location and shape of the distribution. Although people are aware of this lack of knowledge, they do not appropriately incorporate it into their SPDs. Our results offer new insights into the causes of overconfidence and shed light on potential ways to address this fundamental bias. This paper was accepted by Yuval Rottenstreich, behavioral economics and decision analysis. Funding: Support for this research was provided by the Fuqua School of Business at Duke University and the Haas School of Business at the University of California at Berkeley. Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mnsc.2019.00660 .

OP141 Expert Knowledge Elicitation in Health Technology Assessment: Our Experience Using the Sheffield Elicitation Framework

IntroductionExpert judgement has an important role in health technology assessment (HTA), including as a source of evidence to inform economic modeling when published data are lacking. Quantitative information may be elicited from experts to inform model inputs and associated uncertainty using one of many expert elicitation methodologies. Here, the feasibility and potential benefits of one expert elicitation method, the Sheffield Elicitation Framework (SHELF), to the HTA process is examined.MethodsThe SHELF method seeks to express the knowledge of multiple experts in the form of a subjective probability distribution. Eliciting a subjective probability distribution allows the uncertainty of experts to be included in probabilistic sensitivity analysis, which is becoming an increasingly prominent feature of HTAs. The individual knowledge of participating experts is combined through behavioral aggregation, where experts participate in a discussion before being asked to provide judgments from the perspective of a rational impartial observer. The whole process is led by a facilitator who ensures all participants contribute and confirm that the final distribution is a product of consensus, not compromise.ResultsWe recently conducted two SHELF elicitations as part of an ongoing project aiming to streamline the assessment of positron emission tomography (PET) in Australia. These elicitations provided insight into the usefulness of SHELF within the HTA setting. Given the constraints imposed by the COVID-19 pandemic, the elicitation sessions were conducted online rather than in the ideal face-to-face manner. In collaboration with one of the developers, we successfully adapted the method by making use of video conferencing technology to provide an online environment that mimicked the face-to-face setup as much as possible.ConclusionsSHELF provides a rigorous and scientific method by which to elicit the knowledge of multiple experts in the form of a probability distribution. However, the method is resource intensive and may be best reserved for when data on key drivers are lacking.

The impact of supply chain disruptions on business expectations during the pandemic

Utilizing the Federal Reserve Bank of Atlanta’s Business Inflation Expectations (BIE) survey, which has been continuously collecting subjective probability distributions over own-firm future unit costs since October 2011, we document two facts about firms’ marginal cost expectations and risk during the COVID-19 pandemic. First, in the early months of the pandemic, firms, on net, saw COVID-19 largely as a demand shock and lowered their one-year ahead expectations. However, as the pandemic wore on, firms’ one-year ahead unit cost expectations rose sharply alongside their views on supply chain and operating capacity disruptions. Second, the balance of unit cost risks shifted sharply over the course of the pandemic and by the end of 2022, upside risks had sharply outweighed perceived downside risks over the year ahead. We find that both positive demand shocks (e.g. large order backlogs) and negative supply shocks (e.g. long supplier delivery times and labor shortages) have contributed to elevated short-term unit cost expectations and risk. Specifically, supply shocks accounted for roughly 40% of the increase in manufacturers’ and nearly one-third of service-providers’ unit cost expectations.

Simple informative prior distributions for Type A uncertainty evaluation in metrology

The result of a measurement, including the expression of uncertainty in the measurement, should represent a carefully considered opinion based on the metrologist’s experience and expertise, as well as on the data and other information sources. This is the position of the Guide to the expression of uncertainty in measurement (GUM), where the requirement for such judgment is clear in the case of Type B (non-statistical) evaluation. However, when making Type A evaluations, involving statistical analysis of data, the GUM and related GUM documents implicitly consider the data to be the only relevant information. This perspective is unfortunate, and arguably unscientific, when, as is frequently the case, the metrologist could bring other relevant information to bear. Bayesian statistical methods allow the use of prior information in addition to the data in Type A evaluation and have been advocated by several authors. However, prior information is in principle subjective and, as in other fields, there is some resistance in the metrology community at large to embrace Bayesian methods using meaningful, subjective prior probability distributions. We address our paper to metrologists in measurement and calibration laboratories whose workload is such that new techniques will only be adopted if they have proven advantages and are straightforward to apply routinely. We present two prior distributions for use in the most basic of all Type A evaluations, where the data comprise a sample of indications assumed to be normally distributed. These distributions represent prior information about the observation error variance in a simple form that is readily justified in practice. We show the gains to be achieved by using these prior distributions, both in the single Type A evaluation and in a more complex measurement model, and present simple guidance for verifying their validity.

Balance of Risks and the Anchoring of Consumer Expectations

This paper shows that expected inflation risks pose threats to the anchoring of expectations. I propose a new method for fitting subjective probability distributions to density forecasts that allows for asymmetric beliefs over inflation outcomes. Using data from the Federal Reserve Bank of New York’s Survey of Consumer Expectations, I show that medium run expectations move in the direction of perceived short run risks. A diffusion index of consumers’ perceived balance of risks to inflation shows that high short run inflation expectations coincide with the balance of medium risks being weighted to the upside.

Subjective Probability Distributions of Nonlinear Payoffs: Recovering Option Payoff and Agent's Utility Distributions

Subjective Probability Distributions of Nonlinear Payoffs: Recovering Option Payoff and Agent's Utility Distributions

Fuzzy Distribution Sets

We introduce a new type of fuzzy set called Fuzzy Distribution Set (FDS). Fuzzy distribution sets are fuzzy sets defined on a finite domain subject to a sum of membership values equal to 1. Such fuzzy sets can serve as models of subjective probability distributions and subjective weight distributions. Considering these distributions as fuzzy sets gives a possibility to extend on such distributions the operations of fuzzy sets and, more generally, the calculus of fuzzy restrictions developed during the last decades. Recently Yager introduced the concept of negation of probability distributions. In our works, we studied several classes of such negations. Here we consider an involutive negation of probability distributions as a complement of FDS. We introduce the operations of union and intersection of fuzzy distribution sets. These basic operations on FDS can serve as a basis for the application of fuzzy logic methods to subjective probability and weight distributions. These operations can be used for the development of reasoning models with subjective probability distributions and subjective weighting functions. Weight distributions can be used in multi-criteria, multi-person, and multi-attribute decision-making models.

Recovering subjective probability distributions

AbstractThis study proposes a direct estimation method for recovering subjective probability distributions from option prices. We find that the subjective cumulative distribution function and subjective statistics are represented as static portfolios composed of plain vanilla options. The portfolio weights of the options are determined by the levels of the reciprocal of the pricing kernel and their differentiation. Using options data, we investigate the historical behavior of subjective probability distributions of returns on the S&P 500 index. We also analyze subjective equity risk premiums and subjective variance risk premiums, which can be regarded as ex ante risk premiums.

Testing the effectiveness of debiasing techniques to reduce overprecision in the elicitation of subjective continuous probability distributions

Formal expert elicitation is a widely used method for quantifying uncertain variables in decision and risk analysis. When estimating uncertain variables, experts and laypeople exhibit overprecision, meaning that the ranges of their estimates are too narrow. Overprecision, a form of overconfidence, is pervasive and hard to correct, thus posing a challenge to expert elicitation. Following the increasing interest toward improving judgments in Behavioral Operational Research (OR), and the limited evidence about the effectiveness of debiasing tools, the aim of our research is to test the effectiveness of commonly employed practices for debiasing overprecision. We conducted two experiments, testing a set of debiasing techniques when eliciting points of a cumulative distribution functions for general knowledge questions. The debiasing procedures included hypothetical bets, counterfactual argumentation, and automatic stretching to increase the ranges of subjects’ initial estimates. We find that two debiasing strategies that require further reasoning after initial estimates (hypothetical bets and counterfactuals) were not very effective for reducing overprecision, while the use of multipliers that increase the initial range of distributions, coupled with a re-elicitation of the distribution with the new range, provided more positive results. We provide some recommendations for expert elicitation in OR practice, based on our findings, and suggest avenues for further research into debiasing overprecision.

Inflation disasters and consumption

Consumers with longer-tailed subjective probability distributions of inflation anticipate lower real consumption growth and are more favorably inclined to purchasing durable goods. I propose a model in which rare inflation disasters increase the cost of future credit by raising debt issuance costs, prompting consumers to stock up on debt and move purchases to the present. Consistent with this theory, consumers with longer-tailed distributions anticipate higher future interest rates. The effects of anticipated tail risks on consumption plans are more pronounced among credit market participants.

Firms’ subjective uncertainty and forecast errors: survey evidence from Japan

ABSTRACT Using original survey data linked with the financial statements of Japanese listed firms, this study presents an ex post evaluation of firms’ ex ante subjective uncertainty. Ex ante forecast uncertainty in terms of sales and employment growth is derived from firms’ subjective confidence interval around their point forecasts. Ex post forecast error is calculated as the deviation of the realized figures from the point forecasts. The results indicate that ex ante subjective uncertainty has a significant positive association with realized absolute forecast error. The subjective confidence interval for a firm’s own business forecast, in comparison with that for macroeconomic variables, is reliable as a measure of uncertainty. These findings indicate that the subjective probability distribution of business outlook captured by firm surveys contains valuable information for measuring economic uncertainty at the micro-level.

A Simple Agro-Economic Model for Optimal Farm Nitrogen Application under Yield Uncertainty

Farmers in the developed world tend to over-apply fertilizer, and we explore a model for decision-making under uncertainty in yields. This article proposes an agro-economic model for farmer decision-making based on subjective expected yield and crop response to fertilization. The model explores subjective yield probability distributions that are both better suited to subjective crop yields than the previously proposed probability distribution and is easier to extract from farmers. The model allows the analysis of the impact of changes in fertilizer price and variance of expected yields. The model result is consistent with observed farmer behavior based on the rule of “fertilizing for the good years” that appears, according to our model, as rational and consistent with expected profit maximization under yield uncertainty since the cost of over-application is lower than that of the opportunity cost of under-application. The goal of increasing the efficiency of nitrogen use requires both technical innovation and an expansion of the knowledge on the socioeconomic factors underlying excessive crop fertilization that must be improved both to meet future food demands and to prevent environmental degradation and climate change.

The Capital Asset Pricing Model as Modern Austrian Business Cycle Theory

According to Austrian macroeconomic theory, capital accumulation today generates supply tomorrow. Will that supply match tomorrows demand? In this paper we study this question in a multi-good and risky environment with an efficient financial system. Towards this end we develop a model based on the Capital Asset Pricing Model (CAPM), Rational Expectations, and Linear Homogeneous Production Functions. Within this framework we show how the financial system in the form of CAPM allocates resources across the different sectors in the economy so as to maximize the ratio of expected returns to a measure of risk like the standard deviation of returns for some initial time period t=0. In t=1 a Linear Homogeneous Production Function subject to a random productivity shock determines actual output and the structure of production. The link between t=0 and t=1 is Rational Expectations, the assumption that the subjective probability distribution in t=0 on which real investment decision are made is equivalent to the objective distribution generating the actual output/income in t=1. If total output is more (or less) than expected in t=0, we have a cyclical expansion (or recession) the magnitude of which depends on the spread of the subjective probably distribution in t=0. With this model we discuss: 1) monetary policy under a full reserve banking system with Central bank digital accounts for all; 2) an all-inclusive transaction tax to substitute for the U.S. present tax system; 3) the trade-off between economic growth and the cyclical volatility of the economy; and 4) the advantages and disadvantages of trade deficits.

Belief elicitation with multiple point predictions

We propose a simple, incentive compatible procedure based on binarized linear scoring rules to elicit beliefs about real-valued outcomes - multiple point predictions. Simultaneously eliciting multiple point predictions with linear incentives reveals the subjective probability distribution without pre-defined intervals or probabilistic statements. We show that the approach is theoretically as robust as existing methods, while adapting flexibly to different beliefs. In a laboratory experiment, we compare our procedure to the standard approach of eliciting discrete probabilities on pre-defined intervals. We find that elicitation with multiple point predictions is faster, perceived as less difficult and more consistent with a subsequent decision. We further find that multiple point predictions are more accurate if beliefs vary between participants. Finally, we provide experimental evidence that pre-defined intervals anchor reports.

An Expert Elicitation of the Effects of Low Salinity Water Exposure on Bottlenose Dolphins

There is increasing concern over anthropogenically driven changes in our oceans and seas, from a variety of stressors. Such stressors include the increased risk of storms and precipitation, offshore industries and increased coastal development which can affect the marine environment. For some coastal cetacean species, there is an increased exposure to low salinity waters which have been linked with a range of adverse health effects in bottlenose dolphins. Knowledge gaps persist regarding how different time–salinity exposures affect the health and survival of animals. In such data-poor instances, expert elicitation can be used to convert an expert’s qualitative knowledge into subjective probability distributions. The management implications of this stressor and the subjective nature of expert elicitation requires transparency; we have addressed this here, utilizing the Sheffield Elicitation Framework. The results are a series of time response scenarios to estimate time to death in bottlenose dolphins, for use when data are insufficient to estimate probabilistic summaries. This study improves our understanding of how low salinity exposure effects dolphins, guiding priorities for future research, while its outputs can be used to support coastal management on a global scale.

Accounting information, disclosure, and expected utility: Do investors really abhor uncertainty?

AbstractInvestors are said to “abhor uncertainty,” but if there were no uncertainty they could earn only the risk‐free rate. A fundamental result in the analytical accounting literature shows that investors buying into a CARA‐normal CAPM market pay lower asset prices, gain higher ex‐ante expected returns, and obtain higher expected utility, when the market payoff has higher variance. New investors obtain similar “welfare” gains from risk under a log/power utility CAPM. These results do not imply that investors “abhor information.” To realize investors' ex‐ante expectations, the subjective probability distributions representing market expectations must be accurate. Greater payoff risk can add to investors' expected utility, but higher ex‐post(realized) utility comes from better information and more accurate ex‐ante expectations. An important implication for accounting is that greater disclosure can have the simultaneous effects of (i) exposing more fully or perceptibly firms' payoff uncertainty, thereby increasing new investors' expected utility, and (ii) improving market estimates of firms' payoff parameters (means, variances, covariances), thereby giving investors a better chance of realizing their expectations. Paradoxically, better information can be valuable to new investors by exposing more fully and more accurately the risk in firms' business operations and results–new investors maximizing expected utility want both more risk and better information.

Assessment of Subjective Probability Distributions

Assessment of Subjective Probability Distributions

Optimal Stochastic Process Optimizer: A New Metaheuristic Algorithm With Adaptive Exploration-Exploitation Property

Metaheuristic algorithms are constructed to solve optimization problems, but they cannot solve all the problems with best solutions. This work proposes a novel self-adaptive metaheuristic optimization algorithm, named Optimal Stochastic Process Optimizer (OSPO), which can solve different kinds of optimization problems with promising performance. Specifically, OSPO regards the procedure of optimization as a realization of stochastic process, and with the help of Subjective Probability Distribution Function (SPDF) and Receding Sampling Strategy proposed in this paper, OSPO can control the exploration-exploitation property online by the adaptive modification of the parameters in SPDF. This adaptive exploration-exploitation property of OSPO contributes to dealing with different kinds of problems; thus, it makes OSPO have the potential to solve at least a vast majority of optimization problems. The proposed algorithm is first benchmarked on uni-modal, multi-modal and composite test functions both in low and high dimensions. The results are verified by comparative studies with seven well-performed metaheuristic algorithms. Then, 21 real-world optimization problems are used to further investigate the effectiveness of OSPO. The winners of CEC2020 Competition on Real-World Single Objective Constrained Optimization, SASS algorithm, sCMAgES algorithm, EnMODE algorithm and COLSHADE algorithm are used as four comparative algorithms in real-world optimization problems. The analysis of simulations demonstrates that OSPO is able to provide very competitive performance compared to the comparative meta-heuristics both in benchmark functions and in real-world optimization problems; thus, the potential of OSPO to solve at least a vast majority of optimization problems is verified. A corresponding MATLAB APP demo is available on https://github.com/JiahongXu123/OSPO-algorithm.git.

Accounting Information, Disclosure and Expected Utility: Do Investors Really Abhor Uncertainty?

Investors are said to abhor uncertainty, but if there were no uncertainty they could earn only the risk-free rate. A fundamental result in the analytical accounting literature shows that investors buying into a CARA-normal CAPM market pay lower asset prices, earn higher expected returns, and obtain higher expected utility, when the market payoff has higher variance. New investors obtain similar welfare gains from risk under a log/power utility CAPM. These results do not imply that investors abhor To realize investors' ex ante expectations, the subjective probability distributions representing market expectations must be accurate. Greater payoff risk can add to investors' expected utility, but higher ex post (realized) utility comes from better information and more accurate ex ante expectations. An important implication for accounting is that greater disclosure can have the simultaneous effects of (i) exposing more accurately firms' payoff uncertainty and thereby increasing new investors' expected utility, and (ii) improving market estimates of firms' payoff parameters (means, variances, covariances), thereby giving investors a better chance of realizing their expectations. Paradoxically, better information can be valuable to new investors by exposing more accurately the uncertainty in firms' business operations and results. New investors maximizing expected utility typically want both more uncertainty and better information.

Rationally Uncertain Expectations

When financial securities are modeled as claims on stochastic processes, each trader’s beliefs at time can be summarized by a subjective probability distribution . The dominant Rational Expectations approach typically treats as a singleton that correctly gauges risks. In reality, financial risks are too unstable to squeeze out reasonable doubt. Rational learning can generate behavior that looks turbulently myopic although all it really indicates is high responsiveness to news. While Rational Expectations provides better insights than notions of unquantifiable expectations, it faces a mounting behaviorist critique. Rationally Uncertain Expectations accounts for a broader gamut of evidence with fewer internal contradictions.