Subjective Probability Judgments Research Articles

Bridging the gap between subjective probability and probability judgments: The quantum sequential sampler.

Bridging the gap between subjective probability and probability judgments: The quantum sequential sampler.

Supplemental Material for Bridging the Gap Between Subjective Probability and Probability Judgments: The Quantum Sequential Sampler

Supplemental Material for Bridging the Gap Between Subjective Probability and Probability Judgments: The Quantum Sequential Sampler

How Sure is the Driver? Modelling Drivers’ Confidence in Left-Turn Gap Acceptance Decisions

When a person makes a decision, it is automatically accompanied by a subjective probability judgment of the decision being correct, in other words, a confidence judgment. A better understanding of the mechanisms responsible for these confidence judgments could provide novel insights into human behavior. However, so far confidence judgments have been mostly studied in simplistic laboratory tasks while little is known about confidence in naturalistic dynamic tasks such as driving. In this study, we made a first attempt of connecting fundamental research on confidence with naturalistic driver behavior. We investigated the confidence of drivers in left-turn gap acceptance decisions in a driver simulator experiment (N = 17). We found that confidence in these decisions depends on the size of the gap to the oncoming vehicle. Specifically, confidence increased with the gap size for trials in which the gap was accepted, and decreased with the gap size for rejected gaps. Similarly to more basic tasks, confidence was negatively related to the response times and correlated with action dynamics during decision execution. Finally, we found that confidence judgments can be captured with an extended dynamic drift–diffusion model. In the model, the drift rate of the evidence accumulator as well as the decision boundaries are functions of the gap size. Furthermore, we demonstrated that allowing for post-decision evidence accumulation in the model increases its ability to describe confidence judgments in rejected gap decisions. Overall, our study confirmed that principles known from fundamental confidence research extend to confidence judgments in dynamic decisions during a naturalistic task.

Age-Related Differences in Moral Judgment: The Role of Probability Judgments.

Research suggests that moral evaluations change during adulthood. Older adults (75+) tend to judge accidentally harmful acts more severely than younger adults do, and this age-related difference is in part due to the greater negligence older adults attribute to the accidental harmdoers. Across two studies (N = 254), we find support for this claim and report the novel discovery that older adults' increased attribution of negligence, in turn, is associated with a higher perceived likelihood that the accident would occur. We propose that, because older adults perceive accidents as more likely than younger adults do, they condemn the agents and their actions more and even infer that the agents' omission to exercise due care is intentional. These findings refine our understanding of the cognitive processes underpinning moral judgment in older adulthood and highlight the role of subjective probability judgments in negligence attribution.

Probabilistic Truth, Relativism, and Objective Chance

AbstractIn Probabilistic Knowledge Sarah Moss proposes that our credences and subjective probability judgments (SPJs) can constitute knowledge. Mossean probabilistic knowledge is grounded in probabilistic beliefs that are justified, true, and unGettiered. In this paper I aim to address and solve two challenges that arise in the vicinity of the factivity condition for probabilistic knowledge: the factivity challenge and the challenge from probabilistic arguments from ignorance (probabilistic AIs). I argue that while Moss's deflationary solution to the factivity challenge formally works, it leaves us ill-equipped to handle probabilistic AIs. An account of probabilistic knowledge that cannot overcome probabilistic AIs makes knowledge of thoroughly probabilistic contents a rare and unstable phenomenon, at best, or, at worst, impossible. I hold that establishing a metaphysically enriched account of probabilistic truth is therefore mandatory. I go on to develop a truth-conditional approach on probabilistic factivity that is relativistic in its nature and centers on objective chances. I show that while the approach is still compatible with Moss's overall semantics for probabilistic knowledge, it provides us with a simple but forceful answer to probabilistic AIs.

When alternative hypotheses shape your beliefs: Context effects in probability judgments

When people are asked to estimate the probability of an event occurring, they sometimes make different subjective probability judgments for different descriptions of the same event. This implies the evidence or support recruited to make these judgments is based on the descriptions of the events (hypotheses) instead of the events themselves, as captured by Tversky and Koehler’s (1994) support theory. Support theory, however, assumes each hypothesis elicits a fixed level of support (support invariance). Here, across three studies, we tested this support invariance assumption by asking participants to estimate the probability that an event would occur given a set of relevant statistics. We show that the support recruited about a target hypothesis can depend on the other hypotheses under consideration. Results reveal that for a pair of competing hypotheses, one hypothesis (the target hypothesis) appears more competitive relative to the other when a dud–a hypothesis dominated by the target hypothesis–is present. We also find that the target hypothesis can appear less competitive relative to the other when a resembler–a hypothesis that is similar to the target hypothesis–is present. These context effects invalidate the support invariance assumption in support theory and suggest that a similar process that drives preference construction may also underlie belief construction.

It happens because I'm watching it: Observing an uncertain event can affect subjective probability judgments

AbstractA longstanding body of research has investigated various factors that bias subjective probability judgments. The current research explores a novel bias in judgments of probability—that observing an event affects probability estimation of the event's outcome. Our findings from multiple online and laboratory experiments demonstrate that individuals overestimate the likelihood of a positive outcome occurring when they observe (vs. do not observe) the uncertain event unfolding in front of them. Study 1 explores the effect of watching on probability estimation using a random card draw conducted in an auditorium. Studies 2 and 3 use an online procedure to further demonstrate the effect. In Study 4 we manipulated an illusory sense of control followed by a football game scenario and in Study 5 we used a live versus previously recorded urn game to further investigate the watching effect. We suggest that illusion of control, wishful thinking, and availability are among the possible explanations for this effect.

Diversity effects in subjective probability judgment

Previous research has shown that the judged probability of an event depends on whether its description mentions examples (“What is the probability that a randomly chosen Italian businessman will travel during the next month to Warsaw, Budapest, Prague or some other European city?”) or does not mention examples (“What is the probability that a randomly chosen Italian businessman will travel during the next month to a European city?”). Here, we examined descriptions that mention examples and manipulated whether these are relatively similar (e.g., Warsaw, Budapest, Prague) or diverse (e.g., Warsaw, Marseilles, Helsinki). Four experiments (N = 1112) revealed a diversity effect: Overall, descriptions with diverse examples received higher probability judgments than descriptions with similar examples. We discuss several possible mechanisms for this effect, such as that descriptions with diverse examples prompt fuller representations of the target category or that the effect is driven by a representativeness or proximity heuristic.

Are Jurors Intuitive Statisticians? Bayesian Causal Reasoning in Legal Contexts.

In criminal trials, evidence often involves a degree of uncertainty and decision-making includes moving from the initial presumption of innocence to inference about guilt based on that evidence. The jurors’ ability to combine evidence and make accurate intuitive probabilistic judgments underpins this process. Previous research has shown that errors in probabilistic reasoning can be explained by a misalignment of the evidence presented with the intuitive causal models that people construct. This has been explored in abstract and context-free situations. However, less is known about how people interpret evidence in context-rich situations such as legal cases. The present study examined participants’ intuitive probabilistic reasoning in legal contexts and assessed how people’s causal models underlie the process of belief updating in the light of new evidence. The study assessed whether participants update beliefs in line with Bayesian norms and if errors in belief updating can be explained by the causal structures underpinning the evidence integration process. The study was based on a recent case in England where a couple was accused of intentionally harming their baby but was eventually exonerated because the child’s symptoms were found to be caused by a rare blood disorder. Participants were presented with a range of evidence, one piece at a time, including physical evidence and reports from experts. Participants made probability judgments about the abuse and disorder as causes of the child’s symptoms. Subjective probability judgments were compared against Bayesian norms. The causal models constructed by participants were also elicited. Results showed that overall participants revised their beliefs appropriately in the right direction based on evidence. However, this revision was done without exact Bayesian computation and errors were observed in estimating the weight of evidence. Errors in probabilistic judgments were partly accounted for, by differences in the causal models representing the evidence. Our findings suggest that understanding causal models that guide people’s judgments may help shed light on errors made in evidence integration and potentially identify ways to address accuracy in judgment.

An account of subjective probability judgment for joint events: Conjunctive and disjunctive.

Probability judgment is a vital part of many aspects of everyday life. In the present paper, we present a new theory of the way in which individuals produce probability estimates for joint events: conjunctive and disjunctive. We propose that a majority of individuals produce conjunctive (disjunctive) estimates by making a quasi-random adjustment, positive or negative, from the less (more) likely component probability with the other component playing no obvious role. In two studies, we produce evidence supporting propositions that follow from our theory. First, the component probabilities do appear to play the distinct roles we propose in determining the joint event probabilities. Second, contrary to probability theory and other accounts of probability judgment, we show that the conjunctive-less likely probability difference is unrelated to the more likely disjunctive probability difference (in normative theory these quantities are identical). In conclusion, while violating the norms of probability judgment, we argue that estimates produced in the manner we propose will be close enough to the normative values especially given the changing nature of the external environment and the incomplete nature of available information.

Heuristics can produce surprisingly rational probability estimates: Comment on Costello and Watts (2014).

Costello and Watts (2014) present a model assuming that people's knowledge of probabilities adheres to probability theory, but that their probability judgments are perturbed by a random noise in the retrieval from memory. Predictions for the relationships between probability judgments for constituent events and their disjunctions and conjunctions, as well as for sums of such judgments were derived from probability theory. Costello and Watts (2014) report behavioral data showing that subjective probability judgments accord with these predictions. Based on the finding that subjective probability judgments follow probability theory, Costello and Watts (2014) conclude that the results imply that people's probability judgments embody the rules of probability theory and thereby refute theories of heuristic processing. Here, we demonstrate the invalidity of this conclusion by showing that all of the tested predictions follow straightforwardly from an account assuming heuristic probability integration (Nilsson, Winman, Juslin, & Hansson, 2009). We end with a discussion of a number of previous findings that harmonize very poorly with the predictions by the model suggested by Costello and Watts (2014).

The role of ANS acuity and numeracy for the calibration and the coherence of subjective probability judgments.

The purpose of the study was to investigate how numeracy and acuity of the approximate number system (ANS) relate to the calibration and coherence of probability judgments. Based on the literature on number cognition, a first hypothesis was that those with lower numeracy would maintain a less linear use of the probability scale, contributing to overconfidence and nonlinear calibration curves. A second hypothesis was that also poorer acuity of the ANS would be associated with overconfidence and non-linearity. A third hypothesis, in line with dual-systems theory (e.g., Kahneman and Frederick, 2002) was that people higher in numeracy should have better access to the normative probability rules, allowing them to decrease the rate of conjunction fallacies. Data from 213 participants sampled from the Swedish population showed that: (i) in line with the first hypothesis, overconfidence and the linearity of the calibration curves were related to numeracy, where people higher in numeracy were well calibrated with zero overconfidence. (ii) ANS was not associated with overconfidence and non-linearity, disconfirming the second hypothesis. (iii) The rate of conjunction fallacies was slightly, but to a statistically significant degree decreased by numeracy, but still high at all numeracy levels. An unexpected finding was that participants with better ANS acuity gave more realistic estimates of their performance relative to others.

Probabilistic Coherence Weighting for Optimizing Expert Forecasts

Methods for eliciting and aggregating expert judgment are necessary when decision-relevant data are scarce. Such methods have been used for aggregating the judgments of a large, heterogeneous group of forecasters, as well as the multiple judgments produced from an individual forecaster. This paper addresses how multiple related individual forecasts can be used to improve aggregation of probabilities for a binary event across a set of forecasters. We extend previous efforts that use probabilistic incoherence of an individual forecaster's subjective probability judgments to weight and aggregate the judgments of multiple forecasters for the goal of increasing the accuracy of forecasts. With data from two studies, we describe an approach for eliciting extra probability judgments to (i) adjust the judgments of each individual forecaster, and (ii) assign weights to the judgments to aggregate over the entire set of forecasters. We show improvement of up to 30% over the established benchmark of a simple equal-weighted averaging of forecasts. We also describe how this method can be used to remedy the “fifty–fifty blip” that occurs when forecasters use the probability value of 0.5 to represent epistemic uncertainty.

Cognitive closure and risk sensitivity in the fear of crime

PurposeThis study was designed to answer two questions. First, does the risk sensitivity model of worry about crime replicate in three European countries? Second, can the model be extended to include need for cognitive closure?MethodA national probability survey in Italy, Bulgaria, and Lithuania measured worry about criminal victimization, risk perception, and need for cognitive closure. Additive and interactive relationships between latent constructs were tested using latent moderated structural equation modelling.ResultsFirst, perceived likelihood, control, and consequence were statistically significant additive predictors of worry about crime. Second, the association between subjective probability judgements and worry about crime was stronger among people who associated the uncertain event with serious personal consequences and among people who had a high need for cognitive closure. Third, need for cognitive closure was associated with greater perceived consequences of victimization, but not with different perceptions of the likelihood and controllability of personal victimization.ConclusionsThis study provides empirical support for an extended risk sensitivity model in three European countries. Findings suggest that risk perception involves multiple – and interacting – dimensions that constitute sensitivity to risk, as well as individual differences in knowledge construction, information judgement, and processing. Future work should address (1) whether probability judgements shift psychological distance to uncertain future outcomes, and (2) whether the effect of psychological distance on worry about crime is greater among people who construe the outcome to be severe in consequence and who desire definite knowledge and dislike uncertainty in their lives.

The Measurement of Subjective Probability: Evaluating the Sensitivity and Accuracy of Various Scales

The RISK of an event generally relates to its expected severity and the perceived probability of its occurrence. In RISK research, however, there is no standard measure for subjective probability estimates. In this study, we compared five commonly used measurement formats--two rating scales, a visual analog scale, and two numeric measures--in terms of their ability to assess subjective probability judgments when objective probabilities are available. We varied the probabilities (low vs. moderate) and severity (low vs. high) of the events to be judged as well as the presentation mode of objective probabilities (sequential presentation of singular events vs. graphical presentation of aggregated information). We employed two complementary goodness-of-fit criteria: the correlation between objective and subjective probabilities (sensitivity), and the root mean square deviations of subjective probabilities from objective values (accuracy). The numeric formats generally outperformed all other measures. The severity of events had no effect on the performance. Generally, a rise in probability led to decreases in performance. This effect, however, depended on how the objective probabilities were encoded: pictographs ensured perfect information, which improved goodness of fit for all formats and diminished this negative effect on the performance. Differences in performance between scales are thus caused only in part by characteristics of the scales themselves-they also depend on the process of encoding. Consequently, researchers should take the source of probability information into account before selecting a measure.

Comparability Effects in Probability Judgments

Psychological theories of subjective probability judgments assume that accumulated evidence (support) mediates the relation between the description of a to-be-judged event (hypothesis) and the judgment. To scale a probability from the support for a hypothesis, these psychological theories make a strong independence assumption. This assumption is stated in the form of the product rule, in which the support garnered for a particular hypothesis is independent of the support for the alternative hypothesis. In the study reported here, I asked participants to judge the likelihood of a bicyclist winning a simulated race. Results showed that the independence assumption was systematically violated. Observed judgments suggest that when a probability judgment is made, the comparability or similarity of the hypotheses on one dimension increases the weight that judges allocate to differences on the other dimensions. These results speak against the simple scalability-processing assumption of support theory, and they illustrate the need for a theory of judgment processes that describes how the similarity between hypotheses shapes judgment.

Seizing, Freezing and the Fear of Crime: Developing the Risk Sensitivity Model in Three European Countries

Purpose. This study was designed to answer two questions. First, does the risk sensitivity model of worry about crime replicate in three European countries? Second, can the model be extended to include need for cognitive closure?Method. A national probability survey in Italy, Bulgaria and Lithuania measured worry about criminal victimisation, risk perception, and need for cognitive closure. Additive and interactive relationships between latent constructs were tested using latent moderated structural equation modelling. Results. First, perceived likelihood, control and consequence were statistically significant additive predictors of worry about crime. Second, the association between subjective probability judgments and worry about crime was stronger among people who associated the uncertain event with serious personal consequences and among people who had a high need for cognitive closure. Third, need for cognitive closure was associated with greater perceived consequences of victimisation, but not with different perceptions of the likelihood and controllability of personal victimisation.Conclusions. This study provides empirical support for an extended risk sensitivity model in three European countries. Findings suggest that risk perception involves multiple – and interacting – dimensions that constitute sensitivity to risk, as well as individual differences in knowledge construction, information judgement and processing. Future work should address (a) whether probability judgements shift psychological distance to uncertain future outcomes, and (b) whether the effect of psychological distance on worry about crime is greater among people who construe the outcome to be severe in consequence and who desire definite knowledge and dislike uncertainty in their lives.

How Many Alternatives? Partitions Pose Problems for Predictions and Diagnoses

This paper focuses on one matter that poses a problem for both human judges and standard probability frameworks, namely the assumption of a unique (privileged) and complete partition of the state‐space of possible events. This is tantamount to assuming that we know all possible outcomes or alternatives in advance of making a decision, but it is clear that there are many practical situations in prediction, diagnosis, and decision‐making where such partitions are contestable and/or incomplete. The paper begins by surveying the impact of partitions on the choice of priors in formal probabilistic updating frameworks, and on human subjective probability judgements. That material is followed by an overview of strategies for dealing with partition dependence, including considerations of how a rational agent’s preferences may determine the choice of a partition.

Partition Priming in Judgments of Imprecise Probabilities

Abstract On grounds of insufficient reason, a probability of 1/K is assigned to K mutually exclusive possible events when nothing is known about the likelihood of those events. Fox and Rottenstreich (2003) present evidence that subjective probability judgments are typically biased towards this ignorance prior, and therefore depend on the partition K. Results from two studies indicate that lower-upper (imprecise) probability judgments by naive judges also exhibit partition dependence, despite the potential that imprecise probabilities provide for avoiding it. However, beta regression reveals two kinds of priming effects, one of which is modeled by mixture distributions. Another novel finding suggests that when partition primes conflict with a normatively correct partition some judges widen their probability intervals to encompass both partitions. The results indicate that imprecise probability judgments may be better suited than precise probabilities for handling conflicting or ambiguous information about ...

Augmenting understanding of the relationship between situation awareness and confidence using calibration analysis

The present study represents a preliminary examination of the relationship between situation awareness (SA) and confidence within a distributed information-sharing environment using the calibration methodology. The calibration methodology uses the indices of calibration, resolution and over/under-confidence to examine the relationship between the accuracy of the responses and the degree of confidence that one has in these responses, which leads to a measure of an operator's meta-SA. The results of this study revealed that, although the participants were slightly overconfident in their responses, overall they demonstrated good meta-SA. That is, the participants' subjective probability judgements corresponded to their pattern of SA response accuracy. It is concluded that the use of calibration analysis represents a better methodology for expanding our understanding of the relationship between SA and confidence and ultimately how this relationship can impact decision-making and performance in applied settings than can be achieved by examining SA measures alone.