Probability Estimation Task Research Articles

Probability estimation in accounting: subjective numeracy matters

PurposeThis study examined subjective numeracy and its relationship with accounting judgements on probability issues.Design/methodology/approachA subjective numeracy scale (SNS) questionnaire was distributed to 231 accounting students to measure self-evaluated numeracy. Modified Bayesian reasoning tasks were applied in an accounting-related probability estimation, manipulating presentation formats.FindingsThe study revealed a positive relationship between self-evaluated numeracy and performance in accounting probability estimation. The findings suggest that switching the format of probability expressions from percentages to frequencies can improve the performance of participants with low self-evaluated numeracy.Research limitations/implicationsAdding objective numeracy measurements could enhance results. Future numeracy research could add objective numeracy items and assess whether this influences participants' self-perceived numeracy. Based on this sample population of accounting students, the findings may not apply to large populations of accounting-information users.Practical implicationsInvestors' ability to exercise sound judgement depends on the accuracy of their probability estimations. Manipulating the format of probability expressions can improve probability estimation performance in investors with low self-evaluated numeracy.Originality/valueThis study identified a significant performance gap among participants in performing accounting probability estimations: those with high self-evaluated numeracy performed better than those with low self-evaluated numeracy. The authors also explored a method other than additional training to improve participants' performance on probability estimation tasks and discovered that frequency formats enhanced the performance of participants with low self-evaluated numeracy.

Kustaanheimo–Stiefel Variables for Planetary Protection Compliance Analysis

Planetary protection in trajectory design aims to assess the impact probability of space-mission-disposed objects based on their initial uncertain conditions, to avoid contaminating other planetary environments. High-precision dynamic models and propagation methods are required to reach high confidence levels on small estimated impact probabilities. These requirements have so far confined planetary protection analyses to robust Monte Carlo–based approaches, using the Cartesian formulation of the full force problem dynamics. This work presents the improvements brought by adopting the Kustaanheimo–Stiefel (KS) formulation of the dynamics. The KS formulation is combined with reference frame switch procedures and adaptive nondimensionalization upon detection of close encounters. The fibration property of the KS space, namely, the parameterizable locus of point arising when mapping to a higher-dimensional space, is exploited to minimize the computational time, because of the minimized numerical stiffness of the system. Impact probability estimation tasks become more efficient than the single-simulation case, since the regularization of the dynamics removes the singularity of gravitational potentials for distances approaching zero. Despite an almost halved computational burden for Monte Carlo analysis, the precision of the single simulations is increased by nearly one order of magnitude, setting a new performance benchmark for planetary protection tasks.

Individual differences in the perception of probability.

In recent studies of humans estimating non-stationary probabilities, estimates appear to be unbiased on average, across the full range of probability values to be estimated. This finding is surprising given that experiments measuring probability estimation in other contexts have often identified conservatism: individuals tend to overestimate low probability events and underestimate high probability events. In other contexts, repulsive biases have also been documented, with individuals producing judgments that tend toward extreme values instead. Using extensive data from a probability estimation task that produces unbiased performance on average, we find substantial biases at the individual level; we document the coexistence of both conservative and repulsive biases in the same experimental context. Individual biases persist despite extensive experience with the task, and are also correlated with other behavioral differences, such as individual variation in response speed and adjustment rates. We conclude that the rich computational demands of our task give rise to a variety of behavioral patterns, and that the apparent unbiasedness of the pooled data is an artifact of the aggregation of heterogeneous biases.

Optimistic amnesia: how online and offline processing shape belief updating and memory biases in immediate and long-term optimism biases.

When people are confronted with feedback that counters their prior beliefs, they preferentially rely on desirable rather than undesirable feedback in belief updating, i.e. an optimism bias. In two pre-registered EEG studies employing an adverse life event probability estimation task, we investigated the neurocognitive processes that support the formation and the change of optimism biases in immediate and 24 h delayed tests. We found that optimistic belief updating biases not only emerged immediately but also became significantly larger after 24 h, suggesting an active role of valence-dependent offline consolidation processes in the change of optimism biases. Participants also showed optimistic memory biases: they were less accurate in remembering undesirable than desirable feedback probabilities, with inferior memories of undesirable feedback associated with lower belief updating in the delayed test. Examining event-related brain potentials (ERPs) revealed that desirability of feedback biased initial encoding: desirable feedback elicited larger P300s than undesirable feedback, with larger P300 amplitudes predicting both higher belief updating and memory accuracies. These results suggest that desirability of feedback could bias both online and offline memory-related processes such as encoding and consolidation, with both processes contributing to the formation and change of optimism biases.

O6.5. JUMPING TO CONCLUSIONS ABOUT DECISION NOISE? A COMPUTATIONAL ANALYSIS OF THE RELATIONSHIP BETWEEN BELIEF UPDATING AND PSYCHOTIC SYMPTOMS IN A LARGE UK BIRTH COHORT

BackgroundA number of studies show that people with psychotic disorders have abnormal belief-updating processes. In a commonly-used decision-making task, the beads task, participants infer which of two jars, each with a different ratio of coloured beads, a presented bead is drawn from, with an option to request further beads before reaching a decision. Previous studies suggest that people with psychotic symptoms request fewer beads (draws to decision; DTD) indicative of a ‘Jumping to conclusion’ (JTC) bias. In a modified version of this task, participants estimate the probability that beads have been drawn from one of the two jars on a sliding scale over a sequence of beads and are also told that the jar the beads are drawn from may switch. In this task, people with psychotic symptoms revise their estimations disproportionately in response to a change in colour of beads in a sequence (overadjustment bias).It is not clear what specific belief-updating processes drive these biases, how they arise, or if their association with psychotic symptoms is independent of confounding. We examined whether abnormal belief-updating processes are associated with psychotic experiences in a large, population-based sample, and whether they mediate the association between trauma and psychotic symptoms.MethodsWe used data from the Avon Longitudinal Study of Parents and Children birth cohort (n=2,879). Past-year frequent or distressing psychotic experiences (PEs) were assessed using the semi-structured PLIKS interview at age 24. Performance on the DTD and probability estimation tasks at age 24 were assessed using behavioural indices and computational modelling parameters (using ‘costed Bayesian’ and Hidden Markov Models respectively). Logistic regression was used to examine the association between belief-updating parameters (DTD task: cost of sampling, decision noise; Probability estimation task: adjustment rate, inference length, decision confidence, prior expectation of reversal, decision noise) and PEs. Estimates were adjusted for confounders (genetic risk for schizophrenia, socio-economic status, cognitive function). Mediation analysis tested abnormal belief-updating processes as a mediator between exposure to trauma (assessed ages 0–17 years) and age-24 PEs.ResultsIn the DTD task, increased decision noise was associated with PEs (adjusted OR=1.89, 95% CI: 1.14, 3.13, p=0.014). There was little evidence of an association between the JTC bias and PEs (OR= 1.13; 95% CI: 0.45, 2.82). For the probability estimation task, there was an association between a higher prior expectation that the jars that will switch during the sequence (expectation of reversal) and PEs (adjusted OR = 2.28; 95% CI 1.39, 3.74, p=0.001). Our findings were minimally attenuated by confounding (<10%). Exposure to trauma was also associated with greater decision noise in the DTD task, but there was little evidence that this abnormal belief-updating parameter mediated the relationship between trauma and PEs (<1% mediated).DiscussionOur results suggest that abnormal belief-updating processes (increased decision noise; greater prior expectation of reversal) are associated with PEs, and that this is not explained by general cognitive ability, shared genetic risk, or social background. Previous observations of association between the JTC bias and psychosis may be due to sub-optimal performance rather than a bias for making a decision on less evidence. The results also suggest that an increased expectation of change is associated with the early stages of psychosis symptom development. Our mediation result does not support the hypothesis that the belief-updating processes examined here lie on the causal pathway between trauma exposure and PEs.

Individual Differences in the Perception of Probability

In recent studies of humans estimating non-stationary probabilities, estimates appear to be unbiased on average, across the full range of probability values to be estimated. This finding is surprising given that experiments measuring probability estimation in other contexts have often identified conservatism: individuals tend to overestimate low probability events and underestimate high probability events. In other contexts, repulsive biases have also been documented, with individuals producing judgments that tend toward extreme values instead. Using extensive data from a probability estimation task that produces unbiased performance on average, we find substantial biases at the individual level, and we document the coexistence of both conservative and repulsive biases in the same context. Individual biases persist despite extensive experience with the task, and are also correlated with other behavioral differences, such as individual variation in response speed and adjustment rates. We conclude that the rich computational demands of our task give rise to a variety of behavioral patterns, and that the apparent unbiasedness of the pooled data is an artifact of aggregation of heterogeneous biases.

Optimal structure of metaplasticity for adaptive learning.

Learning from reward feedback in a changing environment requires a high degree of adaptability, yet the precise estimation of reward information demands slow updates. In the framework of estimating reward probability, here we investigated how this tradeoff between adaptability and precision can be mitigated via metaplasticity, i.e. synaptic changes that do not always alter synaptic efficacy. Using the mean-field and Monte Carlo simulations we identified ‘superior’ metaplastic models that can substantially overcome the adaptability-precision tradeoff. These models can achieve both adaptability and precision by forming two separate sets of meta-states: reservoirs and buffers. Synapses in reservoir meta-states do not change their efficacy upon reward feedback, whereas those in buffer meta-states can change their efficacy. Rapid changes in efficacy are limited to synapses occupying buffers, creating a bottleneck that reduces noise without significantly decreasing adaptability. In contrast, more-populated reservoirs can generate a strong signal without manifesting any observable plasticity. By comparing the behavior of our model and a few competing models during a dynamic probability estimation task, we found that superior metaplastic models perform close to optimally for a wider range of model parameters. Finally, we found that metaplastic models are robust to changes in model parameters and that metaplastic transitions are crucial for adaptive learning since replacing them with graded plastic transitions (transitions that change synaptic efficacy) reduces the ability to overcome the adaptability-precision tradeoff. Overall, our results suggest that ubiquitous unreliability of synaptic changes evinces metaplasticity that can provide a robust mechanism for mitigating the tradeoff between adaptability and precision and thus adaptive learning.

Probability estimation in poker: A qualified success for unaided judgment

AbstractPoker players make strategic decisions on the basis of imperfect information, which are informed by their assessment of the probability they will hold the best set of cards among all players at the conclusion of the hand. Exact mental calculations of this probability are impossible—therefore, players must use judgment to estimate their chances. In three studies, 69 moderately experienced poker players estimated the probability of obtaining the best cards among all players, based on the limited information that is known in the early stages of a hand. Although several of the conditions typically associated with well‐calibrated judgment did not apply, players' judgments were generally accurate. The correlation between judged and true probabilities was r &gt; .8 for over five‐sixths of the participants, and when judgments were averaged across players and within hands this correlation was .96. Players slightly overestimated their chance of obtaining the best cards, mainly where this probability was low to moderate (&lt;.7). Probability estimates were slightly too strongly related to the strength of the two cards that a player holds (known only to themselves), and insufficiently influenced by the number of opponents. Seemingly, players show somewhat insufficient regard for the cards that other players could be holding and the potential for opponents to acquire a strong hand. The results show that even when judgment heuristics are used to good effect in a complex probability estimation task, predictable errors can still be observed at the margins of performance. Copyright © 2009 John Wiley &amp; Sons, Ltd.

Probability estimations and delusion-proneness

The present study investigated the reasoning of high and low-delusion-prone individuals on probability estimation tasks using emotionally neutral and delusional narratives. Undergraduate students ( N = 33) who were classified as high-delusion-prone or low-delusion-prone based on their scores on a widely used measure of delusion-proneness in reasoning research (the Peters et al. Delusions Inventory) were asked to rate the probability that five neutral and eight unusual situations, presented in narratives that ranged in terms of degree of likelihood, could be true. Results indicated that compared to low-delusion-prone participants, high-delusion-prone participants assigned equivalent probability estimations to neutral narratives, but considered delusional narratives to be more likely ( p < .001). Differences emerged between groups at all probability levels for delusional narratives except the least likely. Findings support the presence of a reasoning bias among high-delusion-prone individuals for delusional material similar to that previously found among actively delusional individuals. Results are discussed as they relate to reasoning, psychosis and delusion-proneness.

Contrasting stochastic and support theory accounts of subadditivity

Numerous studies have found that likelihood judgment typically exhibits subadditivity in which judged probabilities of events are less than the sum of judged probabilities of constituent events. Whereas traditional accounts of subadditivity attribute this phenomenon to deterministic sources, this paper demonstrates both formally and empirically that subadditivity is systematically influenced by the stochastic variability of judged probabilities. First, making rather weak assumptions, we prove that regressive error (or variability) in mapping covert probability judgments to overt responses is sufficient to produce subadditive judgments. Experiments follow in which participants provided repeated probability estimates. The results support our model assumption that stochastic variability is regressive in probability estimation tasks and show the contribution of such variability to subadditivity. The theorems and the experiments focus on within-respondent variability, but most studies use between-respondent designs. Numerical simulations extend the work to contrast within- and between-respondent measures of subadditivity. Methodological implications of all the results are discussed, emphasizing the importance of taking stochastic variability into account when estimating the role of other factors (such as the availability bias) in producing subadditive judgments.

Sequential Observation and Selection with Rank-Dependent Payoffs: An Experimental Study

We consider a class of sequential observation and selection decision problems in which applicants are interviewed one at a time, decision makers only learn the applicant's quality relative to the applicants that have been interviewed and rejected, only a single applicant is selected, and payoffs increase in the absolute quality of the selected applicant. Compared to the optimal decision policy, which we compute numerically, results from two experiments show that subjects terminated their search too early. We competitively test three behavioral decision rules and find that a multithreshold rule, which has the same form as the optimal decision policy but is parameterized differently, best accounts for the data. Results from a probability estimation task show that subjects tend to overestimate the absolute quality of early applicants and give insufficient consideration to the yet-to-be-seen applicants.

Waiting for the bus: When base-rates refuse to be neglected

The paper reports the results from 16 versions of a simple probability estimation task, where probability estimates derived from base-rate information have to be modified by case knowledge. In the bus problem [adapted from Falk, R., Lipson, A., & Konold, C. (1994). The ups and downs of the hope function in a fruitless search. In G. Wright & P. Ayton (Eds.), Subjective probability (pp. 353–377). Chichester, UK: Wiley], a passenger waits for a bus that departs before schedule in 10% of the cases, and is more than 10 min delayed in another 10%. What are Fred’s chances of catching the bus on a day when he arrives on time and waits for 10 min? Most respondents think his probability is 10%, or 90%, instead of 50%, which is the correct answer. The experiments demonstrate the difficulties people have in replacing the original three-category 1/8/1 partitioning with a normalized, binary partitioning, where the middle category is discarded. In contrast with typical studies of “base-rate neglect”, or under-weighing of base-rates, this task demonstrates a reversed base-rate fallacy, where frequentistic information is overextended and case information ignored. Possible explanations for this robust phenomenon are briefly discussed.

Application of autonomous neural network systems to medical pattern classification tasks

This paper presents a study of the application of autonomously learning multiple neural network systems to medical pattern classification tasks. In our earlier work, a hybrid neural network architecture has been developed for on-line learning and probability estimation tasks. The network has been shown to be capable of asymptotically achieving the Bayes optimal classification rates, on-line, in a number of benchmark classification experiments. In the context of pattern classification, however, the concept of multiple classifier systems has been proposed to improve the performance of a single classifier. Thus, three decision combination algorithms have been implemented to produce a multiple neural network classifier system. Here the applicability of the system is assessed using patient records in two medical domains. The first task is the prognosis of patients admitted to coronary care units; whereas the second is the prediction of survival in trauma patients. The results are compared with those from logistic regression models, and implications of the system as a useful clinical diagnostic tool are discussed.

An Incremental Adaptive Network for On-line Supervised Learning and Probability Estimation

In this paper, a novel hybrid utilization of the Fuzzy ARTMAP (FAM) neural network and the Probabilistic Neural Network (PNN) is proposed for on-line learning and probability estimation tasks. There are two distinct advantages to the hybrid network. First, FAM is used as an underlying clustering algorithm to classify the input patterns into different recognition categories during the learning phase, resulting in a significant reduction in the number of pattern nodes required in the PNN. Second, a non-parametric posterior probability distribution estimation procedure, in accordance with the PNN paradigm (i.e. the Parzen-windows estimator), is employed during the prediction phase, where a probabilistic interpretation corresponding to Bayes decision theory can be provided for the predictions of FAM. In addition, several modifications are proposed to integrate both of the networks effectively into a unified platform for enhancing generalization. This hybrid approach also realizes an incremental learning system in which the necessity to specify a static network configuration a priori is eliminated as the network is able to “grow” to accommodate new input patterns sequentially and can thus operate in non-stationary environments. The performance of the network is evaluated with benchmark classification tasks and the results are compared with other approaches. Simulation results indicate that this hybrid network is capable of achieving a value near to the Bayes optimal classification rate. © 1997 Elsevier Science Ltd.

An explanation of conservatism in the bookbag-and-pokerchips situation

An experiment is described in which subjective probability revisions were obtained in a standard probability estimation task, the ‘bookbag-and-pokerchips’situation. Three aspects of probability revision were examined: conservatism, sequential effects, and coherence. Under two experimental conditions, the conservatism effect obtained was closely related to subjects' use of a simple strategy. A recency effect was also obtained. Coherence of the probability estimates was excellent. Conditions under which the observed strategy leads to conservatism are explored and previously published results are reconsidered in the light of this strategy. Conservatism in the bookbag- and-pokerchips situation is explained as an artefact of subjects' strategies

The importance of the data-generating model in probability estimation

When subjects appear to do poorly in a complex probability estimation task, they may be making careful estimates based on a different data-generating model than the one used by the experimenters. Eleven subjects estimated for 150 trials the probability that a dropped “bomb” had been aimed at one of two cities. Correlations between subjects' estimates and probabilities generated by the correct circular normal model and by a simple incorrect ratio of distances model showed six subjects using the wrong model with great accuracy. The usual analysis of such data, in which a possible confusion about the data-generation model is disregarded, would have led to the misleading conclusion that these subjects were very inaccurate at the task.