The pseudodiagnosticity trap: Should participants consider alternative hypotheses?

Abstract

The pseudodiagnosticity task has been used as an example of the tendency on the part of participants to incorrectly assess Bayesian constraints in assessing data, and as a failure to consider alternative hypotheses in a probabilistic inference task. In the task, participants are given one value, the anchor value, corresponding to P(D1|H) and may choose one other value, either P(D1|¬!H), P(D2|H), or P(D2|not;!H). Most participants select P(D2|H), or P(D2|¬!H) which have been considered inappropriate (and called pseudodiagnostic) because only P(D1|¬!H) allows use of Bayes' theorem. We present a new analysis based on probability intervals and show that selection of either P(D2|H), or P(D2|¬!H) is in fact pseudodiagnostic, whereas choice of P(D1|¬!H) is diagnostic. Our analysis shows that choice of the pseudodiagnostic values actually increases uncertainty regarding the posterior probability of H, supporting the original interpretation of the experimental findings on the pseudodiagnosticity task. The argument illuminates the general proposition that evolutionarily adaptive heuristics for Bayesian inference can be misled in some task situations.