Uncertainty in Bayes.

Abstract

Most standard discussions of Bayes' formula treat sensitivity, specificity, and the prior probability as fixed parameters for probability revision, but in fact these usually have associated variability. This in turn generates predictable patterns of uncertainty in the posterior probabilities. Although these have not been investigated in detail, they have important implications for the interpretation of posterior probabilities and the clinical use of Bayesian probability revision. For a test with a high likelihood ratio for a positive result, the positive predictive value (PPV) is strongly affected by uncertainties in the prior probability when the prior probability is small, but PPV is almost independent of such uncertainties at high values of the prior probability. The PPV is more affected by changes in specificity than by changes in sensitivity, and uncertainty in specificity has its maximal impact on the PPV at low prior probability values. These patterns are most pronounced for tests with high likelihood ratios of positive results. Similar results can be shown for the negative predictive value. These results imply that for suitability good tests, probability revision in certain definable ranges of prior probability may be so strongly affected by errors in the estimations of both the prior probability and the operating characteristics that the posterior probabilities may be unstable in practice. On the other hand, at other values of the prior probability, the posterior probabilities are almost constant, and formal probability revision will not have much impact. These patterns indicate limitations to the reliability and usefulness of calculated posterior probabilities, and have important implications for the clinical use of Bayes' formula.