An exposition of the false confidence theorem

Abstract

A recent paper (Balch et al. (), ‘Satellite conjunction analysis and the false confidence theorem’, arXiv 5 preprint arXiv:1706.08565.) presents the “false confidence theorem,” which has potentially broad implications for statistical inference using Bayesian posterior uncertainty. This theorem says that with an arbitrarily large (sampling/frequentist) probability, there exists a set that does not contain the true parameter value but has an arbitrarily large posterior probability. As the use of Bayesian methods has become increasingly popular in applications of science, engineering and business, it is critically important to understand when Bayesian procedures lead to problematic statistical inferences or interpretations. In this paper, we consider a number of examples demonstrating the paradoxical nature of false confidence to begin to understand the contexts in which the false confidence theorem does (and does not) play a meaningful role in statistical inference. Our examples illustrate that models involving marginalization to non‐linear, not one‐to‐one functions of multiple parameters play a key role in more extreme manifestations of false confidence. © 2018 John Wiley & Sons, Ltd.