Logical coherence in Bayesian simultaneous three-way hypothesis tests

Abstract

Hypothesis testing is a key component in Statistics. In practice, it is common for a practitioner to test several hypotheses, the so-called simultaneous hypothesis testing problem. Unfortunately, simultaneous hypothesis tests can be logically incoherent: for instance, if hypothesis H1 implies hypothesis H2, a procedure that rejects H2 should also reject H1, a property not always met by multiple test procedures. Indeed, previous results show that standard two-way hypothesis tests cannot be logically coherent. Three-way tests allow more nuanced data-based decisions. This paper studies whether Bayesian simultaneous three-way hypothesis tests can be logically coherent. Two types of results are obtained. First, under the standard error-wise constant loss, only for a limited set of models a Bayes simultaneous test can be logically coherent. Second, if more general loss functions are used, then it is possible to obtain Bayes simultaneous tests that are always logically coherent. An explicit example of such a loss function is provided. These results provide guidelines on how to build a logically coherent posterior probability three-way hypothesis test or more general Bayesian three-way hypothesis tests.