Bayesian tests for composite alternative hypotheses in cross-tabulated data

Abstract

We present a methodology for constructing significance tests for “difficult” composite alternative hypotheses that have no natural test statistic. We apply our methodology to construct exact tests for cross-tabulated data, and our motivating example is constructing a test for discovering Simpson’s Paradox. Our tests are Bayesian extensions of the likelihood ratio test; they are optimal with respect to the prior distribution and are also closely related to Bayes factors and Bayesian FDR controlling testing procedures.