Tests for the homogeneity of two binomial proportions in extremely unbalanced 2 × 2 contingency tables

Abstract

Asymptotic tests such as the Pearson chi-square test are unreliable for testing the homogeneity of two binomial probabilities in extremely unbalanced cases. Two exact tests (conditional and unconditional) are available as alternatives and can be implemented easily in StatXact 6.0. In equal sample cases it is well known that the unconditional exact test is more powerful than the conditional exact test. However, in this paper, we show that the opposite result holds in extremely unbalanced cases. The reason is that the peaks of the type I error occur at the extremes of the nuisance parameter when the imbalance among the sample sizes becomes severe. After we show that the conditional exact test is more powerful than the unconditional exact test in extremely unbalanced cases whose sample ratio is greater than 20, we compare the conditional exact test with the Berger and Boos approach (J. Amer. Stat. Assoc. 1994; 89:1012-1016) in which the supremum is taken over a confidence interval for the nuisance parameter. The Berger and Boos approach turns out to be slightly more powerful than the conditional exact test in extremely unbalanced data. A real example is provided.