Exact Properties of Some Exact Test Statistics for Comparing Two Binomial Proportions

Abstract

Abstract Using exact unconditional distributions, we evaluated the size and power of four exact test procedures and three versions of the χ2 statistic for the two-sample binomial problem in small-to-moderate sample sizes. The exact unconditional test (Suissa and Shuster 1985) and Fisher's (1935) exact test are the only tests whose size can be guaranteed never to exceed the nominal size. Though the former is distinctly more powerful, it is also computationally difficult. We propose an alternative that approximates the exact unconditional test by computing the exact distribution of the χ2 statistic at a single point, the maximum likelihood estimate of the common success probability. This test is a modification of the test of Liddell (1978), which considered the exact distribution of the difference in the sample proportions. Our test is generally more powerful than either the exact unconditional or Liddell's test, and its true size rarely exceeds the nominal size. The uncorrected χ2 statistic is frequently a...