Optimality of the Mid P-Value for Testing Marginal Homogeneity in Binary Matched Pairs Data

Abstract

This paper considers the problem of testing marginal homogeneity in binary matched pairs data, which can be viewed as a sample from a 2×2 multinomial trial. We apply the estimated truth approach in Hwang and Pemantle (1997) and Neyman-Pearson theory to derive the (two-sided) optimal p-value, namely the expected p-value. The expected p-value can be shown to be identical to the mid p-value, which is defined as half the conditional probability of observed McNemar's test statistic plus the conditional probabilities of more extreme values. Under the squared error loss, the mid p-value and McNemar's p-value, derived from McNemar's test, have better risk performance among all p-values we consider. Numerical evidence shows that the mid p-value leads to the test which has the (actual) significance level very close to the nominal level for small to moderate sample sizes. Moreover, numerical calculations for small sample sizes reveal that McNemar's P-value is anticonservative since its actual significance level is much larger than the nominal level.