382: A Note on the Chi-Square Test for Pair-Matched Samples

Abstract

SUMMARY In a departure from previous comparisons of the relative efficiency of pair-matched with independent samples, this note considers matching as a method of bias elimination and compares it with an alternative method- stratification of independent samples. The sampling model is also modified to recognize the loss of information generally incurred in the matching process. The appropriate chi-square tests for both methods of bias removal are compared analytically, subject to restrictions on sample size, and more generally using Monte Carlo techniques. For the situations considered, it appears that pair-matching does not give a more powerful test than that for stratified samples. In a recent paper concerning the analysis of pair-matched data with a dichotomous response, Pike and Morrow [1970] compared the approximate chi-square test statistic for matched pairs with that for independent samples of the same size and derived essentially the same results as Worcester [1964]. The conclusion in both papers was that the matched pairs statistic would only exceed the value of the statistic for independent samples provided the correlation between responses in the different populations was high-implying an excess of like response pairs of the type (0, 0), (1, 1). Another way of viewing this result is to consider random pair-matching of independent samples from each population. Let Yi denote the response variable for the ith population, and P(Yi = 0) = pi , i = 1, 2. Then with random matching we can expect an excess of like response pairs only when pi < p2 < 0.5 or when 0.5 < pi < P2 (with p2 chosen arbitrarily to exceed pi). Clearly, matching is not random in practice, but is usually performed within discrete categories representing the distribution of a covariate (or the joint distribution of two or more covariates), which is (are) correlated with the response in both populations. However, even in this situation, with the pi now conditional on the covariate distribution the argument holds, regardless of the strength of correlation between response and covariate. One criticism of this finding by Pike and Morrow and Worcester is that the comparison of pair-matched with independent samples is somewhat unsatisfactory. In observational studies, in which random assignment to treatment is not possible and one is concerned primarily with removing undesired effects, this comparison is essentially between one method of bias elimination and no elimination at all. A comparison could more appropriately be made between a matched pairs test and a test for partial association from stratified, independent samples. Moreover, given the almost inevitable reduction of initial sample size during the matching process, this loss of information should also be recognized in the comparison of tests for these two study designs. The extent of such losses has been discussed fully in an earlier paper (McKinlay [1974]). 731