Testing the Effect of Treatment in Experiments with Correlated Binary Outcomes

Abstract

This paper considers the problem of testing for treatment effect in a randomized experiment with correlated binary outcomes, representing success or failure for different "parts" of a randomized unit. Attention is restricted to tests that are based on a summary score for each individual randomized, and thus are valid regardless of the precise nature of the correlation among parts. The focus is on the efficiency of such tests under various correlation structures, with special emphasis on the case in which the correlation among parts within an individual differs across treatment groups. A class of summary score statistics is defined, and optimal testing is discussed for some simple situations. Three potential general-purpose tests also are described: (1) the ratio estimate test discussed by Henderson et al. (1988, Controlled Clinical Trials 9, 189-205); (2) a modified ratio estimate test with adjusted weighting based on the within-individual correlation between parts; (3) a test defined by applying the Mantel-Haenszel procedure to the proportion of individuals with at least one failure, stratifying by the number of parts. For these general-purpose tests, numerical calculations of asymptotic efficiency are presented under a wide range of designs and correlation structures. On the basis of these results, some practical recommendations for choosing a test are made.