Interval Estimation of Treatment Effects In Randomized Trials: When do Confidence Intervals Have Nominal Coverage?

Abstract

SummaryIn the classical theory of randomized trials (RTs), the treatment effect is defined as a mean difference of potential outcomes. To achieve nominal coverage probability for confidence intervals (CIs) on treatment effects in RTs, certain assumptions are necessary. Specifically, one must either make assumptions about the joint distribution of potential outcomes or enroll subjects in the trial by random sampling of the target population on which the treatment effect is defined. In practice, no such sampling usually takes place and assumptions about the joint distribution of potential outcomes cannot be verified based on observed data. Furthermore, the most common of these assumptions, such as treatment‐unit additivity (TUA) or independence are biologically implausible in most RTs involving human subjects. Hence, it is not usually possible to construct CIs on treatment effects with nominal coverage probability. However, for any joint distribution of potential outcomes, the standard estimator of the variance of the difference of two independent sample means produces CIs with asymptotic coverage at least at the nominal level. This interpretation of CIs as conservative bounds may not always hold in conventional regression models applied to RT data.