A Broader Template for Analyzing Broken Randomized Experiments

Abstract

Randomized experiments suffering from missing data and noncompliance are a recurring problem for experimenters whose subjects are human. Until recently, analysts of such broken randomized experiments were largely forced to squeeze the data into the idealized template of the randomized experiment with neither noncompliance nor missing data. Such practices necessitate throwing away information and making strong, and often unwarranted, assumptions. The Milwaukee Parental Choice Program, a natural experiment, is used to illustrate the flexibility of a new template, which allows for missing data and certain forms of simple noncompliance. The generality of this new template, which is based on a formulation of causal effects called the Rubin causal model, is contrasted with existing alternatives. The multiple imputation technology needed to proceed with analyses from the framework of this template is briefly described, and technical aspects will be presented in depth in subsequent work.