Model-Based Inference for Completely Randomized Experiments

Abstract

INTRODUCTION As discussed in Chapters 5 and 6, both Fisher's and Neyman's approaches for assessing treatment effects in completely randomized experiments viewed the potential outcomes as fixed quantities, some observed and some missing. The randomness in the observed outcomes was generated primarily through the assignment mechanism, and sometimes also through random sampling from a population. In this chapter, as in the preceding chapter on regression methods, we consider a different approach to inference, where the potential outcomes themselves are also viewed as random variables, even in the finite sample. Because all of the potential outcomes are considered random variables, any functions of them will also be random variables. This includes any causal estimand of interest – for example, the average treatment effect or the median causal effect. We begin by building a stochastic model for all potential outcomes that generally depends on some unknown parameters. Using the observed data to learn about these parameters, we stochastically draw the unknown parameters and use the postulated model to impute the missing potential outcomes given the observed data, and use this in turn to conduct inference for the estimand of interest. At some level, all methods for causal inference can be viewed as imputation methods, although some more explicitly than others. Because any causal estimand depends on missing potential outcomes, any estimate for such an estimand is, implicitly or explicitly, based on estimates of these missing potential outcomes. The discussion in the current chapter puts this imputation perspective front and center. Because the imputations and resulting inferences are especially straightforward from a Bayesian perspective, we primarily focus on the Bayesian approach, but we also discuss the implementation of frequentist approaches, as well as how the two differ. This model-based approach is very flexible compared to the Fisher's exact p-value approach, Neyman's repeated sampling approach, or regression methods. For instance, this method can easily accommodate a wide variety of estimands – we may be interested not only in average treatment effects but also in quantiles, or in measures of dispersion of the distributions of potential outcomes.