Causal Inference in Block-Randomized Experiments: Analysis Based on Neyman’s Stochastic Causal Model

Abstract

In Neyman’s causal model (NCM), each unit included in a two-arm randomized experiment has a pair of potential outcomes – one outcome would be observed under treatment, the other under control. In the stochastic version of NCM, both potential outcomes are viewed as (possibly) non-degenerate random variables to allow for stochastic effects of post-interventional factors, such as random measurement error. The unit-level treatment effect is the expected outcome under treatment minus that under control. The sample average treatment effect (SATE) is the mean of the unit-level effects in the set of all units included in the experiment. The population average treatment effect (PATE) is the mean of the unit-level effects defined on the population of all units eligible for experiment. The purpose of this paper is to develop a non-parametric theory of inference about SATE and PATE in block-randomized experiments, using the mathematical formalism of stochastic NCM. Inference about SATE is examined under randomization distribution without a probability model for selection of units into experiment. For inference about PATE two probability models for selection are considered: (1) simple random sampling and (2) stratified random sampling, each followed by block-randomized treatment assignment. It is shown that under these conditions, the ordinary “difference of means” estimator (mean observed outcome under treatment minus that under control) is consistent for both SATE and PATE. The variance of this estimator is derived under randomization distribution alone and under simple random sampling and stratified random sampling as selection models. Variance estimators producing confidence intervals with nominal asymptotic coverage for PATE under these selection models are proposed, and it is also shown that these intervals have at least nominal asymptotic coverage for SATE under randomization distribution. Thus, when a selection model is correctly specified, the proposed methods guarantee valid inference about both SATE and PATE, while under incorrect specification of the selection model valid inference about SATE is still guaranteed. This is an important property because in many types of randomized experiments (e.g., clinical trials in medicine), units are not selected into experiment by a mechanism with known selection probabilities.