Evaluating Treatment Protocols Using Data Combination

Abstract

Listen

Translate article

In real-life, individuals are often assigned by external planners to binary treatments. Taste-based allocation by planners would make such assignments productively inefficient in that the expected returns to treatment for the marginal treatment recipient would vary across covariates and be larger for discriminated groups. This cannot be directly tested if a planner observes more covariates than the researcher, because then the marginal treatment recipient is not identified. We present (i) a partial identification approach to detecting such inefficiency which is robust to selection on unobservables and (ii) a novel way of point-identifying counterfactual distributions needed to calculate treatment returns by combining observational datasets with experimental estimates. Our methods can also be used to (partially) infer risk-preferences of the planner, which can rationalize the observed data. The most risk neutral solution may be obtained via maximizing entropy. We illustrate our methods using survival data from the Coronary Artery Surgery Study which combined experimental and observational components. Such data combination can be useful even when outcome distributions are partially known. Collecting such data is no harder than running field experiments and its use is analogous to using validation data for measurement error analysis. Our methods apply when individuals cannot alter their potential treatment outcomes in response to the planner's actions, unlike in the case of law enforcement.