Sensitivity of Treatment on Treated Effects in the Housing Vouchers Welfare Experiment to Alternative Measures of Compliance

Abstract

In the social experimentation literature, the treatment-control outcome difference is the “intent to treat” and the adjustment of that difference to reflect actual participation in treatment as the “treatment on treated” (TOT) effect of the intervention. Previous contributions to this literature have been silent on the sensitivity of TOT to alternative definitions of treatment. In this paper, we apply alternative methods of estimating treatment-on-the-treated to data from the Welfare-to-Work Housing Voucher experiment. The final report on that experiment employs an original method of calculation of TOT, and finds that early negative impacts on earnings fade out after 1.5-2 years. We test for sensitivity of these results to alternative concepts of participation: participation at time of measurement; exposure to treatment over time; definition of the intervention as housing assistance per se, rather than vouchers. We also test for sensitivity to assumptions about the effects of program exposure over time. We find that the published TOT results are qualitatively robust to the definition of treatment. We believe this finding is likely to apply more generally in large, well-controlled experiments.