Can Propensity-Score Methods Match the Findings from a Random Assignment Evaluation of Mandatory Welfare-to-Work Programs?

Abstract

This paper assesses nonexperimental estimators using results from a six-state random assignment study of mandatory welfare-to-work programs. The assessment addresses two questions: which nonexperimental methods provide the most accurate estimates; and do the best methods work well enough to replace random assignment? Three tentative conclusions emerge. Nonexperimental bias was larger in the medium run than in the short run. In-state comparison groups produced less average bias than out-of-state comparison groups. Statistical adjustments did not consistently reduce bias, although some methods reduced the estimated bias in some circumstances and propensity-score methods provided a specification check that eliminated some large biases.