Design Issues for Evaluations of Community Care Demonstrations

Abstract

The evaluation of community care demonstrations poses a number of interdependent program and methodological design choices. Whether or not the program to be tested has a system-wide mission, for example, determines whether the program should be implemented on a small or large scale. This in turn has important implications for whether a randomized or a comparison site design is the appropriate evaluation methodology. This paper examines the relation between system-wide mission and scale considerations, on the one hand, and appropriate choice of methodology, on the other. It concludes that a small scale program with a randomized evaluation design is easiest to implement, costs less, and is more likely to provide defensible answers to a limited set of policy questions than a large scale program with a system-wide mission evaluated using a comparison site methodology. Despite the greater difficulty and cost of a large scale demonstration, however, such a demonstration should be seriously considered whenever increased funding makes it a feasible option, because it is able to address a number of significant policy questions that cannot be answered within the constraints of a small scale demonstration.