On Rescuing the Nonequivalent-Control-Group Design

Abstract

The nonequivalent-control-group design is important because true experimental designs are frequently either infeasible or undesirable and other quasi-experimental designs have only quite limited applications. This design, however, has been disparaged as nearly useless because it depends upon statistical methods that may give biased results when applied to it. The design is too important to let lie in this condition. It is suggested that slight modifications, a few of which have been offered by others, might render it more reliable. One such modification, quite simple, widely applicable, and highly restorative of internal validity, is suggested here. The bias associated with the standard design is presented as resulting from basic violations of the assumptions of statistical methods. Both reduction of the bias and estimation of its extent are shown to be possible if the comparison group is selected at random from the relevant population and used by itself, rather than in conjunction with the experimental group, for the preliminary estimation of parameters. The modified design is shown to have various advantages relative to its unmodified form and even, at times, relative to true experimental designs. A secondary purpose of this article, supportive of the first, is to clarify the analysis of evaluation designs by conceptualizing the issues in terms of ordinary least-squares regression.