Statistical adjustments and uncontrolled studies.

Abstract

Many evaluations of social interventions are based on uncontrolled assignments of individuals to treatment groups. Statistical adjustments are often used to compensate for naturally occurring differences between groups. There is much confusion and controversy about the adequacy of these statistical methods. A variety of interrelated problems have been identified, including measurement error, unequal growth rates across groups, and regression artifacts. In this article it is shown that these problems can all be subsumed under a general conceptual framework, as particular examples of model misspecincation. This perspective is helpful in revealing clearly the nature of the problems posed by lack of experimental control. The important case of linear adjustment (analysis of covariance) is given special attention. An expression is derived for the proportion of bias remaining after adjustment, in terms of easily interpretable parameters. Implications of these results for research and evaluation design are considered.