Designing Programs for Heterogeneous Populations: The Value of Covariate Information

Abstract

Normative judgments embodied in the American legal system mandate that, in certain respects, public policy should treat all members of the population uniformly. Nevertheless, the legal system permits many forms of disparate treatment of the population. The Medicare program provides health-care benefits to persons age 65 and older, but not to younger Americans. The federal welfare-to-work program known as TANF permits states to treat welfare recipients differentially, placing some in job-training and others in basic-skills classes. Judicial sentencing guidelines variously permit or require judges to sentence convicted offenders differentially based on past convictions. Public high schools track students, making class assignments vary with past student achievement. In these and other settings where legal constraints do not preclude disparate treatment, society may choose among many alternative treatment rules. A program could mandate uniform treatment of the population or require that treatment vary in particular ways with observable covariates of the persons treated (e.g., age in the case of Medicare, past convictions in the case of sentencing), or permit agents of society (e.g., judges, welfare case managers, school counselors) to make their own treatment choices, subject to specified constraints. Research on program evaluation can help to inform public policy through efforts to learn the consequences of alternative treatment rules. In particular, evaluation research should seek to characterize how treatment response varies across the population. If we learn that all persons respond to treatment in much the same manner, then the best policy may be one that treats all persons uniformly. However, if we learn that treatment response varies with observable covariates of the persons treated, then society may be able to do better by designing programs in which treatment varies appropriately with these covariates. For example, society may be able to lower recidivism among criminal offenders by sentencing some offenders to prison and others to probation. It may be able to increase the life-cycle earnings of welfare recipients by placing some in job-training and others in basic-skills classes. In these and many other cases, the key to success is to determine which persons should receive which treatments. Regrettably, evaluation research has had little to say about how treatment response varies with observable covariates of the persons treated. A common practice, especially in observational studies, has been to assume that all persons respond to treatment in the same manner. Studies that are sensitive to possible variation in treatment response may report findings by race or gender or age, but they rarely disaggregate the population more finely. As a consequence, policymakers seeking to design programs for heterogeneous populations have to speculate on the consequences of alternative treatment rules. This short article draws on my recent research (Manski, 1997, 2000a, b) to argue that increased attention to observable variation in treatment response would enhance the value of evaluation research.