A Procedure to Link Evaluation and Funding Decisions

Abstract

Davis and Salasin (1975) have summarized many of the issues in the use of evaluation results, including statements by evaluators that their findings are not used and those by administrators that evaluation findings are not available when decisions have to be made. Although there is much discussion of the need to relate evaluation and decision making, there have been few efforts to specify the manner in which this might occur. Edwards, Guttentag, and Snapper (1975) proposed and applied a method called multiattribute utility measurement to assist the Office of Child Development in defining the major dimensions of importance in developing priorities for funding research projects. The multiattribute utility measurement method is one of a set of methods classified as decision aids by Slovic, Fischoff, and Lichtenstein (1977), as opposed to formal behavioral decision theory models. One framework for the present study is provided by the procedures used to aid decisions. A second framework is the problem of defining for the diverse set of programs funded under the basic grants given in the Vocational Education Act of 1976 (VEA). Bernstein and Freeman (1975) define impact evaluation as movement or change toward the desired objectives of a project, that is, prespecified, operationally defined goals and criteria of success. In the current work, these definitions are established externally to the individual projects. Projects will have varying goals and any one project may not meet all the priorities of the funding program. By operationally defining the set of variables that define impact, developing a scale or distribution of these variables, and summing scale values for each grant application or funded project, individual projects can be evaluated as high or low impact projects. A related study has been conducted by Ory, Harris, Dueitt, and Clark (1978). They developed and field tested a vocational education evaluation model for programs at the community college level, based on subjectively and objectively derived data. Weights for six criteria were derived from paired comparison ratings by state and local educators, legislators, college The research reported herein was supported under VEA Grant No. 78-3A-540 of the Division of Occupational Education Supervision of the New York State Education Department to the Institute for Research and Development in Occupational Education of the Center for Advanced Study in Education of the Graduate Center of the City University of New York.