How should Program Evaluation Standards inform the use of cost-benefit analysis in evaluation?

Abstract

Background: Cost-benefit analysis (CBA), like any other evaluation method, should be used in ways that uphold program evaluation standards and should be subjected to metaevaluation. In contrast to the broad remit of program evaluation standards, guidelines for economic evaluation focus mainly on technical aspects of evaluation quality, aimed at ensuring precision, accuracy, and reliability. Can CBA be conducted in adherence both to program evaluation standards and to its own methodological principles, or are there areas where expectations conflict? Purpose: Assess the potential for CBA to be conducted in keeping with the Program Evaluation Standards (PES) of the Joint Committee on Standards for Educational Evaluation. Setting: Analysis applies to any setting in which CBA is being considered as an evaluation method. Intervention: N/A Research Design: Methodological principles underpinning CBA were systematically assessed against the PES, to determine the extent to which CBA can be conducted in a manner aligned with these standards. CBA was rated according to whether it can follow each standard in principle, not the extent to which economists follow a given standard in practice. Data Collection and Analysis: This assessment was undertaken from a theoretical perspective, through analysis of relevant literature. The ratings are evaluative; they represent the judgments of the author, made on the basis of explicit definitions. Findings: Some ethical principles espoused in the PES are also required in CBA. On the other hand, some of the PES are not explicit requirements in CBA, though they could be applied by evaluators or economists when conducting a CBA. However, some PES logically cannot be met by CBA if it is used as a stand-alone method. All PES can theoretically be met when an evaluation combines CBA with other methods. In order to use CBA in adherence to PES, evaluators and economists must take an explicit interest in the effects of their analysis on people’s lives. This has significant implications for the way CBA should be used, including the nature and extent of stakeholder involvement, the potential use of CBA in conjunction with other methods, and decisions about when not to use CBA. As with any evaluation method, deliberation is necessary over whether, when, and how to use CBA. Keywords: cost-benefit analysis, program evaluation standards, metaevaluation