Evidence of an Output Bias in the Judgment of Public Performance: A Replication and Extension

Abstract

Despite calls for an evidence-based focus on outcomes as a way to enhance accountability for public performance, findings from a prior study suggest that the public may be more impressed by high frequency (low cost) but ambiguous outputs (such as people served) rather than more meaningful but costly outcomes (causal effects). We attempt to replicate and extend the investigation of this output bias through a pair of survey experiments involving judgments about two evidence-based, highly effective social programs: one, an HIV/AIDS prevention program (adapted from the prior study), the other, a program for special needs high school students (Check and Connect). Our findings confirm that respondents viewed both programs more favorably when given information about mere outputs (people served) in comparison with more rigorous outcomes (causal effects). We then tested an extension of the Check and Connect experiment in which we modified the framing of cost and performance information in ways that reduced the tendency toward an output bias. We speculate on the possible mechanism that may lead to an output bias, and we discuss the implications of our findings for evidence-based public policy and management.