Rewarding for Causal Impact: Integrating Policy Evaluation and Incentive Alignment in the Design of Cross-Sector Contracts

Abstract

Despite the emergence of contracts between governments and private actors involved in public services, a critical issue is how to enforce payments based on outcomes such as student learning or prisoner recidivism. Imperfect measurement may fail to reflect changes in outcomes that were caused by superior managerial effort. A possible way to address this problem is to adopt policy evaluation techniques that use control groups to examine what would have happened without the intervention, as is done in randomized controlled trials (RCTs) and other methods of counterfactual assessment. However, our database of outcome-based contracts launched or in development worldwide shows that the majority of contracts (70%) simply assess changes in outcomes measured in the treated population. In this paper we bridge the literatures of contractual incentives and policy evaluation to explain this puzzle. We develop a theoretical model to compare payment rules based on the assessment of the treated to payment rules based on a comparison between treated and control groups. Building on the idea of statistical power, we show that the later contracts may under certain conditions undermine effort. This occurs, for instance, when the number of treated subjects is small and there is limited investment per treated subject, which is associated with potential effect size. Preliminary evidence from our database of outcome-based contracts is consistent with our predictions. Our results suggest that under certain conditions RCTs and other evaluation techniques may not be optimal to reward for social outcomes, even though they are considered best practice to assess policy interventions.