Why Are Counterfactual Assessment Methods Not Widespread in Outcome-Based Contracts? A Formal Model Approach

Abstract

Abstract Counterfactual assessment techniques involving treated and control groups, such as randomized control trials, might be used in outcome-based contracts to avoid rewarding or sanctioning service providers for social outcomes that they did not cause. However, few outcome-based contracts adopt payment rules based on counterfactual assessment techniques. Potential explanations are that these techniques are complex and involve substantial transaction costs. In this paper, we develop a theoretical formal model that integrates the literatures of incentives and policy evaluation to propose the following alternative explanation: counterfactual techniques may lead to counterproductive incentive effects if they reduce the likelihood of payment even if project managers exert sufficient effort to promote the expected interventions. Our model shows that counterfactual assessment may undermine effort when the number of treated subjects is small and there is limited investment per treated subject. Our formal model also suggests that the increased experience of the contract sponsors may inhibit the adoption of counterfactual assessment. Simulations and descriptive evidence from a unique database of 350 outcome-based contracts designed or initiated throughout the world and from linear probability models are aligned with our predictions. By offering additional explanations on why counterfactual assessment methods are not widespread in outcome-based contracts and by identifying the boundary conditions under which these methods are used in incentive contracts, this work informs the literature on cross-sector outcome-based contracts and illustrates the use of formal models to develop novel theories in public administration.