Towards data-driven project design: Providing optimal treatment rules for development projects

Abstract

It is increasingly commonplace among development practitioners to employ impact evaluations to measure the performance and effectiveness of their activities and investments. However, there remains a substantial gap between the evidence generated by these assessments and its use for prospective planning and design of future development projects. Specifically, the retrospective data generated in counterfactual-based evaluations on what worked, how, and for whom, is not routinely or easily applied by policymakers for decision-making after an evaluation's closure. We address this gap through the development of an optimal policy learning (OPL) tool for rural development projects that leverages observational data to drive data-driven project design through identification of welfare-maximizing targeting and selection rules that can maximize project impacts. In so doing, we solve the policymaker's policy assignment problem, i.e. deciding who to treat and where. Further, we define distinct roles for the policymaker and the analyst in which the latter is tasked with generating a menu of potential selection rules while the former weighs each rule's costs and benefits against their objective function, addressing the practical constraints poised by optimal policy learning's use for project design. To illustrate the utility of our approach we apply OPL to two projects funded and evaluated by the International Fund for Agricultural Development (IFAD). We show that OPL and this division of labor not only identifies the welfare-maximizing policy assignment but also allows policymakers to gain deeper insights into the trade-offs, costs, and benefits of different objectives, policies, and demands facilitating more informed decision-making and more effective policies and development interventions.