The contribution of policy design to realist evaluation

Abstract

The literature on policy design provides insightful resources to improve policy evaluation. We compare how this literature addresses causation, evaluation, instrumentation, and intervention, based on different methodologies. Then, we explain how the realist approach can contribute to theory-based evaluation with multi-methods. We go through the process leading from the adoption of policy aims to the production of a policy outcome by a causal mechanism linking agenda setting to policy formulation, cross-sectorial coordination, political interplays, and this policy’s outcome. This hypothesis is then tested against a typology of policy instruments classified by the State resources. These instruments are hereby treated as expected empirical observations that provide an empirical basis to assess the theoretical process following a Bayesian logic. We eventually present a five-step protocol aimed at making realist policy evaluation easier, which includes process theorization and operationalization, empirical tests design, case selection and justification, congruence analysis, and deep within-case study.