Is Realist Evaluation a Realistic Approach for Complex Reforms?

Abstract

The realist evaluation (RE) approach is a promising contribution to the literature on theory-driven evaluation.Thus far, RE has mainly been applied to the evaluation of human service programmes, but the aim of this article is to investigate if it can be extended to large-scale public sector reforms, in casu: the reform of the electricity market in Denmark. Five areas of regulation under the reform were selected and hypotheses on the context, mechanism and outcome (CMO) for each intervention were formulated. The evaluation of three of these interventions is summarized: opening markets to foreign trade, price regulation of monopolies and consumer protection. The article concludes that the CMO framework is applicable not just to interventions at the programme level, but also to large-scale public sector reform characterized by high technical complexity and strong policy coalitions. However, it only serves as a general framework, which has to be substantiated.This is done, first, by generating theories — i.e. programme theories — on each specific intervention. Second, theories are produced at the meso level to identify the institutional patterns that function as conditions for the effects to be obtained.