Evidence and Policy and Naming and Shaming

Abstract

The debate on the potential of evidence-based policy is now in full flow, though opinions differ on whether we have reached zenith or nadir in our capacity to transform public data into public service. Clearly, we are not short of 'evidence'; there is a plethora of programme evaluators and policy analysts out there, and a surfeit of their publications - official, academic and grey. What is at issue is the method of synthesizing that evidence, a research strategy that goes by the name of 'systematic review'. In particular, there is considerable doubt about how to handle the variegated and contested evidence base that typifies most sectors of public policy. Can it ever be made to yield to the formal statistical techniques of meta-analysis? And if not, can evidence-based policy ever match the precision of the demanding blueprint represented by evidence-based medicine? This paper seeks to go beyond these old antagonisms by proposing a novel model of evidence synthesis based on the principles of realist evaluation. Under this approach, systematic review changes from a data-driven exercise to a process of theory development and refinement. The core of the paper is a demonstration project, consisting of an embryonic review of public disclosure initiatives, a policy better known as 'naming and shaming'. Six initiatives are examined in order to develop an elementary theory of the conditions under which shaming sanctions might work. The paper concludes with reflections on how such a process of 'realist synthesis' may feed into policy making.