Evidence and policy: interpreting the Delphic oracle

Abstract

So wrote Edwin Chadwick more than a hundred years ago. Not only was he one of the pioneers of the public health movement, equally, he was the 19th century father of much 20th century social inquiry. To cite him is to underline that there is nothing new—except, possibly, the phrase itself— about evidence-based policy (EBP). As Royal Commissions and government committees of inquiry have demonstrated over the decades, policy makers have always sought evidence. The extent to which subsequent policy decisions were actually based on that evidence, as distinct from the use of such evidence to legitimate them, is another question: Chadwick himself has often been criticized for manipulating evidence to support his preconceived ideas, notoriously so in the case of the 1834 Poor Law Report. So why the sudden rush of enthusiasm for EBP today? One answer must be that it reflects the success of the evidence-based medicine (EBM) movement (and this is a ‘movement’ insofar as it has prophets, missionaries and zealots). If medicine can be based on evidence, so surely can policy. The proposition seems self-evident. The trouble is that a sleight of hand is involved in making the transition. EBM is distinguished by the fact that it privileges particular kinds of evidence—‘scientific’ evidence, with a strong emphasis on randomized controlled trials and systematic reviews. It is not at all self-evident that this model is appropriate for, or indeed relevant to, the making of policy. In the case of policy, evidence tends to be something of a Delphic oracle—difficult to decipher and apt to be misinterpreted. Much has been written about the pitfalls of, and the delusive hopes held out by, EBP. So this paper asks whether it is possible to plot out a sensible course between a platitude and a nonsense. The platitude is that policy should be informed by evidence. Who could possibly disagree? The nonsense is that policy should be based on scientific evidence. This is to misunderstand the nature of both the policy process and the role of evidence in it.2–4 The way forward is to disaggregate the notions of both policy and evidence: different stages of the policy process may call for different types of evidence. Consider, first, the different types of evidence or knowledge that are relevant for the policy process. There does exist scientific evidence—i.e. research-based, usually peer-reviewed, evidence. Next there is what might be called organizational evidence—in the case of health policy, the experience of those actually working in the NHS. As a Cabinet Office paper on policy-making5 has put it: