Minimum unit pricing and the messiness of evidence-based policy.

Abstract

On the face of it, minimum unit pricing is a textbook case of evidence-based policymaking. Yet as John Holmes shows, policy evidence can be messy and ambiguous. To make sense of it, we should not be afraid to draw on wider evidence of pricing policies in other contexts. As somebody with a keen interest in evidence-based policy, and somebody who has been involved in United Kingdom alcohol policy analysis and debate for the past 7 years, following minimum unit pricing (MUP) in Scotland has been a fascinating experience. The Scottish Government's approach to evaluation has been, as John Holmes remarks, ‘notable for its ambition and comprehensiveness’ [1]. It is in many ways a model for other policymakers to follow. The Government embarked on an experimental new policy approach and committed to reviewing it after 6 years. In advance, it engaged a range of independent researchers, led by the arms-length body Public Health Scotland (PHS), to assess whether the policy is working. If the policy is successful, the Government will retain MUP. If it fails, the regulation will be dropped. It all seems a most admirable approach. The trouble, as John Holmes’ analysis shows, is that demonstrating a policy is ‘working’ is trickier than it seems. There may be some interventions that have such a huge and obvious impact that their success is incontrovertible. However, they are the exception. Most policies—and MUP seems to be among them—have more modest, ambiguous effects that may take some time to emerge. Modest and ambiguous does not mean trivial. A key piece of the evidence jigsaw puzzle that has emerged since Holmes’ article was completed is PHS’ estimate that MUP reduced deaths wholly attributable to alcohol by 13% [2]. That implies that hundreds of lives have been saved—PHS put the total at 156 a year. Yet sceptics of the policy will attribute such figures—alongside the 3% to 3.5% reduction in alcohol consumption in Scotland relative to England identified by Holmes—to random noise. They are likely to put more weight on the ambiguous evidence regarding the responsiveness of the heaviest and most harmful drinkers. In my view, the evidence we have so far is still just about consistent with the possibility that MUP had no real effect on harmful drinking, although you need to squint a bit, or apply some bad faith, to think it is the most likely explanation of the facts. This is the valley in which MUP finds itself. On the balance of probabilities, a success, but not yet an undeniable one. In assessing all this for myself, I have been struck by how far I have ended up relying on my ‘prior’ (as Bayesians would put it) that pricing policies are effective. In other words, I am more inclined to say that the balance of probabilities counts in favour of MUP being effective because the evidence we have on it is consistent with the theory and evidence that making alcohol more expensive reduces harm. The model of evidence-based policy I sketched earlier implied that we should only need to look at what is happening on the ground in Scotland to know whether MUP is achieving its aims. Yet, I personally have found it much more helpful to put it in the context of the range of studies, combined in multiple meta-analyses, confirmed by independent replication studies, that reinforce the conclusion that people drink less if prices go up, and if they drink less that has positive health and social consequences [3-6]. If we start with that context in mind, I do not think there is much in the accumulated evidence on MUP in Scotland that should shake us from our expectation that the policy should produce positive outcomes. Indeed, the evidence that we have is clearly consistent with those positive outcomes being achieved. I can see the argument that this amounts to a sort of prejudice, when evidence-based policy should mean judging individual policies on their merits. I am not so sure though. When the information we get from individual evaluations is as contestable as it is, I think we do ourselves a disservice if we fail to draw on the other things we know to reach balanced conclusions. To reach a proper understanding of MUP's effectiveness in Scotland, we may do well to widen our lenses. None. No data involved in this paper.