Don’t panic: Bringing complexity thinking to UK Government evaluation guidance

Martha Bicket,Helen Wilkinson,Dione Hills,Alexandra Penn

doi:10.1177/1356389020980479

Abstract

Central government guidance seeks to ensure and enhance the quality of practice and decision-making across – and sometimes beyond – government. The Magenta Book, published by HM Treasury, is the key UK Government resource on policy evaluation, setting out central government guidance on how to evaluate policies, projects and programmes. The UK Centre for the Evaluation of Complexity Across the Nexus was invited to contribute its expertise to the UK Government’s 2020 update of the Magenta Book by developing an accompanying guide on policy evaluation and ‘complexity’. A small multidisciplinary team worked together to produce a set of guidance, going through multiple stages of work and drawing on a variety of sources including academic and practitioner literature and experts and stakeholders in the fields of evaluation, policy and complexity. It also drew on Centre for the Evaluation of Complexity Across the Nexus’ own work developing and testing evaluation methods for dealing with complexity in evaluation. The resulting Magenta Book 2020 Supplementary Guide: Handling Complexity in Policy Evaluation explores the implications of complexity for policy and evaluation and how evaluation can help to navigate complexity. This article, designed primarily for practitioners who might be interested in this guidance and how it was developed, describes the processes involved, particularly related to the interdisciplinary dialogue and consultation with other key stakeholders that this involved. It also briefly outlines the content and key messages in the guidance, with reflections on the experiences of the authors in developing the guide – including the challenges and insights that arose during the process, particularly around the challenges of communicating complexity to a broad audience of readers.

Highlights

A complex and multifaceted policy intervention put in place within an complex setting can pose considerable challenges for those seeking to commission its evaluation and for evaluators themselves
The resulting Magenta Book 2020 Supplementary Guide: Handling Complexity in Policy Evaluation explores the implications of complexity for policy and evaluation and how evaluation can help to navigate complexity
By 2019, a great deal of experience had been gained within CECAN, on the most effective way to engage policy makers and analysts in issues related to complexity, and about the evaluation approaches and methods that were of particular interest

Summary

Introduction

A complex and multifaceted policy intervention put in place within an complex setting can pose considerable challenges for those seeking to commission its evaluation and for evaluators themselves. This article gives an account of CECAN’s experience of developing this guidance, the Magenta Book 2020 Supplementary Guide: Handling Complexity in Policy Evaluation (‘the Supplementary Guide’) (Bicket et al, 2020), reflecting on the challenges that arose, the joint learning that took place and the resulting guidance itself.

Results

Conclusion