How was research engaged with and used in the development of 131 policy documents? Findings and measurement implications from a mixed methods study

Anna Williamson,Sally Redman,Steve R Makkar

doi:10.1186/s13012-019-0886-2

Anna Williamson, Sally Redman + Show 1 more

Open Access

https://doi.org/10.1186/s13012-019-0886-2

Copy DOI

Abstract

BackgroundMuch has been written about the use of evidence in policy; however, there is still little known about whether and how research is engaged with and used in policy development or the impact of reported barriers and facilitators. This paper aims to (1) describe the characteristics of 131 policy documents, (2) describe the ways in which research was engaged with (e.g. was searched for, appraised or generated) and used (e.g. to clarify understanding, persuade others or inform a policy) in the development of these policy documents, and (3) identify the most commonly reported barriers and facilitators and describe their association with research engagement and use.MethodsSix health policy and program development agencies based in Sydney, Australia, contributed four recently finalised policy documents for consideration over six measurement periods. Structured, qualitative interviews were conducted with the policymakers most heavily involved in developing each of the 131 policy documents. Interviews covered whether and how research was engaged with and used in the development of the policy product and any barriers or facilitators related to this. Interviews were scored using the empirically validated SAGE tool and thematically analysed. Descriptive statistics were calculated for all key variables and comparisons made between agencies. Multiple regression analyses were used to estimate the impact of specific barriers and facilitators on research engagement and use.ResultsOur data shows large variations between policy agencies in the types of policy documents produced and the characteristics of these documents. Nevertheless, research engagement and use was generally moderate across agencies. A number of barriers and facilitators to research use were identified. No barriers were significantly associated with any aspects of research engagement or use. Access to consultants and relationships with researchers were both associated with increased research engagement but not use. Thus, access to consultants and relationships with researchers may increase the extent and quality of the evidence considered in policy development.ConclusionsOur findings suggest that those wishing to develop interventions and programs designed to improve the use of evidence in policy agencies might usefully target increasing access to consultants and relationships with researchers in order to increase the extent and quality of the research considered, but that a greater consideration of context might be required to develop strategies to increase evidence use.

Highlights

Much has been written about the use of evidence in policy; there is still little known about whether and how research is engaged with and used in policy development or the impact of reported barriers and facilitators
Our large-scale examination of how research was engaged with and used in the development of specific policy documents highlights the diversity of research use cultures and contexts in policy agencies, and how these relate to agency remit
Agencies were eligible to participate in Supporting Policy In health with Research (SPIRIT) provided that (i) a significant proportion of their work was in health policy or program development, (ii) at least 20 of their staff members were involved in policy or program design, development or evaluation, and (iii) they were located in Sydney

Summary

Introduction

Much has been written about the use of evidence in policy; there is still little known about whether and how research is engaged with and used in policy development or the impact of reported barriers and facilitators. Some studies have attempted to use objective measures of research use [25], for example, Zardo and Collie and Bunn et al identify research cited within specific policy documents [7, 15]. This method does not account for the contribution of uncited research which may have played a part, conceptual or otherwise, in the development of the policy. A third approach has been to ask participants to rate the extent to which they drew on a range of evidence sources in developing specific policies; information on the ways in which each evidence type was used or the type of influence it had was not elicited [26]

Objectives

Methods

Results

Discussion

Conclusion