Reporting of context and implementation in studies of global health interventions: a pilot study.

Margaret A Maglione,Tanja Perry,Jill Luoto,Breanne Johnsen,Paul G Shekelle

doi:10.1186/1748-5908-9-57

Margaret A Maglione, Tanja Perry + Show 3 more

Open Access

https://doi.org/10.1186/1748-5908-9-57

Copy DOI

Abstract

BackgroundThere is an increasing push for ‘evidence-based’ decision making in global health policy circles. However, at present there are no agreed upon standards or guidelines for how to evaluate evidence in global health. Recent evaluations of existing evidence frameworks that could serve such a purpose have identified details of program context and project implementation as missing components needed to inform policy. We performed a pilot study to assess the current state of reporting of context and implementation in studies of global health interventions.MethodsWe identified three existing criteria sets for implementation reporting and selected from them 10 criteria potentially relevant to the needs of policy makers in global health contexts. We applied these 10 criteria to 15 articles included in the evidence base for three global health interventions chosen to represent a diverse set of advocated global health programs or interventions: household water chlorination, prevention of mother-to-child transmission of HIV, and lay community health workers to reduce child mortality. We used a good-fair-poor/none scale for the ratings.ResultsThe proportion of criteria for which reporting was poor/none ranged from 11% to 54% with an average of 30%. Eight articles had ‘good’ or ‘fair’ documentation for greater than 75% of criteria, while five articles had ‘poor or none’ documentation for 50% of criteria or more. Examples of good reporting were identified.ConclusionsReporting of context and implementation information in studies of global health interventions is mostly fair or poor, and highly variable. The idiosyncratic variability in reporting indicates that global health investigators need more guidance about what aspects of context and implementation to measure and how to report them. This lack of context and implementation information is a major gap in the evidence needed by global health policy makers to reach decisions.

Highlights

There is an increasing push for ‘evidence-based’ decision making in global health policy circles
We evaluated how context and implementation were reported in recent studies of global health interventions in order to identify areas reported sufficiently well, and areas where action is needed to improve the design, conduct, and reporting of global health interventions to better enable decision-makers to make sound, evidence-based decisions
We used three existing sets of implementation criteria (IC) identified by experts in implementation science: the Consolidated Framework for Implementation Research (CFIR [5]); the proposed criteria for reporting the development and evaluation of complex interventions in healthcare (CReDECI [6]); and those criteria required by the editors of the journal Implementation Science, themselves based on the Workgroup for Intervention Development and Evaluation Research (WIDER) criteria [7]

Summary

Introduction

There is an increasing push for ‘evidence-based’ decision making in global health policy circles. At present there are no agreed upon standards or guidelines for how to evaluate evidence in global health. Recent evaluations of existing evidence frameworks that could serve such a purpose have identified details of program context and project implementation as missing components needed to inform policy. Policy makers in global health are increasingly adopting ‘evidence-based’ decision making practices [1,2]. By using the available evidence to inform their decision making, it is believed that resultant policy choices can improve in terms of their appropriateness in being applied to a particular context, or likelihood of achieving their envisioned aims. At present there are no commonly accepted guidelines within global public health for how to evaluate evidence.

Methods

Results

Conclusion