A review of Grey and academic literature of evaluation guidance relevant to public health interventions

Sarah Denford,Margaret Callaghan,Peter Aighton,Frank De Vocht,Charles Abraham,Steven Arris

doi:10.1186/s12913-017-2588-2

Abstract

BackgroundPublic Health evaluation is essential to understanding what does and does not work, and robust demonstration of effectiveness may be crucial to securing future funding. Despite this, programs are often implemented with poor, incomplete or no evaluation. Public health practitioners are frequently required to provide evidence for the effectiveness of their services; thus, there is a growing need for evaluation guidance on how to evaluate public health programs. The aim of this study is to identify accessible high-quality, evaluation guidance, available to researchers and practitioners and to catalogue, summarise and categorise the content of a subset of accessible, quality guides to evaluation.MethodsWe systematically reviewed grey and academic literature for documents providing support for evaluation of complex health interventions. Searches were conducted January to March 2015, and included academic databases, internet search engines, and consultations with academic and practicing public health experts. Data were extracted by two authors and sent to the authors of the guidance documents for comments.ResultsOur initial search identified 402 unique documents that were screened to identify those that were (1) developed by or for a national or international organization (2) freely available to all (3) published during or after 2000 (4) specific to public health. This yielded 98 documents from 43 organisations. Of these, 48 were reviewed in detail. This generated a detailed catalogue of quality evaluation guidance. The content included in documents covers 37 facets of evaluation.ConclusionsA wide range of guidance on evaluation of public health initiatives is available. Time and knowledge constraints may mean that busy practitioners find it challenging to access the most, up-to-date, relevant and useful guidance. This review presents links to and reviews of 48 quality guides to evaluation as well as categorising their content. This facilitates quick and each access to multiple selected sources of specific guidance.

Highlights

Public Health evaluation is essential to understanding what does and does not work, and robust demonstration of effectiveness may be crucial to securing future funding
The present study We aimed to review this literature in order to assess the extent of available guides to evaluation relevant to public health interventions and to identify the content of such guidance
EMBASE, MEDLINE, MEDLINE-in-process, Health management information consortium (HMIC), Social Policy and Practice (SSP), Web of Science, and PsycINFO were searched using the search strategy presented in Additional file 1

Summary

Introduction

Public Health evaluation is essential to understanding what does and does not work, and robust demonstration of effectiveness may be crucial to securing future funding. Eighteen guides provided an overview of evaluation This included explanation of the nature of evaluation and how it differs from other types of research; why evaluation is needed and what it can tell us; and the benefits of conducting evaluation. Eight guides informed readers how to identify and assess the quality and relevance of existing research and evaluations This frequently included links to quality assessment scales such as CONSORT and the Equator network. Four of these eight guides discuss evidence based practice (EBP); what EBP is, why it is important, and how to conduct evaluations within an evidence based framework. The importance of choosing “best available” methods, even if they are not optimum, are highlighted

Objectives

Methods

Results

Conclusion