Reporting of narrative synthesis in systematic reviews of public health interventions: a methodological assessment

Abstract

BackgroundThe value of systematic reviews for decision making is their transparency, reproducibility, and robust approach to assessing the evidence base. For reviews of public health interventions, quantitative data are often narratively synthesised. Narrative synthesis has been criticised for its lack of transparency, raising concerns about potential bias in many systematic reviews used in public health, and threatening their value for decision making. We systematically assessed the reporting and conduct of narrative synthesis in reviews assessing public health interventions. MethodsWe screened a 20% (474/2372) random sample of systematic reviews published between Jan 1, 2010, and Oct 31, 2015, from the McMaster Health Evidence database (a comprehensive database of systematic reviews of public health interventions), to identify reviews that had synthesised quantitative data narratively. From these reviews we investigated a 30% (n=75) random sample in greater detail. Informed by key guidance on narrative synthesis methods, we extracted data about the description of narrative synthesis methods, presentation of data, and management of heterogeneity by review authors. These data were used to assess the adequacy of reporting of narrative synthesis. FindingsNarrative synthesis was used in 251 (53%) of the 474 reviews sampled. Reporting of narrative synthesis methods and data were limited in most reviews: 58 (77%) of the 75 reviews assessed in more detail provided no rationale for or description of the narrative synthesis process, with 36 (48%) simply stating that meta-analysis was not possible. Presentation of data was often incomplete (40%, n=30), with investigation and management of heterogeneity unclear. Of the 51 reviews in which data were grouped, 22 (43%) tabulated data differently from how the data were grouped in the text. InterpretationNarrative synthesis of quantitative data in public health reviews is often inadequate. Reporting of methods is limited, and available guidance is rarely referred to. Links between the data and the narrative summary are often unclear. This lack of transparency prevents assessment of the reliability of review findings, and threatens the credibility of systematic reviews that use narrative synthesis. For many public health questions, narrative synthesis will continue to be a necessary synthesis method. Updated guidance is therefore required, and review authors need to be transparent in their reporting of narrative synthesis, thereby improving reliability. FundingMC and HT were funded by the UK Medical Research Council Chief Scientist Office Informing Healthy Public Policy programme (MC_UU_12017-15). SVK is supported by a NHS Research Scotland Scottish senior clinical fellowship (SCAF/15/02).