Implications of applying methodological shortcuts to expedite systematic reviews: three case studies using systematic reviews from agri-food public health.

Abstract

BackgroundThe rapid review is an approach to synthesizing research evidence when a shorter timeframe is required. The implications of what is lost in terms of rigour, increased bias and accuracy when conducting a rapid review have not yet been elucidated.MethodsWe assessed the potential implications of methodological shortcuts on the outcomes of three completed systematic reviews addressing agri‐food public health topics. For each review, shortcuts were applied individually to assess the impact on the number of relevant studies included and whether omitted studies affected the direction, magnitude or precision of summary estimates from meta‐analyses.ResultsIn most instances, the shortcuts resulted in at least one relevant study being omitted from the review. The omission of studies affected 39 of 143 possible meta‐analyses, of which 14 were no longer possible because of insufficient studies (<2). When meta‐analysis was possible, the omission of studies generally resulted in less precise pooled estimates (i.e. wider confidence intervals) that did not differ in direction from the original estimate.ConclusionsThe three case studies demonstrated the risk of missing relevant literature and its impact on summary estimates when methodological shortcuts are applied in rapid reviews. © 2016 The Authors. Research Synthesis Methods Published by John Wiley & Sons Ltd.