Balancing the strengths of systematic and narrative reviews

Abstract

The mandate of Human Reproduction Update involves several roles: (i) to provide a synthesis of evidence that can aid scientists and clinicians in their daily work; (ii) to help reproductive specialists understand concepts from related disciplines; and (iii) to summarize current knowledge generated by basic science as the foundation of future scientific and clinical advancement. Given that review and synthesis are central to good scientific and clinical practice, and that a grasp of the current state of knowledge is a prerequisite to designing new studies, it is pertinent to ask which reviews are most likely to fulfil the needs of readers. A related question concerns whether systematic reviews meet the needs of all review topics and all readers. Summarizing evidence or knowledge is a difficult problem in reproductive medicine, as in other branches of science and medical care (Eddy et al., 1992). For each question there may be multiple studies that use different designs and inclusion criteria. For clinical questions, the interventions, outcomes and measures of effect may vary: the effect measures in treatment studies include odds ratios, relative risks and absolute differences. For scientific questions, the experimental species, models and designs may differ. Moreover, it is always uncertain whether all of the relevant evidence has been evaluated. Even when the search has been exhaustive, there are no simple guides on how to interpret conflicting results and whether to accept apparently outlying studies. The choices that the reviewer makes to address the variable conditions and uncertainties may be conservative, strict and exclusive, or liberal, open and inclusive. The decisions made by the reviewer may not be consistent throughout and these choices may or may not satisfy the reader who seeks out the review to address a clinical or research question. Faced with uncertainty and doubt, readers nonetheless must form an impression of the evidence and synthesize the state of knowledge in order to address the clinical or research question that stimulated their interest in the review. We argue that the reader is better served when the choices made in the review, regardless of whether they are strict or open, should be explicit, transparent, clearly stated and reproducible by interested readers. This list of objectives for reviews is more easily satisfied by systematic reviews, which use explicit methods to methodically search, critically appraise and synthesize the available literature on a specific issue. The question or issue need not be clinical: indeed, the concept evolved primarily in psychology studies (Light and Pillemer, 1984). The systematic review attempts to reduce reviewer bias through the use of objective, reproducible criteria to select relevant individual publications and assess their validity. A systematic review may include a meta-analysis or statistical summary of the individual study results: the aggregate of effects from several studies yields an average treatment effect that is more precise than the individual study results (Schlesselman and Collins, 2003). Thus, the systematic review involves explicit, transparent methods which are clearly stated, and reproducible by others. Whether a systematic review of randomized controlled trials adheres to the guidelines can easily be evaluated by means of a widely used checklist (the QUORUM statement) (Moher et al., 1999). The strengths of the systematic review include the narrow focus of the question, the comprehensive search for evidence, the criterion-based selection of relevant evidence, the rigorous appraisal of validity, the objective or quantitative summary, and the evidence-based inferences (Cook et al., 1997). For some review topics, however, the strengths of the systematic review may turn into weaknesses. The primary problem is that the narrow focus and prescribed methods of the systematic review do not allow for comprehensive coverage. For example, the historical review is an irreplaceable means of tracing the development of a scientific principle or clinical concept, but the narrative thread could be lost in the strict rules of systematic review. As other examples, it would be burdensome to apply systematic methods to a survey on aneuploidy and fertility in the aging female or to an assessment of mouse knockout models and polycystic ovarian phenotype. Such topics would require the wider scope of a traditional narrative review, in Human Reproduction Update, Vol.11, No.2 pp. 103–104, 2005 doi:10.1093/humupd/dmh058 Advance Access publication December 23, 2004