Improving systematic reviews: guidance on guidance and other options and challenges

Abstract

Multiple guideline tools are available for systematic reviews. These tools intend to standardize protocol development, require comprehensive reporting, and improve methodological rigor, including risk of bias assessment of primary studies and appraisal of the conduct of the reviews themselves. We recently published a guidance paper concerning these instruments and we hope that it will prove useful to producers, appraisers, and users of systematic reviews. There are still numerous open frontiers in improving systematic reviews. These include but are not limited to training of systematic reviewers; education of peer-reviewers, editors, and publishers; improving funder-based incentives, diminishing redundancy, increasing transparency, requiring protocol registration, confirming reporting and conduct standards, and establishing expectations of current meta-analysis methods. Each of these issues has caveats and challenges. Moreover, too many influential reviews continue to be non-systematic and expert opinion based. We need to understand why these reviews continue to be favored in the literature. Additional opportunities and need for research arise in the connection between primary evidence and systematic reviews. In some cases, the two may become indistinguishable. Living reviews become increasingly attractive in the currently evolving research circumstances but require additional safeguards. The connection between systematic reviews and guidelines or other implementation and decision-making tools is transitioning as well. Guidance efforts gain increasing attention, and may indeed help improve evidence synthesis but proper meta-research is needed to rigorously assess any improvements. We should maximize the contribution of systematic reviews, but also reduce the chances of producing checklist-heavy, ritual-burdened documents of questionable utility.