Rapid reviews methods series: practical guidance for conducting timely evidence syntheses.

INTRODUCTION:Rapid reviews are of increasing importance within Health Technology Assessment (HTA) due to the need for timely evidence to underpin the assessment of new technologies as well as financial constraints. There are many rapid review methods available (1) although there is little guidance as to the most suitable methods (2). A recent paper outlines issues to consider when selecting rapid review methods (3). The aim of this presentation is to present key aspects to consider when selecting rapid review methods.METHODS:We searched the evidence base for guidance on the selection of rapid review methods. We also examined three recently completed systematic reviews to identify rapid review methods used, the reasons for selection and the strengths and weaknesses of each method. Finally we identified key aspects to consider when selecting rapid review methods.RESULTS:The evidence on guidance identified for the selection of rapid review methods was very limited. The analysis of the three reviews found that each review had distinctly different challenges, such as large numbers of relevant trials and heterogeneity in terms of populations, interventions, comparators and outcomes. All reviews included at least ten randomized controlled trials and numerous outcome measures. Three different approaches to the rapid review of the evidence were used in the three reviews. Key themes to consider when selecting rapid review methods were identified. These include: the size and nature of the evidence base, the characteristics of included studies and the expectations of those commissioning the review.CONCLUSIONS:Rapid review methods need to be chosen to fit the needs of the review, each of which may have different challenges. Collaboration between those producing rapid reviews and commissioners is crucial when choosing methods to ensure that the needs of commissioners are met and limitations associated with the chosen methods are understood.

Using preprints in evidence synthesis: Commentary on experience during the COVID-19 pandemic

ObjectiveTo simulate possible changes in systematic review results if rapid review methods were used. Study Design and SettingWe recalculated meta-analyses for binary primary outcomes in Cochrane systematic reviews, simulating rapid review methods. We simulated searching only PubMed, excluding older articles (5, 7, 10, 15, and 20 years before the search date), excluding smaller trials (<50, <100, and <200 participants), and using the largest trial only. We examined percentage changes in pooled odds ratios (ORs) (classed as no important change [<5%], small [<20%], moderate [<30%], or large [≥30%]), statistical significance, and biases observed using rapid methods. ResultsTwo thousand five hundred and twelve systematic reviews (16,088 studies) were included. Rapid methods resulted in the loss of all data in 3.7–44.7% of meta-analyses. Searching only PubMed had the smallest risk of changed ORs (19% [477/2,512] were small changes or greater; 10% [260/2,512] were moderate or greater). Changes in ORs varied substantially with each rapid review method; 8.4–21.3% were small, 1.9–8.8% were moderate, and 4.7–34.1% were large. Changes in statistical significance occurred in 6.5–38.6% of meta-analyses. Changes from significant to nonsignificant were most common (2.1–13.7% meta-analyses). We found no evidence of bias with any rapid review method. ConclusionSearching PubMed only might be considered where a ∼10% risk of the primary outcome OR changing by >20% could be tolerated. This could be the case in scoping reviews, resource limitation, or where syntheses are needed urgently. Other situations, such as clinical guidelines and regulatory decisions, favor more comprehensive systematic review methods.

BackgroundRapid reviews are a form of knowledge synthesis in which components of the systematic review process are simplified or omitted to produce information in a timely manner. Although numerous centers are conducting rapid reviews internationally, few studies have examined the methodological characteristics of rapid reviews. We aimed to examine articles, books, and reports that evaluated, compared, used or described rapid reviews or methods through a scoping review.MethodsMEDLINE, EMBASE, the Cochrane Library, internet websites of rapid review producers, and reference lists were searched to identify articles for inclusion. Two reviewers independently screened literature search results and abstracted data from included studies. Descriptive analysis was conducted.ResultsWe included 100 articles plus one companion report that were published between 1997 and 2013. The studies were categorized as 84 application papers, seven development papers, six impact papers, and four comparison papers (one was included in two categories). The rapid reviews were conducted between 1 and 12 months, predominantly in Europe (58 %) and North America (20 %). The included studies failed to report 6 % to 73 % of the specific systematic review steps examined. Fifty unique rapid review methods were identified; 16 methods occurred more than once. Streamlined methods that were used in the 82 rapid reviews included limiting the literature search to published literature (24 %) or one database (2 %), limiting inclusion criteria by date (68 %) or language (49 %), having one person screen and another verify or screen excluded studies (6 %), having one person abstract data and another verify (23 %), not conducting risk of bias/quality appraisal (7 %) or having only one reviewer conduct the quality appraisal (7 %), and presenting results as a narrative summary (78 %). Four case studies were identified that compared the results of rapid reviews to systematic reviews. Three studies found that the conclusions between rapid reviews and systematic reviews were congruent.ConclusionsNumerous rapid review approaches were identified and few were used consistently in the literature. Poor quality of reporting was observed. A prospective study comparing the results from rapid reviews to those obtained through systematic reviews is warranted.Electronic supplementary materialThe online version of this article (doi:10.1186/s12916-015-0465-6) contains supplementary material, which is available to authorized users.

Rapid reviews using abbreviated systematic review methods are of increasing importance for evidence-informed decision-making in education, although there is little guidance about the most suitable approach. Three recently completed rapid review reports are compared to inform discussions on the utility of this type of review in education and to highlight appropriate methods for producing evidence syntheses in a limited time frame. Rapid review methods need to be chosen to fit the needs of the review, which involves: thinking broadly about different kinds of team experience and expertise; estimating the size and nature of the literature to be reviewed; considering the review purpose and nature of the topic; choosing an appropriate synthesis method for the review purpose, evidence base and reviewers’ expertise; fully describing the review approach, and discussing the potential limitations of chosen methods; and understanding the anticipated audiences and tailoring outputs accordingly. Rapid reviews to address urgent and high-priority questions provide the benefits of timeliness and reduced resource requirements. However, it is crucial to understand caveats and limitations to the rapid conduct of evidence syntheses for decision-making purposes. This article offers guidance to support researchers, postgraduate students and commissioners who wish to conduct rapid reviews in a transparent and systematic way, addressing complex questions of relevance to evidence-informed decision-making in education.

BackgroundHealth policy-makers must often make decisions in compressed time frames and with limited resources. Hence, rapid reviews have become a pragmatic alternative to comprehensive systematic reviews. However, it is important that rapid review methods remain rigorous to support good policy development and decisions. There is currently little evidence about which streamlined steps in a rapid review are less likely to introduce unacceptable levels of uncertainty while still producing a product that remains useful to policy-makers.MethodsThis paper summarizes current research describing commonly used methods and practices that are used to conduct rapid reviews and presents key considerations and options to guide methodological choices for a rapid review.ResultsThe most important step for a rapid review is for an experienced research team to have early and ongoing engagement with the people who have requested the review. A clear research protocol, derived from a needs assessment conducted with the requester, serves to focus the review, defines the scope of the rapid review, and guides all subsequent steps. Common recommendations for rapid review methods include tailoring the literature search in terms of databases, dates, and languages. Researchers can consider using a staged search to locate high-quality systematic reviews and then subsequently published primary studies. The approaches used for study screening and selection, data extraction, and risk-of-bias assessment should be tailored to the topic, researcher experience, and available resources. Many rapid reviews use a single reviewer for study selection, risk-of-bias assessment, or data abstraction, sometimes with partial or full verification by a second reviewer. Rapid reviews usually use a descriptive synthesis method rather than quantitative meta-analysis. Use of brief report templates and standardized production methods helps to speed final report publication.ConclusionsResearchers conducting rapid reviews need to make transparent methodological choices, informed by stakeholder input, to ensure that rapid reviews meet their intended purpose. Transparency is critical because it is unclear how or how much streamlined methods can bias the conclusions of reviews. There are not yet internationally accepted standards for conducting or reporting rapid reviews. Thus, this article proposes interim guidance for researchers who are increasingly employing these methods.

BackgroundRapid reviews are of increasing importance within health technology assessment due to time and resource constraints. There are many rapid review methods available although there is little guidance as to the most suitable methods. We present three case studies employing differing methods to suit the evidence base for each review and outline some issues to consider when selecting an appropriate method.MethodsThree recently completed systematic review short reports produced for the UK National Institute for Health Research were examined. Different approaches to rapid review methods were used in the three reports which were undertaken to inform the commissioning of services within the NHS and to inform future trial design. We describe the methods used, the reasoning behind the choice of methods and explore the strengths and weaknesses of each method.ResultsRapid review methods were chosen to meet the needs of the review and each review had distinctly different challenges such as heterogeneity in terms of populations, interventions, comparators and outcome measures (PICO) and/or large numbers of relevant trials. All reviews included at least 10 randomised controlled trials (RCTs), each with numerous included outcomes. For the first case study (sexual health interventions), very diverse studies in terms of PICO were included. P-values and summary information only were presented due to substantial heterogeneity between studies and outcomes measured. For the second case study (premature ejaculation treatments), there were over 100 RCTs but also several existing systematic reviews. Data for meta-analyses were extracted directly from existing systematic reviews with new RCT data added where available. For the final case study (cannabis cessation therapies), studies included a wide range of interventions and considerable variation in study populations and outcomes. A brief summary of the key findings for each study was presented and narrative synthesis used to summarise results for each pair of interventions compared.ConclusionsRapid review methods need to be chosen to meet both the nature of the evidence base of a review and the challenges presented by the included studies. Appropriate methods should be chosen after an assessment of the evidence base.

The Use of Rapid Review Methods for the U.S. Preventive Services Task Force

BackgroundPolicy makers and others often require synthesis of knowledge in an area within six months or less. Traditional systematic reviews typically take at least 12 months to conduct. Rapid reviews streamline traditional systematic review methods in order to synthesize evidence within a shortened timeframe. There is great variation in the process of conducting rapid reviews. This review sought to examine methods used for rapid reviews, as well as implications of methodological streamlining in terms of rigour, bias, and results.MethodsA comprehensive search strategy--including five electronic databases, grey literature, hand searching of relevant journals, and contacting key informants--was undertaken. All titles and abstracts (n = 1,989) were reviewed independently by two reviewers. Relevance criteria included articles published between 1995 and 2009 about conducting rapid reviews or addressing comparisons of rapid reviews versus traditional reviews. Full articles were retrieved for any titles deemed relevant by either reviewer (n = 70). Data were extracted from all relevant methodological articles (n = 45) and from exemplars of rapid review methods (n = 25).ResultsRapid reviews varied from three weeks to six months; various methods for speeding up the process were employed. Some limited searching by years, databases, language, and sources beyond electronic searches. Several employed one reviewer for title and abstract reviewing, full text review, methodological quality assessment, and/or data extraction phases. Within rapid review studies, accelerating the data extraction process may lead to missing some relevant information. Biases may be introduced due to shortened timeframes for literature searching, article retrieval, and appraisal.ConclusionsThis review examined the continuum between diverse rapid review methods and traditional systematic reviews. It also examines potential implications of streamlined review methods. More of these rapid reviews need to be published in the peer-reviewed literature with an emphasis on articulating methods employed. While one consistent methodological approach may not be optimal or appropriate, it is important that researchers undertaking reviews within the rapid to systematic continuum provide detailed descriptions of methods used and discuss the implications of their chosen methods in terms of potential bias introduced. Further research comparing full systematic reviews with rapid reviews will enhance understanding of the limitations of these methods.

BackgroundThe objective of this work was to inform the design of a rapid response program to support evidence-informed decision-making in health policy and practice for the Americas region. Specifically, we focus on the following: (1) What are the best methodological approaches for rapid reviews of the research evidence? (2) What other strategies are needed to facilitate evidence-informed decision-making in health policy and practice? and (3) How best to operationalize a rapid response program?MethodsThe evidence used to inform the design of a rapid response program included (i) two rapid reviews of methodological approaches for rapid reviews of the research evidence and strategies to facilitate evidence-informed decision-making, (ii) supplementary literature in relation to the “shortcuts” that could be considered to reduce the time needed to complete rapid reviews, (iii) four case studies, and (iv) supplementary literature to identify additional operational issues for the design of the program.ResultsThere is no agreed definition of rapid reviews in the literature and no agreed methodology for conducting them. Better reporting of rapid review methods is needed. The literature found in relation to shortcuts will be helpful in choosing shortcuts that maximize timeliness while minimizing the impact on quality. Evidence for other strategies that can be used concurrently to facilitate the uptake of research evidence, including evidence drawn from rapid reviews, is presented. Operational issues that need to be considered in designing a rapid response program include the implications of a “user-pays” model, the importance of recruiting staff with the right mix of skills and qualifications, and ensuring that the impact of the model on research use in decision-making is formally evaluated.ConclusionsWhen designing a new rapid response program, greater attention needs to be given to specifying the rapid review methods and reporting these in sufficient detail to allow a quality assessment. It will also be important to engage in other strategies to facilitate the uptake of the rapid reviews and to evaluate the chosen model in order to make refinements and add to the evidence base for evidence-informed decision-making.Electronic supplementary materialThe online version of this article (doi:10.1186/s13012-016-0472-9) contains supplementary material, which is available to authorized users.

This paper, part of the Cochrane Rapid Review Methods Group series, offers guidance on determining when to conduct a rapid review (RR) instead of a full systematic review (SR). While...

Rapid reviews (RRs) are a helpful evidence synthesis tool to support urgent and emergent decision-making in healthcare. RRs involve abbreviating systematic review methods and are conducted in a condensed timeline...

Delphi consensus reached to produce a decision tool for SelecTing Approaches for Rapid Reviews (STARR)

IntroductionThe Health Technology Assessment (HTA) Department of the “State Expert Center of the Ministry of Health of Ukraine” was established in 2019 for conducting assessments and development of recommendations for informing decisions of the Ukrainian Ministry of Health regarding financing of the health technologies. Since 2015, procurement of medicines by the state budget in Ukraine was carried out through international specialized organizations. The central procurement program covers forty-one programs for different disease areas. The list of medicines and medical devices purchased were based of procurement agreements with specialized organizations approved by Cabinet of Ministers Decree of Ukraine № 255 of 13 March 2019.MethodsDescriptive analysis of methodological processes around rapid review methods in HTA used for the procedure of central procurements of medicines. In the process of conducting rapid assessment we use a modern tool based on the HTA Core Model® for Rapid Relative Effectiveness Assessments version (V3.0). HTA Guidance in Ukraine was developed in 2017.ResultsThe list of government programs, for which rapid review methods in HTAhave been conducted, included following diseases: adult and pediatric oncology, juvenile rheumatoid arthritis, hemophilia, and orphan diseases. Reports include an overview of comparative efficacy, effectiveness and safety, as well as analysis of reports from HTA agencies and thresholds, budget impact for informing decisions for development of the central procurement programs in 2019. The HTA Department conducted more than forty reviews in less than 6 months.ConclusionsUse of rapid review methods in HTA in Ukraine for informing decisions for the central procurement programs of medicines is the perspective for rational resource allocation and spending. This increases and improves patients’ access to effective, safe and cost-effective medicines.

BackgroundPolicymakers and healthcare stakeholders are increasingly seeking evidence to inform the policymaking process, and often use existing or commissioned systematic reviews to inform decisions. However, the methodologies that make systematic reviews authoritative take time, typically 1 to 2 years to complete. Outside the traditional SR timeline, “rapid reviews” have emerged as an efficient tool to get evidence to decision-makers more quickly. However, the use of rapid reviews does present challenges. To date, there has been limited published empirical information about this approach to compiling evidence. Thus, it remains a poorly understood and ill-defined set of diverse methodologies with various labels. In recent years, the need to further explore rapid review methods, characteristics, and their use has been recognized by a growing network of healthcare researchers, policymakers, and organizations, several with ties to Cochrane, which is recognized as representing an international gold standard for high-quality, systematic reviews.PurposeIn this commentary, we introduce the newly established Cochrane Rapid Reviews Methods Group developed to play a leading role in guiding the production of rapid reviews given they are increasingly employed as a research synthesis tool to support timely evidence-informed decision-making. We discuss how the group was formed and outline the group’s structure and remit. We also discuss the need to establish a more robust evidence base for rapid reviews in the published literature, and the importance of promoting registration of rapid review protocols in an effort to promote efficiency and transparency in research.ConclusionAs with standard systematic reviews, the core principles of evidence-based synthesis should apply to rapid reviews in order to minimize bias to the extent possible. The Cochrane Rapid Reviews Methods Group will serve to establish a network of rapid review stakeholders and provide a forum for discussion and training. By facilitating exchange, the group will strive to conduct research to advance the methods of rapid reviews.