Priority setting for Cochrane Review Groups

Abstract

The number of clinical questions is almost infinite, whereas the resources to answer them are limited. Accordingly, Cochrane Review Groups (CRGs) must establish priorities. To do so, we need to understand the information needs of patients, their clinicians, and perhaps health administrators responsible for prioritizing health resources. In the past, this was undertaken implicitly, mostly based on pragmatic considerations (Can the proposed team achieve the review without gargantuan input from the CRG? Is there any evidence for the question? Is the question addressing currently used, or at least contemplated, clinical practice?), but we now know that clinicians and clinical researchers make errors in assuming what is important to know for patients. For example, priority setting of an organization setting research outcome measures in rheumatology (OMERACT, https://omeract.org/) had to be completely rejigged after patients with rheumatoid disease were explicitly asked for their research outcome priorities (tiredness being reported by patients as much more important than pain and function, their previously assumed priorities) [[1]Hewlett S.A. Patients and clinicians have different perspectives on outcomes in arthritis.J Rheumatol. 2003; 30: 877-879PubMed Google Scholar]. This is true for many other clinical areas [2van Veenendaal H. Grinspun D.R. Adriaanse H.P. Educational needs of stroke survivors and their family members, as perceived by themselves and by health professionals.Patient Educ Couns. 1996; 28: 265-276Crossref PubMed Scopus (80) Google Scholar, 3Hourts P.S. Rusenas I. Simmonds M.A. Hufford D.L. Information needs of families of cancer patients: a literature review and recommendations.J Cancer Educ. 1991; 6: 255-261Google Scholar]. The acute respiratory infections (ARIs) CRG editorial executive set out to objectively prioritize information needs of patients, clinicians, clinical researchers, and health administrators. The task is not straightforward for two reasons: (1) the range of ARI illnesses and treatments is very large and heterogeneous; (2) simply people find it difficult to declare what they want to know (“I don't even know what I don't know”). Lists generated as a starting point for prioritization to make this easier risk prejudice. We decided to limit our investigation to intervention questions, which make up most Cochrane reviews (although diagnostic and prognostic reviews are being produced now). We first undertook an audit of the ARI Group's Cochrane (systematic) reviews and compared the topics here with randomized controlled trials (RCTs) that are collected in the ARI CRG's trial register, the largest collection of RCTs available, containing not only what is in standard databases (such as Medline and Embase), but unpublished trials (such as conference proceedings). This identified gaps between the RCTs (a proxy for what research trialists thought important) and Cochrane reviews, which was 41% of the RCTs [[4]Alloo J. Vallath S. Del Mar C. Carter M. Thorning S. Clark J. Determining the gaps between Cochrane reviews and trials of effectiveness of interventions for acute respiratory infections: an audit.Syst Rev. 2017; 6: 82Crossref PubMed Scopus (5) Google Scholar]. There was overrepresentation of some topics (eg, antibiotic vs. placebo for pneumonia had 11 Cochrane Reviews and 205 RCTs), whereas in six Cochrane reviews, there were no RCTs (so-called “empty reviews”) [[4]Alloo J. Vallath S. Del Mar C. Carter M. Thorning S. Clark J. Determining the gaps between Cochrane reviews and trials of effectiveness of interventions for acute respiratory infections: an audit.Syst Rev. 2017; 6: 82Crossref PubMed Scopus (5) Google Scholar]. The next step was to present a list of the 649 systematic review topics (selected for lacking a Cochrane Review but with existing RCT evidence), generated by the audit, to stakeholders (who were contacted electronically through listservs inside and outside Cochrane, to obtain as much diversity as possible from consumers, clinicians, researchers—including Cochrane authors—and health administrators) [[5]Scott A.M. Clark J. Dooley L. Jones A. Jones M. Del Mar C. Cochrane acute respiratory infections group's stakeholder engagement project identified systematic review priority areas.J Clin Epidemiol. 2018; 102: 63-68Abstract Full Text Full Text PDF PubMed Scopus (4) Google Scholar]. Stakeholders were given a month to respond for each of the two rounds. The first identified the top 25 systematic review topics, with an option to add in extra topics not listed, and the second round asked the participants to each nominate their top 10, yielding a collective top 20. There was a broad representation with 154 respondents from 33 countries—clinicians and researchers as well as those classifying themselves as “other.” That list of 20 priority topics is now available for decision making [[5]Scott A.M. Clark J. Dooley L. Jones A. Jones M. Del Mar C. Cochrane acute respiratory infections group's stakeholder engagement project identified systematic review priority areas.J Clin Epidemiol. 2018; 102: 63-68Abstract Full Text Full Text PDF PubMed Scopus (4) Google Scholar]. Finally, we analyzed a data set of 314,346 clinical questions about ARIs asked by clinicians (and an unknown proportion of consumers) from an evidence-answering service (TRIP database www.tripdatabase.com/). The searches were difficult to analyze, requiring us to infer the question being asked when an illness and clinical management could be identified, which was possible in a minority of patients (45,497). We set out the inferred questions by illness along one axis and management along the other. When both illness and treatment were the same for different clinical questions, we classified them as identical. This method allowed us to address not only treatment questions (which made up 91%) but also diagnostic (5%) and prognosis questions (2%). The 20 most common questions (which addressed 67%) partly overlapped some common questions of the stakeholder survey. All but two already had Cochrane reviews addressing them [[6]Clark J. Carter M. Scott A.M. Brassey J. Del Mar C. The TRIP database showed most Acute Respiratory Infections questions were already addressed by Cochrane reviews.J Clin Epidemiol. 2019; 107 ([under editorial review]): 60-65Abstract Full Text Full Text PDF Scopus (1) Google Scholar]. These three studies have provided us with objectively derived priority lists, not perfect (eg, we could have collected the data in many other ways, probably yielding different priorities), but providing three incompletely overlapping lists to support our prioritization decisions (Table 1). Clearly, these cannot be used slavishly—they do not form an algorithm, and there are other important factors to consider, including the ability of the proposed team to successfully complete and update a review.Table 1Items ranked from the three priority setting exercises of the Cochrane Acute Respiratory Infections Group, for comparisonDiseasesManagementsTRIP analysisDelphi SurveyRCTs unreviewed by CochraneTRIP analysisDelphi SurveyRCTs unreviewed by CochranePneumoniaARI nonspecificPneumoniaAntiviralAntibioticsAntibiotic vs. antibioticInfluenzaInfluenzaBronchitis, acuteNSAIDNonspecific treatmentVaccinationCommon coldOtitis mediaPharyngitis/sore throatVitamins/SupplementsVaccinationImmunotherapyAcute Otitis MediaSore throatOtitis mediainfection controlCorticosteroidNSAIDPharyngitisSinusitisARI nonspecificSaline/Steam/lavageVitamins/SupplementsAntiviralSinusitisBronchiolitisSinusitisSore throatProbioticVaccinationMeningitisARI nonspecificPolioComplementary/AlternativePhysiotherapyVitamins/supplementsBronchiolitisInfluenzaBronchiolitisCorticosteroidProcalcitoninAntitussive/decongestantCroupPneumoniaCommon coldDiagnostic testAntitussiveComplementary/AlternativeMortality outcomesCommon coldPharyngitis/sore throatVaccinationSurgeryVaccination reminderPneumococcusCommon coldImmunotherapyHumidification/steamCoughHerpes simplexCPAPAntiviralRespiratory tract infectionOtitis mediaAntihistamineVaccinationDiagnostic questionsARI nonspecificAntiviralPertussisStaphylococcusParacetamolBronchitisBronchiolitisComplementary/AlternativeMeaslesInfluenzaImmunoglobulinWheezePneumoniaRadiographyPrognosis questionsRespiratory syncytial virusNasal irrigationRubellaStaphylococcusHumidificationRespiratory syncytial virusStreptococcusBarrier masksMumpsBronchodilatorInfectious mononucleosisEducationLaryngitisExercise/PositioningSARSNSAIDRhinorrheaOxygen therapyNasopharyngitisHelioxInfection controlHand washingVitamin AXylitolLeukotrieneFluid therapySurfactantStatinsPublic healthGlycerolAnticholinergicsImmunostimulantAbbreviations: ARI, acute respiratory infection; RCT, randomized controlled trial.The data from the Delphi survey were taken from round 2 (the final) table; from the Cochrane ARI Group were of RCTs with no reviews. Open table in a new tab Abbreviations: ARI, acute respiratory infection; RCT, randomized controlled trial. The data from the Delphi survey were taken from round 2 (the final) table; from the Cochrane ARI Group were of RCTs with no reviews. Download .xml (.0 MB) Help with xml files Data Profile