Evaluating the Quality of Systematic Reviews & Meta-Analyses Published on Direct Oral Anticoagulants in the Past 5 Years

Abstract

Introduction In the past decades evidence-based medicine has begun to drive clinical decision making. With direct oral anti-coagulants (DOACs) emerging as alternatives to warfarin for the treatment and prevention of thromboembolic disorders, it has become crucial that clinicians utilize unbiased and robust evidence to inform their decisions about their use. Systematic reviews (SRs) sit at the "top" of the hierarchy of research evidence - the goal of this study was to evaluate the quality of SRs published on DOACs using AMSTAR criteria. Methods A comprehensive search of Medline, EMBASE, and the Cochrane Database of Systematic Reviews from Jan 2013 to February 2019 was performed. Screening was done across two stages with title and abstract followed by full-text analysis. Any study that was a SR (with or without a meta-analysis) published on DOACs was included. Data extracted included AMSTAR rating, journal of publication, year of publication, number of studies included, reporting adherence to PRISMA guidelines, number of citations, and journal of publication impact factor. Screening and data extraction were both done by two reviewers independently in duplicate. AMSTAR evaluation was done by three reviewers, one of which was a senior author. Statistical analyses comparing AMSTAR scores in relation to the above factors were done. Results A total of 3729 articles were found with 249 being included for analysis. Quality of SRs was highly variable across years with the mean (SD) being 5.68 (2.21). [Figure 1]. There were no significant relationships between quality vs citation rate (r=-0.04; 95% CI -0.17, 0.09; p=0.26) and impact factor (r=-0.05; 95% CI -0.18, 0.08; p=0.219). One-way ANOVA revealed no significant difference of AMSTAR scores between years (F6,242 = 1.85 p=0.09) [Figure 1.]. Reporting adherence to PRISMA guidelines increased the likelihood of being moderate (AMSTAR Score = 5-8) or high-quality evidence (AMSTAR Score = 9-11) (OR = 4.159; 95% CI 2.32, 7.46, p<0.01). Studies included/excluded in reviews (17, 7%) and conflicts of interests in both the review and included studies (21, 8%) were the least reported AMSTAR criteria while characteristics of included studies (226, 90%) and appropriate use of combining findings (217, 87) were the most. [Figure 2.] Conclusions The overall quality of SRs published on DOACs was moderately low and there was no relationship between journal impact factor and quality of the reviews that journals published. Our findings highlight specific areas within which authors can improve their reporting. Reviewers and editors of journals should familiarize themselves with AMSTAR criteria to ensure robust and transparent quality reporting in an effort to increase the quality of evidence being used to guide clinical decision making. Disclosures Crowther: Diagnostica Stago: Other: preparing educational material and/or providing educational presentations, Research Funding; Bayer: Other: Data and Safety Monitoring Board, Research Funding, Speakers Bureau; BMS Canada: Membership on an entity's Board of Directors or advisory committees, Research Funding; Servier Canada: Membership on an entity's Board of Directors or advisory committees, Research Funding, Speakers Bureau; Pfizer: Other: preparing educational material and/or providing educational presentations; CSL Behring: Other: preparing educational material and/or providing educational presentations; Asahi Kasei: Membership on an entity's Board of Directors or advisory committees; Octapharma: Membership on an entity's Board of Directors or advisory committees; Shionogi: Membership on an entity's Board of Directors or advisory committees; Alexion: Speakers Bureau; Alnylam: Equity Ownership.