Evidence in crisis: a closer look into the quality of published systematic reviews in the cardiology literature

Abstract

Abstract Background Systematic reviews are usually considered as the highest level of evidence and are increasingly used in shaping cardiology policies and guidelines. However, as the rate of publishing systematic reviews increases annually, there are rising concerns regarding their quality and reporting standards. Purpose The current analysis provides an insight into the quality of published systematic reviews in cardiology and provides recommendations for researchers, clinicians, and stakeholders in this regard. Methods Using a comprehensive Medline/PubMed search, we retrieved all systematic reviews, published between 2009 and 2019 in five general cardiology journals with the highest impact factor as per the Clarivate Analytics 2019 Journal Impact Factor List (Circulation, European Heart Journal, Journal of the American College of Cardiology, Circulation Research, and JAMA Cardiology). We assessed the methodological characteristics, eligibility criteria, reporting standards, as well as review quality scores according to the AMSTAR tool. Results Among 352 retrieved reviews, 275 (75.3%) performed direct head-to-head analysis and 164 (46.6%) included only clinical trials. The median numbers of searched databases and included studies were 3 (IQR: 2, 3) and 13 (IQR: 7, 30). The primary outcomes were often hard clinical endpoints as mortality (39.2%) and stroke (11.9%). 64 (18.2%) registered their protocol, 208 (58.4%) used validated tools for risk of bias assessment, 177 (52.3%) assessed for publication bias, and 221 (62.8%) adhered to the PRISMA checklist. Thirty-five reviews detected significant publication bias, which was significantly associated with heterogeneity of the primary outcome. The AMSTAR quality scores were low or critically low in 71% of evaluated reviews. Further, 87 (24.7%) did not report on whether they received funding or not, 33 (9.4%) reported receiving no funding, and 232 adequately reported on their funding sources [70 (19.9%) from governmental/academic sources, 120 (34.1%) from pharmaceutical companies, and 42 (11.9%) from both sources]. analysis showed that reviews with advanced statistical analysis, those that included RCTs, adhered to the PRISMA checklist, or had higher AMSTAR quality scores had significantly higher citation metrics (p<0.05). Conclusion Due to the widespread low quality and poor reporting in cardiovascular systematic reviews, clinicians should be educated on the value of methodological quality in interpreting systematic review findings. In addition, academic societies and guideline writing groups should implement rigorous critical appraisal and peer review policies to improve the synthesis and utilization of systematic reviews in evidence-based cardiovascular medicine. Funding Acknowledgement Type of funding sources: None.