24 Quality appraisal of systematic reviews of HIV treatment adherence and guidance to reduce research waste

Abstract

Objectives Guideline developers and healthcare decision makers rely on high-quality evidence to make sound evidence-based decisions. The quality appraisal step is critical to ensuring a balanced representation of the evidence. The overall results of systematic reviews (SRs) should not be accepted as evidence-based if this step was performed inadequately. Impact factor is incorrectly being used as an indicator of the quality of papers. The purpose of this study is to systematically evaluate the quality of SRs that report the association between HIV adherence levels and specific outcomes, to determine the impact factor or reputation of the publication journal, and to provide guidance for reducing research waste. Method A literature search was conducted in April 2018 in Ovid MEDLINE, EMBASE, CINAHL, PubMed Central, the Cochrane Library, Science Citation Index, Web of Science, ScIELO Citation Index, and Ovid Emcare. Records were screened in Covidence by at least 2 reviewers using pre-specified eligibility criteria and definitions. Methodological quality of the reviews was assessed independently by 2 reviewers using the AMSTAR 2 tool; additional information about the literature searches and conflicts of interest was extracted. The quality assessment was qualitatively compared to the impact factor of the journals in which the papers were published. Results Our literature search identified 1141 unique records. Ultimately, 9 articles met our inclusion criteria. The overall confidence in the results of 78% of the SRs was critically low (1 critical flaw with or without non-critical weaknesses). Frequent problems identified were lack of protocols, incomplete literature searches, study selection and/or data extraction not done in duplicate, lack of formal quality appraisal tools, inadequate consideration of the effect of risk of bias of individual studies on results, and missing key information on populations, interventions, comparisons, outcomes, study designs included or rationale for studies included, funding sources, and conflicts. Impact factor or the reputation of a journal is not an indication of the overall quality of these SRs. Conclusions This research emphasizes the importance of using quality appraisal tools and reporting guidance. The majority of SRs do not meet quality standards despite the availability of tools and guidance. The number of published SRs is increasing. This does not necessarily translate to more precise answers based on high-quality evaluations of the evidence for the ultimate goal of improving healthcare decision-making and patient care. Low-quality evidence syntheses are a huge burden on everyone involved and may cause harm. All parties involved in healthcare decisions should require critical appraisal of evidence regardless of the reputation or impact factor of an author, organization, or journal, and be prepared to perform such an evaluation prior to using, applying, or distributing SRs. The problem can be corrected if we work together to find ways in which this can be done and continue to develop innovative methods and tools to streamline the SR process without compromising quality.