Abstract
BackgroundSystematic reviews are pivotal components in the development of evidence-based clinical guidelines. To evaluate the methodological quality of these systematic reviews, several tools have been proposed. Among them, the assessment of multiple systematic reviews (AMSTAR) checklist is probably used most frequently. This tool comprises 11 items related to the steps taken when conducting a systematic review, and it is claimed to have good face and content validity. The objective of this debate paper was twofold: (a) to critically evaluate the ability of all AMSTAR checklist items to adequately determine the methodological quality of a systematic review; and (b) to describe difficulties regarding interpretation of the checklist, and provide potential solutions for these difficulties.DiscussionSome items of the AMSTAR checklist seem to assess the quality of reporting of a systematic review more than its methodological quality. For example, item 7 may not “capture” the true methodological quality of primary studies included in the systematic review. Item 10 does not likely result in the collection of in-depth information on the presence of publication bias in the systematic review. Furthermore, some items may be difficult to interpret, hindering accurate assessment. For example, item 5 does not explicitly indicate whether a list of documents excluded in each phase of selection (i.e., after evaluation of titles and abstracts, and after full-text assessment) should be reported.SummaryThe present debate paper evaluated and discussed some methodological limitations of the AMSTAR checklist, as well as challenges involved in evaluation of the checklist’s items. Several suggestions are also made to optimize the use of this checklist. The information in this paper may stimulate further discussion among systematic reviewers, methodologists and clinicians.
Highlights
Systematic reviews are pivotal components in the development of evidence-based clinical guidelines
The authors of the assessment of multiple systematic reviews (AMSTAR) checklist explicitly state the rationale for the inclusion of each item, clear guidance on the use of the items to evaluate a systematic review is lacking in some cases
The main purpose of this debate paper is to critically evaluate the ability of all AMSTAR checklist items to adequately determine the methodological quality of a systematic review
Summary
The question “was the conflict of interest included?” does not specify who should declare the COI, and the note suggests that reporting of sources of funding or support for the systematic review and primary studies is sufficient This item should be clarified to enable more indepth assessment. COI involves, for example, a strong relationship between study participants and the enterprise(s) supporting the study (e.g., researchers owning stocks in a company or serving on its board) Summary This debate paper presents a critical appraisal of the characteristics of the AMSTAR checklist, a validated and recognized tool for the evaluation of the methodological quality of SRs. Some challenges of assessment have been described from the point of view of the assessor, and potential solutions to improve the reliability of the tool and homogeneity of its use have been offered. The author is member of the Cochrane Bias Methods Group and GRADE Working Group
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
