Abstract
BackgroundSystematic reviews (SRs) must be of high quality. The purpose of our research was to compare the methodological and reporting quality of original versus updated Cochrane SRs to determine whether updating had improved these two quality dimensions.MethodsWe identifed updated Cochrane SRs published in issue 4, 2002 of the Cochrane Library. We assessed the updated and original versions of the SRs using two instruments: the 10 item enhanced Overview Quality Assessment Questionnaire (OQAQ), and an 18-item reporting quality checklist and flow chart based upon the Quality of Reporting of Meta-analyses (QUOROM) statement. At least two reviewers extracted data and assessed quality. We calculated the percentage (with a 95% confidence interval) of 'yes' answers to each question. We calculated mean differences in percentage, 95% confidence intervals and p-values for each of the individual items and the overall methodological quality score of the updated and pre-updated versions using OQAQ.ResultsWe assessed 53 SRs. There was no significant improvement in the global quality score of the OQAQ (mean difference 0.11 (-0.28; 0.70 p = 0.52)). Updated reviews showed a significant improvement of 18.9 (7.2; 30.6 p < .01) on the OQAQ item assessing whether the conclusions drawn by the author(s) were supported by the data and/or analysis presented in the SR. The QUOROM statement showed that the quality of reporting of Cochrane reviews improved in some areas with updating. Improvements were seen on the items relating to data sources reported in the abstract, with a significant difference of 17.0 (9.8; 28.7 p = 0.01), review methods, reported in the abstract 35 (24.1; 49.1 p = 0.00), searching methods 18.9 (9.7; 31.6 p = 0.01), and data abstraction 18.9 (11.7; 30.9 p = 0.00).ConclusionThe overall quality of Cochrane SRs is fair-to-good. Although reporting quality improved on certain individual items there was no overall improvement seen with updating and methodological quality remained unchanged. Further improvement of quality of reporting is possible. There is room for improvement of methodological quality as well. Authors updating reviews should address identified methodological or reporting weaknesses. We recommend to give full attention to both quality domains when updating SRs.
Highlights
Systematic reviews (SRs) must be of high quality
Jadad found that Cochrane reviews had greater methodological rigor, more frequent updates and higher overall quality scores than those published in peerreviewed paper journals, though both types were found to contain extensive and serious flaws
There were improvements on seven individual items, only one item showed a significant improvement (item 9 – Were the conclusions made by the author(s) supported by the data and/or analysis reported in the overview?) (percent original reviews complying with item 76% percentage updated reviews complying with item 94% (95% 88.1; 100.0) difference 18.9%)
Summary
Systematic reviews (SRs) must be of high quality. The purpose of our research was to compare the methodological and reporting quality of original versus updated Cochrane SRs to determine whether updating had improved these two quality dimensions. A number of papers have been published on the methodological and reporting quality of reviews. Shea [3] compared the methodological quality of paper-based and, electronic systematic reviews and found little difference and a lot of room for improvement. Assendelft [4] reviewed 51 reviews and noted that reviews that favoured a given intervention tended to have higher methodological quality scores. Jadad and McQuay [5] reviewed 80 systematic reviews published between 1980 and 1992 and found a disconcerting link between reviews whose results favoured an intervention and poor methodological quality. Jadad found that Cochrane reviews had greater methodological rigor, more frequent updates and higher overall quality scores than those published in peerreviewed paper journals, though both types were found to contain extensive and serious flaws
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
