Application of the Composite Quality Score (CQS-2B) versus Cochrane’s Risk of Bias tool (Version 2) in systematic reviews of clinical trials – an exploratory study

Abstract

ObjectivesTo explore whether systematic review conclusions generated from Cochrane’s second version of its Risk of Bias tool (RoB 2) for trial appraisal differ when the Composite Quality Score, Version 2.B (CQS-2B) is used instead and to develop a testable hypothesis based on these findings.MethodsPubMed was searched for one single systematic review. From the review’s accepted trials, data concerning effect estimates and overall bias risk according to the RoB 2 tool were extracted. All trial reports were appraised again using the CQS-2B. Datasets were stratified according to overall bias risk (RoB 2) or corroboration (C-) level (CQS-2B). The effect estimates from trials with ‘low bias risk’ (RoB 2) and with highest C-level (CQS-2B) were pooled separately. These pooled effect estimates were statistically and all clinical conclusions qualitatively compared.ResultsThe pooled effect estimates for trials with ‘low bias risk’ (RoB 2) were −0.07, 95% CI: −0.10 to −0.04 (I2 = 0.0%) and for the highest C-levels (CQS-2B) 0.08, 95% CI: −0.12 to −0.04 (I2 = 57.0%). The difference was statistically not significant (p = 0.70). Contrary to the RoB 2 tool, no clinical conclusions in line with the CQS-2B were made, because the effect estimates were judged to be erroneously overestimated, due to high risk of bias.ConclusionA testable hypothesis was generated suggesting that trial appraisal using the CQS-2B may provide more conservative conclusions based on similar data than with the RoB 2 tool.