The 2-week systematic review (2weekSR) method was successfully blind-replicated by another team: a case study

Abstract

ObjectiveTo assess the replicability of a 2-week systematic review (index 2weekSR) created with the assistance of automation tools using the fidelity method. MethodsA Preferred Reporting Items for Systematic reviews and Meta-Analyses compliant SR protocol was developed based on the published information of the index 2weekSR study. The replication team consisted of three reviewers. Two reviewers blocked off time during the replication. The total time to complete tasks and the meta-analysis results were compared with the index 2weekSR study. Review process fidelity scores (FSs) were calculated for review methods and outcomes. Barriers to completing the replication were identified. ResultsThe review was completed over 63 person-hours (11 workdays/15 calendar days). A FS of 0.95 was achieved for the methods, with 3 (of 8) tasks only partially replicated, and an FS of 0.63 for the outcomes, with 6 (of 7) only partially replicated and one task was not replicated. Nonreplication was mainly caused by missing information in the index 2weekSR study that was not required in standard reporting guidelines. The replication arrived at the same conclusions as the original study. ConclusionA 2weekSR study was replicated by a small team of three reviewers supported by automation tools. Including additional information when reporting SRs should improve their replicability.