Predicting the time needed for environmental systematic reviews and systematic maps.

Abstract

Systematic reviews (SRs) and systematic mapping aim to maximize transparency and comprehensiveness while minimizing subjectivity and bias. These are time-consuming and complex tasks, so SRs are considered resource intensive, but published estimates of systematic-review resource requirements are largely anecdotal. We analyzed all Collaboration for Environmental Evidence (CEE) SRs (n = 66) and maps (n = 20) published from 2012 to 2017 to estimate the average number of articles retained at each review stage. We also surveyed 33 experienced systematic reviewers to collate information on the rate at which those stages could be completed. In combination, these data showed that the average CEE SR takes an estimated 164 d (full-time equivalent) (SD 23), and the average CEE systematic map (SM) (excluding critical appraisal) takes 211 d (SD 53). While screening titles and abstracts is widely considered time-consuming, metadata extraction and critical appraisal took as long or longer to complete, especially for SMs. Given information about the planned methods and evidence base, we created a software tool that predicts time requirements of a SR or map with evidence-based defaults as a starting point. Our results shed light on the most time-consuming stages of the SR and mapping processes, will inform review planning, and can direct innovation to streamline processes. Future predictions of effort required to complete SRs and maps could be improved if authors provide more details on methods and results.