Updating reviews: commitments and opportunities

Abstract

Unlike primary research, all reviews, whether systematicor not, have a limited ‘shelf-life’. As new findings of pri-mary research are reported, dated review findings be-come less reliable as an assessment of the best availableevidence. This can be a particular issue when reviews re-port a quantitative synthesis that provides combinedmeans of effects of interventions or impacts of expo-sures. Combined means can be changed substantially bythe addition of findings from a new, large, well designedstudy. This is particularly true when the evidence base isweak as is often the case in environmental management.Equally, new findings may enable a quantitative synthe-sis, when the previously published review may have con-cluded that no quantitative synthesis was possible. Forthese reasons a key component of systematic review(SR) methodology is the commitment to keep SRs up-dated on an appropriate timescale so that they track thedevelopment of the evidence base. Collaboration forEnvironmental Evidence (CEE) guidelines note that outof date SRs can be misleading and suggest an averagetime period of five years for updates [1]. However, sci-ence in different areas of environmental managementadvances at different rates and so five years is very mucha guideline and not a rule.The guidelines go on to suggest that:– ifareviewisfiveormoreyearsoutofdate,theCEEeditorial team will contact the authors inviting themto update the review.– if the authors are unable to take up this invitation,the review will be marked as ‘update sought’ andupdates will be open to any interested party.– in the case that a new review team is formed toupdate a review, they will be expected to liaise asmuch as possible with the original team, who mayalso be named as authors in the updated review toreflect the intellectual input into the review as awhole.Registration of an update, as with a new SR, is throughthe submission of a protocol. The protocol should citethe original protocol and be clear about how the newone differs from (and possibly improves on) the old. Up-dating is also an opportunity to learn from previous ef-forts and to improve methodology. It is therefore notexpected that an update will be a faithful repetition ofthe original. However, changes should be highlightedand explained.The CEE Library of completed reviews (www.environ-mentalevidence.org/Reviews.html) includes over 20 SRsthat are more than five years old and none have so farbeen updated. The majority were conducted as part ofthe process of developing systematic review method-ology for environmental management and could nodoubt be improved in many ways. Updates can thus bein the form of methodology, as reflected in the devel-opment of CEE Guidelines (now at version 4.2), as wellas adding new research findings. Some reviews can beeven more dated than their publication date suggestsas the searches may have been conducted years earlier.The first example of an update in progress has recentlybeen published as a protocol [2] and relates to a CEESR originally published in 2010 in which the searchwas conducted in 2008 [3].Beside updating CEE SRs, an exciting opportunity existsto update other reviews and meta-analyses to meet CEEstandards. In general, published reviews and meta-analysesare of very variable standard [4] and the raising of thesestandards is a key objective of CEE. It seems sensible thento have dual objectives of updating old reviews by addingnew findings and updating methodology and conduct ofthe review to CEE standards. We call on authors of re-views and meta-analyses to consider if it is the right timeto update their review and to register their protocol withCEE. Again, registration would be the same process exceptthat the protocol would likely be entirely new and referonly to the former review article as a basis for the CEE SR.Of course many original authors may not be motivated toperiodically update a review and so I would like toencourage anyone interested in conducting a SR and