Restrictions and their reporting in systematic reviews of effectiveness: an observational study

Abstract

BackgroundRestrictions in systematic reviews (SRs) can lead to bias and may affect conclusions. Therefore, it is important to report whether and which restrictions were used. This study aims to examine the use of restrictions regarding language, publication period, and study type, as well as the transparency of reporting in SRs of effectiveness.MethodsA retrospective observational study was conducted with a random sample of 535 SRs of effectiveness indexed in PubMed between 2000 and 2019. The use of restrictions and their reporting were analysed using descriptive statistics.ResultsOf the total 535 SRs included, four out of every ten (41.3%) lacked information on at least one of the three restrictions considered (language, publication period, or study type). Overall, 14.6% of SRs did not provide information on restrictions regarding publication period, 19.1% regarding study type, and 18.3% regarding language. Of all included SRs, language was restricted in 46.4%, and in more than half of the SRs with restricted language (130/248), it was unclear whether the restriction was applied during either the search or the screening process, or both. The restrictions were justified for publication period in 22.2% of the respective SRs (33/149), study type in 6.5% (28/433), and language in 3.2% (8/248). Differences in reporting were found between countries as well as between Cochrane and non-Cochrane reviews.ConclusionsThis study suggests that there is a lack of transparency in reporting on restrictions in SRs. Authors as well as editors and reviewers should be encouraged to improve the reporting and justification of restrictions to increase the transparency of SRs.