Abstract
Objective:Reproducibility of systemic reviews (SRs) can be hindered by the presence of citation bias. Citation bias may occur when authors of SRs conduct hand-searches of included study reference lists to identify additional studies. Such a practice may lead to exaggerated SR summary effects. The purpose of this paper is to examine the prevalence of hand-searching reference lists in otolaryngology SRs.Methods:The authors searched for systematic reviews published in eight clinical otolaryngology journals using the Cochrane Library and PubMed, with the date parameter of January 1, 2008, to December 31, 2017. Two independent authors worked separately to extract data from each SR for the following elements: whether reference lists were hand-searched, other kinds of supplemental searching, PRISMA adherence, and funding source. Following extraction, the investigators met to review discrepancies and achieve consensus.Results:A total of 539 systemic reviews, 502 from clinical journals and 37 from the Cochrane library, were identified. Of those SRs, 72.4% (390/539) hand-searched reference lists, including 97.3% (36/37) of Cochrane reviews. For 228 (58.5%) of the SRs that hand-searched reference lists, no other supplemental search (e.g., search of trial registries) was conducted.Conclusions:These findings indicate that hand-searching reference lists is a common practice in otolaryngology SRs. Moreover, a majority of studies at risk of citation bias did not attempt to mitigate the bias by conducting additional supplemental searches. The implication is that summary effects in otolaryngology systematic reviews may be biased toward statistically significant findings.
Highlights
Systematic reviews (SRs) use comprehensive methodologies to summarize a body of evidence on a clinical topic and, when meta-analysis is appropriate, produce a pooled effect estimate for the included primary studies [1]
While many aspects of the SR process may lead to bias, among the most important steps is the systematic search to locate eligible studies, which can lead to sampling or selection bias if the studies retrieved during the search process do not represent the population of available studies [3]
According to the Cochrane Handbook for Systematic Reviews of Interventions, hand-searching reference lists of included studies may lead to the selective inclusion of statistically significant studies with effect sizes similar to other published studies retrieved from database searching [4]
Summary
Systematic reviews (SRs) use comprehensive methodologies to summarize a body of evidence on a clinical topic and, when meta-analysis is appropriate, produce a pooled effect estimate for the included primary studies [1]. While many aspects of the SR process may lead to bias, among the most important steps is the systematic search to locate eligible studies, which can lead to sampling or selection bias if the studies retrieved during the search process do not represent the population of available studies [3]. According to the Cochrane Handbook for Systematic Reviews of Interventions, hand-searching reference lists of included studies may lead to the selective inclusion of statistically significant studies with effect sizes similar to other published studies retrieved from database searching [4]. Hand-searching reference lists may result in exaggerated SR effect estimates
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
