Literature search in systematic reviews: How much is good enough?

Abstract

BackgroundSystematic reviews focus on performing exhaustive literature searches to address specific research questions. However, in practice, many reviews search a limited number of databases. ObjectiveThis analysis was undertaken to identify the potential impact of searching a limited number of databases in systematic reviews. MethodsTwo recently published systematic reviews addressing the same research question, but using different search strategies, were utilized as a case study. Both were published in 2023, less than 6 months apart. One of these searched Pubmed and Embase; whereas the other searched through Pubmed and the Cochrane Library. Information on the review characteristics, detailed methodology, search strategy, reviewers’ results and conclusions; were extracted. Both reviews were appraised for methodological quality using the AMSTAR-2 checklist. ResultsThe search output of the two reviews was vastly different. Using a step-wise screening process, the two reviews finally included 16 and 11 studies. Among these, only 4 studies were included in both reviews, suggesting that both reviews missed multiple eligible studies. The data on the primary outcomes were also very different in the two reviews, suggesting that neither could be considered completely reliable for decision-making by stakeholders. ConclusionThis analysis highlights the risk of bias arising in systematic reviews undertaking limited literature searches, especially limited number of databases for published literature. The commonly employed practice of searching two databases in systematic reviews, needs to be rectified urgently.