Abstract
Objective: We compared the process of developing searches with and without using text-mining tools (TMTs) for evidence synthesis products.Study Design: This descriptive comparative analysis included seven systematic reviews, classified as simple or complex. Two librarians created MEDLINE strategies for each review, using either usual practice (UP) or TMTs. For each search we calculated sensitivity, number-needed-to-read (NNR) and time spent developing the search strategy.Results: We found UP searches were more sensitive (UP 92% (95% CI, 85-99); TMT 84.9% (95% CI, 74.4-95.4)), with lower NNR (UP 83 (SD 34); TMT 90 (SD 68)). UP librarians spent an average of 12 h (SD 8) developing search strategies, compared to TMT librarians’ 5 hours (SD 2).Conclusion: Across all reviews, TMT searches were less sensitive than UP searches, but confidence intervals overlapped. For simple SR topics, TMT searches were faster and slightly less sensitive than UP. For complex SR topics, TMT searches were faster and less sensitive than UP searches but identified unique eligible citations not found by the UP searches.
Highlights
Information retrieval methods research is needed to ensure efficient and effective systematic review (SR) search processes, especially given the tremendous growth in biomedical research publications of late (e.g., 70+% growth during 2010-2020 [1,2] in Medline and/or PubMed records alone)
usual practice (UP) searches generally followed the process detailed in Figure 1; text-mining tools (TMTs) searches followed a similar process but included text mining tools to develop keyword and/or phrase and subject terms searches
This study’s limitations include: (1) the small sample of reviews, which makes it hard to draw conclusions across projects; (2) search processes or review types of participating review teams may differ in meaningful ways from those teams who did not participate; (3) the small sample size of experienced SR librarians were relatively inexperienced with TMTs; (4) non-standardized comparisons; and 5) variability in search procedures (e.g., UP librarians could opt to use either PubMed or Ovid platforms and TMT librarians had a choice of tools and methods)
Summary
Information retrieval methods research is needed to ensure efficient and effective systematic review (SR) search processes, especially given the tremendous growth in biomedical research publications of late (e.g., 70+% growth during 2010-2020 [1,2] in Medline and/or PubMed records alone). An ‘objective’ approach to developing search strategies using TMTs has been adopted by Germany’s Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG) [7,8,9]. Hausner et al define their approach as comprising a set of steps: “generation of a total set (relevant references from SR), splitting of the total set into a development set and comparator set, development of the search strategy with references from the development set
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
