Search methods for prognostic factor systematic reviews: a methodologic investigation.

Leah Boulos,Jill A Hayden,Rachel Ogilvie

doi:10.5195/jmla.2021.939

Leah Boulos, Jill A Hayden + Show 1 more

Open Access

https://doi.org/10.5195/jmla.2021.939

Copy DOI

Abstract

Objective:This study retroactively investigated the search used in a 2019 review by Hayden et al., one of the first systematic reviews of prognostic factors that was published in the Cochrane Library. The review was designed to address recognized weaknesses in reviews of prognosis by using multiple supplementary search methods in addition to traditional electronic database searching.Methods:The authors used four approaches to comprehensively assess aspects of systematic review literature searching for prognostic factor studies: (1) comparison of search recall of broad versus focused electronic search strategies, (2) linking of search methods of origin for eligible studies, (3) analysis of impact of supplementary search methods on meta-analysis conclusions, and (4) analysis of prognosis filter performance.Results:The review's focused electronic search strategy resulted in a 91% reduction in recall, compared to a broader version. Had the team relied on the focused search strategy without using supplementary search methods, they would have missed 23 of 58 eligible studies that were indexed in MEDLINE; additionally, the number of included studies in 2 of the review's primary outcome meta-analyses would have changed. Using a broader strategy without supplementary searches would still have missed 5 studies. The prognosis filter used in the review demonstrated the highest sensitivity of any of the filters tested.Conclusions:Our study results support recommendations for supplementary search methods made by prominent systematic review methodologists. Leaving out any supplemental search methods would have resulted in missed studies, and these omissions would not have been prevented by using a broader search strategy or any of the other prognosis filters tested.

Highlights

Systematic reviews of prognosis studies are being published at an increasing rate. Such reviews typically address one or more of the following aims: identify the most likely course for a specific condition [1]; identify what characteristics are associated with, or predict, a given outcome [2]; identify cohorts with specific characteristics who are more or less likely to experience a given outcome [3]; and/or identify the characteristics or factors that impact the effectiveness of a specific treatment [4]
In addition to database searching, Hayden et al used multiple supplementary search methods: a reference search of previously published systematic reviews, which were retrieved using a broader electronic search that did not include the prognostic factor concept; a forward search of publications citing identified prognostic factor measures; and hand searching, reference searching of included studies, and consultation of personal files to identify additional studies
To determine which studies would have been missed had any of the search strategy components been left out, we linked each included study in Hayden et al to its search methods of origin, focusing only on those studies indexed in Ovid MEDLINE (n=58 of 60 total included studies)

Summary

Introduction

Systematic reviews of prognosis studies are being published at an increasing rate Such reviews typically address one or more of the following aims: identify the most likely course for a specific condition (overall prognosis) [1]; identify what characteristics are associated with, or predict, a given outcome (prognostic factors) [2]; identify cohorts with specific characteristics who are more or less likely to experience a given outcome (predictive models) [3]; and/or identify the characteristics or factors that impact the effectiveness of a specific treatment (treatment effect modification) [4].

Methods

Results

Conclusion