OP46 Development and validation of a sensitive search filter to identify equity-focused studies

Abstract

Background Health inequalities, worse health associated with social and economic disadvantage, are reported by a minority of research articles. For example, fewer than 15% of intervention studies report any outcome by a social determinant of health. Systematic reviewers wishing to conduct an equity review are hampered in identifying these studies by a deficit in standardised terminology, uncertainties around indexing and an absence of validated search filters. Current reporting guidelines recommend not to apply equity filters. The resulting manual effort needed to screen for reports of equity can be considerable. We aimed to design and test the sensitivity of search filters to locate studies that reported outcomes by a social determinant of health. Methods We developed and expanded a ‘specific terms strategy’ using keywords and subject headings compiled from systematic reviews that applied an equity filter. A ‘non-specific strategy’ was compiled from phrases used to describe equity analyses reported in titles and abstracts, and related subject headings. Gold standard papers were independently collated using three different methods and randomly split into evaluation and (internal) validation sets. We set a target of 0.90 sensitivity (95% binomial confidence interval; 0.84, 0.94) in retrieving 150 gold standard validation papers. The filters were developed in MEDLINE, adapted for Embase and tested in both. We noted the reduction in the number needed to screen in a proposed equity-focused systematic review and the proportion of equity-focused systematic reviews (published in 2014–2017) we assessed in the project that applied an equity filter to their search strategy. Results The specific terms strategy filtered out 93%–95% of all records, and had a sensitivity for retrieving the validation set of articles 0.84 in MEDLINE (0.77, 0.89), and 0.87 (0.81, 0.92) in Embase. When combined (Boolean ‘OR’) with the non-specific strategy, sensitivity was 0.92 (0.86, 0.96) in MEDLINE and 0.94 (0.89, 0.97) in Embase. After applying the specific terms strategy the number needed to screen was reduced by 77%, and by 59.7% (MEDLINE) and 63.5% (Embase) after applying the combined strategy. Of the 128 reviews we assessed in the project, 83% applied an equity filter to their search strategy. Conclusion A combined approach of specific and non-specific terms is recommended if systematic reviewers wish to filter studies for reporting outcomes by social determinants. Our filters are lengthy and focus on sensitivity. Future research should examine specificity and validate using external sources. Indexing standardisation improvement is needed for equity studies.