Should methodological filters for diagnostic test accuracy studies be used in systematic reviews of psychometric instruments? a case study involving screening for postnatal depression

Simon M Gilbody,Rachel Mann

doi:10.1186/2046-4053-1-9

Abstract

BackgroundChallenges exist when searching for diagnostic test accuracy (DTA) studies that include the design of DTA search strategies and selection of appropriate filters. This paper compares the performance of three MEDLINE search strategies for psychometric diagnostic test accuracy (DTA) studies in postnatal depression.MethodsA reference set of six relevant studies was derived from a forward citation search via Web of Knowledge. The performance of the 'target condition and index test' method recommended by the Cochrane DTA Group was compared to two alternative strategies which included methodological filters. Outcome measures were total citations retrieved, sensitivity, precision and associated 95% confidence intervals (95%CI).ResultsThe Cochrane recommended strategy and one of the filtered search strategies were equivalent in performance and both retrieved a total of 105 citations, sensitivity was 100% (95% CI 61%, 100%) and precision was 5.2% (2.6%, 11.9%). The second filtered search retrieved a total of 31 citations, sensitivity was 66.6% (30%, 90%) and precision was 12.9% (5.1%, 28.6%). This search missed the DTA study with most relevance to the DTA review.ConclusionsThe Cochrane recommended search strategy, 'target condition and index test', method was pragmatic and sensitive. It was considered the optimum method for retrieval of relevant studies for a psychometric DTA review (in this case for postnatal depression). Potential limitations of using filtered searches during a psychometric mental health DTA review should be considered.

Highlights

Challenges exist when searching for diagnostic test accuracy (DTA) studies that include the design of DTA search strategies and selection of appropriate filters
In this paper we examine whether a useful search strategy is available to identify studies for a DTA review of a brief psychometric measure for postnatal depression
Of the six papers identified as the reference set, only one paper examined the DTA of the questions compared to a ‘gold standard’ diagnostic criteria [20], only this study would be eligible for inclusion in a DTA review of the questions

Summary

Introduction

Challenges exist when searching for diagnostic test accuracy (DTA) studies that include the design of DTA search strategies and selection of appropriate filters. The design of optimum information retrieval strategies for recent developments such as Diagnostic Test Accuracy (DTA) reviews is not yet resolved; challenges that exist when searching for DTA studies have been acknowledged and include the design of DTA search strategies. The Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy [6] recommends that a search strategy for identification of DTA studies focus primarily on search terms in relation to the ‘target condition’ (for example, the illness or medical condition) and the ‘index test’ (for example, the new test to be compared to a gold standard or some other ‘reference’ test); no specific filter terms such as ‘sensitivity’ or ‘specificity’ are recommended.

Results

Discussion

Conclusion