Cochrane Diagnostic Test Accuracy Reviews Research Articles

Diagnosis of placenta accreta spectrum in high-risk women using ultrasonography or magnetic resonance imaging: systematic review and meta-analysis.

To perform a systematic review and meta-analysis of the diagnostic test accuracy of ultrasound and magnetic resonance imaging (MRI) and compare the performance of the two modalities in the diagnosis of placenta accreta spectrum (PAS). This was a systematic review conducted following the Cochrane Diagnostic Test Accuracy Reviews guideline. A literature search was performed in five databases: PubMed, EMBASE, PMC, The Cochrane Library and BVS-Bireme between 27 July and 4 August 2020. The search was updated on 18 August 2021. We included observational studies evaluating diagnostic accuracy in women with risk factors for PAS who had undergone both ultrasound and MRI examinations, published in English between 2011 and 2021. Quality Assessment of Diagnostic Accuracy Studies-2 was used to evaluate the quality of the studies. Forest plots for sensitivity and specificity with 95% CIs and receiver-operating-characteristics curves for ultrasound and MRI were constructed. The literature search identified 266 studies. After reviewing the titles and abstracts of the articles, 51 were selected for full-text review and 17 studies including 1301 women with MRI and ultrasound data available were selected for the meta-analysis. The study population included 457 cases with PAS diagnosed using the gold standard method (intraoperative or histopathological analysis). The overall quality of the evaluated studies was considered satisfactory according to QUADAS-2. The meta-analysis revealed a sensitivity of 0.833 (95% CI, 0.776-0.878) and specificity of 0.834 (95% CI, 0.746-0.897) for ultrasound. For MRI, sensitivity was 0.838 (95% CI, 0.786-0.879) and specificity was 0.831 (95% CI, 0.770-0.878). There was no statistically significant difference between the two modalities. The Cochran's Q values indicated a high level of heterogeneity of sensitivity and specificity of ultrasound and MRI across studies. Ultrasound and MRI have similar accuracy in the diagnosis of PAS. These results suggest that, in a setting with a high prevalence of risk factors, the choice of imaging modality for initial screening for PAS should depend on the availability of equipment and the examiner's expertise. © 2022 International Society of Ultrasound in Obstetrics and Gynecology.

Training sample selection: Impact on screening automation in diagnostic test accuracy reviews.

When performing a systematic review, researchers screen the articles retrieved after a broad search strategy one by one, which is time‐consuming. Computerised support of this screening process has been applied with varying success. This is partly due to the dependency on large amounts of data to develop models that predict inclusion. In this paper, we present an approach to choose which data to use in model training and compare it with established approaches. We used a dataset of 50 Cochrane diagnostic test accuracy reviews, and each was used as a target review. From the remaining 49 reviews, we selected those that most closely resembled the target review's clinical topic using the cosine similarity metric. Included and excluded studies from these selected reviews were then used to develop our prediction models. The performance of models trained on the selected reviews was compared against models trained on studies from all available reviews. The prediction models performed best with a larger number of reviews in the training set and on target reviews that had a research subject similar to other reviews in the dataset. Our approach using cosine similarity may reduce computational costs for model training and the duration of the screening process.

New measures improved the reporting of heterogeneity in diagnostic test accuracy reviews: a metaepidemiological study

ObjectivesTo describe the methods used to quantify heterogeneity and to propose alternative measures to improve reporting of heterogeneity in Cochrane diagnostic test accuracy (DTA) reviews. Study Design and SettingOur metaepidemiological study included all DTA reviews in the Cochrane Library up to October 6th, 2019. We summarized reviews’ characteristics focusing on heterogeneity analysis. We selected reviews with a bivariate model and ≥4 studies for reanalysis. In this group, we fitted bivariate random effects models and we quantified heterogeneity by means of logit variances of sensitivity and specificity, bivariate I2, median odds ratio (OR), and the area of the 95% prediction ellipse. We provided a narrative interpretation of these measures in different scenarios. ResultsThere were 124 Cochrane DTA reviews of which 91 (73%) included meta-analysis. Only in 5 meta-analyses, variances of the logit sensitivity and specificity were reported, and in 21 meta-analyses (23%), the 95% prediction ellipse was reported without any calculation of its area. We selected 60 of these 91 reviews to explore the behavior of all measures of heterogeneity. We found that most reviews described the subjective heterogeneity as moderate or extreme (n = 31/60, 52%), whereas the area of the 95% prediction ellipse and the median OR for sensitivity and specificity showed high variability; the area ranged from 5% to 97%, the median OR of sensitivity ranged from 1.13 to 10.7, and the median OROR of specificity ranged from 1.18 to 19.68. ConclusionCochrane DTA reviews show a poor reporting of between-study heterogeneity. Using median OR and the area of the 95% prediction ellipse will improve reporting and interpretation of this crucial aspect of DTA meta-analysis.

Data sources and methods used to determine pretest probabilities in a cohort of Cochrane diagnostic test accuracy reviews

BackgroundA pretest probability must be selected to calculate data to help clinicians, guideline boards and policy makers interpret diagnostic accuracy parameters. When multiple analyses for the same target condition are compared, identical pretest probabilities might be selected to facilitate the comparison. Some pretest probabilities may lead to exaggerations of the patient harms or benefits, and guidance on how and why to select a specific pretest probability is minimally described. Therefore, the aim of this study was to assess the data sources and methods used in Cochrane diagnostic test accuracy (DTA) reviews for determining pretest probabilities to facilitate the interpretation of DTA parameters. A secondary aim was to assess the use of identical pretest probabilities to compare multiple meta-analyses within the same target condition.MethodsCochrane DTA reviews presenting at least one meta-analytic estimate of the sensitivity and/or specificity as a primary analysis published between 2008 and January 2018 were included. Study selection and data extraction were performed by one author and checked by other authors. Observed data sources (e.g. studies in the review, or external sources) and methods to select pretest probabilities (e.g. median) were categorized.ResultsFifty-nine DTA reviews were included, comprising of 308 meta-analyses. A pretest probability was used in 148 analyses. Authors used included studies in the DTA review, external sources, and author consensus as data sources for the pretest probability. Measures of central tendency with or without a measure of dispersion were used to determine the pretest probabilities, with the median most commonly used. Thirty-two target conditions had at least one identical pretest probability for all of the meta-analyses within their target condition. About half of the used identical pretest probabilities were inside the prevalence ranges from all analyses within a target condition.ConclusionsMultiple sources and methods were used to determine (identical) pretest probabilities in Cochrane DTA reviews. Indirectness and severity of downstream consequences may influence the acceptability of the certainty in calculated data with pretest probabilities. Consider: whether to present normalized frequencies, the influence of pretest probabilities on normalized frequencies, and whether to use identical pretest probabilities for meta-analyses in a target condition.

Lilacs search strategy for systematic reviews of diagnostic test accuracy studies.

There are few publications on search strategies to identify diagnostic test accuracy (DTA) studies in lilacs. To translate and customise medline search strategies for use in lilacs and assess their retrieval of studies in Cochrane DTA systematic reviews. We developed a six-step process to translate and customise medline search strategies for use in lilacs (iAHx interface). We identified medline search strategies of published Cochrane DTA reviews, translated/customised them for use in lilacs, ran searches in lilacs and compared the retrieval results of our translated search strategy versus the one used in the published reviews. Our lilacs search strategies translated/customised from the medline strategies retrieved studies in 70 Cochrane DTA reviews. Only 29 of these reviews stated that they had searched the lilacs database and 21 published their lilacs search strategies. Few had used the lilacs database search tools, none exploded the subject headings, and 86% used only English terms. Translating and tailoring a medline search strategy for the lilacs database resulted in the retrieval of DTA studies that would have been missed otherwise.

Defining the clinical pathway in cochrane diagnostic test accuracy reviews.

BackgroundThe value of a medical test depends on the context in which it might be used. Ideally, questions, results and conclusions of a diagnostic test accuracy (DTA) systematic review should be presented in light of this context. There is increasing acceptance of the value for knowing the impact a test can have on downstream consequences such as costs, implications for further testing and treatment options however there is currently no explicit guidance on how to address this. Authors of a Cochrane diagnostic review have recently been asked to include the clinical pathway in which a test maybe used. We aimed to evaluate how authors were developing their clinical pathways in the light of this.MethodsWe searched the Cochrane Database of Systematic Reviews for all published DTA reviews. We included only those reviews that included a clinical pathway. We developed a checklist, based on the guidance in the Cochrane Handbook for DTA review authors. To this, we added a number of additional descriptors. We checked if the included pathways fulfilled these descriptors as defined by our checklist.ResultsWe found 47 reviews, of which 33 (73 %) contained aspects pertaining to a clinical pathway. The 33 reviews addressed the clinical pathway differently, both in content and format. Of these, 21 provided a textual description and 12 include visual and textual descriptions. There was considerable variation in how comprehensively review authors adhered to our checklist. Eighteen reviews (51 %) linked the index test results to downstream clinical management actions and patient consequences, but only eight went on to differentially report on the consequences for false negative results and nine on the consequences for false positive results.ConclusionThere is substantial variation in the clinical pathway descriptions in Cochrane systematic reviews of test accuracy. Most reviews do not link misclassifications (i.e. false negatives and false positive) to downstream patient consequences. Review authors could benefit from more explicit guidance on how to create such pathways, which in turn can help guide them in their evidence selection and appraisal of the evidence in the context of downstream consequences of testing.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-016-0252-x) contains supplementary material, which is available to authorized users.

Increased workload for systematic review literature searches of diagnostic tests compared with treatments: challenges and opportunities.

BackgroundComprehensive literature searches are conducted over multiple medical databases in order to meet stringent quality standards for systematic reviews. These searches are often very laborious, with authors often manually screening thousands of articles. Information retrieval (IR) techniques have proven increasingly effective in improving the efficiency of this process. IR challenges for systematic reviews involve building classifiers using training data with very high class-imbalance, and meeting the requirement for near perfect recall on relevant studies. Traditionally, most systematic reviews have focused on questions relating to treatment. The last decade has seen a large increase in the number of systematic reviews of diagnostic test accuracy (DTA).ObjectiveWe aim to demonstrate that DTA reviews comprise an especially challenging subclass of systematic reviews with respect to the workload required for literature screening. We identify specific challenges for the application of IR to literature screening for DTA reviews, and identify potential directions for future research.MethodsWe hypothesize that IR for DTA reviews face three additional challenges, compared to systematic reviews of treatments. These include an increased class-imbalance, a broader definition of the target class, and relative inadequacy of available metadata (ie, medical subject headings (MeSH) terms for medical literature analysis and retrieval system online). Assuming these hypotheses to be true, we identify five manifestations when we compare literature searches of DTA versus treatment. These manifestations include: an increase in the average number of articles screened, and increase in the average number of full-text articles obtained, a decrease in the number of included studies as a percentage of full-text articles screened, a decrease in the number of included studies as a percentage of all articles screened, and a decrease in the number of full-text articles obtained as a percentage of all articles screened. As of July 12 2013, 13 published Cochrane DTA reviews were available and all were included. For each DTA review, we randomly selected 15 treatment reviews published by the corresponding Cochrane Review Group (N=195). We then statistically tested differences in these five hypotheses, for the DTA versus treatment reviews.ResultsDespite low statistical power caused by the small sample size for DTA reviews, strong (P<.01) or very strong (P<.001) evidence was obtained to support three of the five expected manifestations, with evidence for at least one manifestation of each hypothesis. The observed difference in effect sizes are substantial, demonstrating the practical difference in reviewer workload.ConclusionsReviewer workload (volume of citations screened) when screening literature for systematic reviews of DTA is especially high. This corresponds to greater rates of class-imbalance when training classifiers for automating literature screening for DTA reviews. Addressing concerns such as lower quality metadata and effectively modelling the broader target class could help to alleviate such challenges, providing possible directions for future research.

Applying Grading of Recommendations Assessment, Development and Evaluation (GRADE) to diagnostic tests was challenging but doable

ObjectivesThe Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group developed an approach to assess the quality of evidence of diagnostic tests. Its use in Cochrane diagnostic test accuracy reviews is new. We applied this approach to three Cochrane reviews with the aim of better understanding the application of the GRADE criteria to such reviews. Study Design and SettingWe selected reviews to achieve clinical and methodological diversities. At least three assessors independently assessed each review according to the GRADE criteria of risk of bias, indirectness, imprecision, inconsistency, and publication bias. Two teleconferences were held to share experiences. ResultsFor the interpretation of the GRADE criteria, it made a difference whether assessors looked at the evidence from a patient-important outcome perspective or from a test accuracy standpoint. GRADE criteria such as inconsistency, imprecision, and publication bias were challenging to apply as was the assessment of comparative test accuracy reviews. ConclusionThe perspective from which evidence is graded can influence judgments about quality. Guidance on application of GRADE to comparative test reviews and on the GRADE criteria of inconsistency, imprecision, and publication bias will facilitate the operationalization of GRADE for diagnostics.

Cochrane diagnostic test accuracy reviews

In 1996, shortly after the founding of The Cochrane Collaboration, leading figures in test evaluation research established a Methods Group to focus on the relatively new and rapidly evolving methods for the systematic review of studies of diagnostic tests. Seven years later, the Collaboration decided it was time to develop a publication format and methodology for Diagnostic Test Accuracy (DTA) reviews, as well as the software needed to implement these reviews in The Cochrane Library. A meeting hosted by the German Cochrane Centre in 2004 brought together key methodologists in the area, many of whom became closely involved in the subsequent development of the methodological framework for DTA reviews. DTA reviews first appeared in The Cochrane Library in 2008 and are now an integral part of the work of the Collaboration.

Cochrane Column * Rapid diagnostic tests can extend access of diagnostic services for uncomplicated Plasmodium falciparum malaria * Commentary: Rapid diagnostic tests for diagnosing uncomplicated Plasmodium falciparum malaria in endemic countries (Review) * Approach to conducting Cochrane Diagnostic Test Accuracy Reviews

Cochrane Column * Rapid diagnostic tests can extend access of diagnostic services for uncomplicated Plasmodium falciparum malaria * Commentary: Rapid diagnostic tests for diagnosing uncomplicated Plasmodium falciparum malaria in endemic countries (Review) * Approach to conducting Cochrane Diagnostic Test Accuracy Reviews