Evidence-based diagnostic accuracy measurement in urine cytology using likelihood ratios

Abstract

Recent cytology classification systems have become more evidence-based and advocate for the use of risk of malignancy (ROM) as a measure of test performance. From the statistical viewpoint, ROM represents the post-test probability of malignancy, which changes with the test result and also with the prevalence of malignancies (or pre-test probability) in each individual practice setting and individual patient presentation. Evidence-based medicine offers likelihood ratios (LRs) as a measure of diagnostic accuracy for multilevel diagnostic tests, superior to sensitivity and specificity as data binarization and information loss are avoided. LRs are used in clinical medicine and could be successfully applied to the practice of cytopathology. Our aim was to establish LRs to compare diagnostic accuracy of The Paris System for Reporting Urinary Cytology (TPS) and of a historic urine cytology reporting system. We analyzed sequential voided urine cytology cases with histologic outcomes: 188 pre-TPS and 167 post-TPS. LRs were calculated as LR = True positive % (per category)/False positive % (per category) [95% confidence interval] and interpreted LRs = 1 nondiagnostic, LR >1 favor, LR >10 strongly favor, LRs <1 favor exclusion, and LR <0.1 strongly favor exclusion of a target condition, respectively. CATmaker open source software and Fagan nomograms were used for calculation and visualization of the corresponding post-test probability (ROM) of high-grade urothelial carcinoma (HGUC) in various scenarios. Both reporting systems show near-similar performance in terms of LRs, with moderate discriminatory power of negative, suspicious, and positive for HGUC test results. The atypical urothelial cell (AUC) category establishes as indiscriminate LR = 1 in the TPS, whereas in pre-TPS it favored a benign condition. We further demonstrate the utility of LRs to determine individual post-test probability (ROM) in a variety of clinical scenarios in a personalized fashion. The LRs allow for a quantitative performance measure in case of urine cytology across different scenarios adding numeric information on diagnostic test accuracy and post-test probability of HGUC. The diagnostic accuracy of pre-TPS and post-TPS remained similar for all but the AUC category. With the TPS, the AUC category has become genuinely diagnostically and statistically indeterminate and requires further patient investigations.