Performance of an artificial intelligence algorithm for reporting urine cytopathology.

Abstract

Unlike Papanicolaou tests, there are no commercially available computer-assisted automated screening systems for urine specimens. Despite The Paris System for Reporting Urinary Cytology, there still is poor interobserver agreement with urine cytology and many cases in which a definitive diagnosis cannot be made. In the current study, the authors have reported on the development of an image algorithm that applies computational methods to digitized liquid-based urine cytology slides. A total of 2405 archival ThinPrep glass slides, including voided and instrumented urine cytology cases, were digitized. A deep learning computational pipeline with multiple tiers of convolutional neural network models was developed for processing whole slide images (WSIs) and predicting diagnoses. The algorithm was validated using a separate test data set comprised of consecutive cases encountered in routine clinical practice. There were 1.9million urothelial cells analyzed. An average of 5400 urothelial cells were identified in each WSI. The algorithm achieved an area under the curve of 0.88 (95% CI, 0.83-0.93). Using the optimal operating point, the algorithm's sensitivity was 79.5% (95% CI, 64.7%-90.2%) and the specificity was 84.5% (95% CI, 81.6%-87.1%) for high-grade urothelial carcinoma. The authors successfully developed a computational algorithm capable of accurately analyzing WSIs of urine cytology cases. Compared with prior studies, this effort used a much larger data set, exploited whole slide-level and not just cell-level features, and used a cell gallery to display the algorithm's output for easy end-user review. This algorithm provides computer-assisted interpretation of urine cytology cases, akin to the machine learning technology currently used for automated Papanicolaou test screening.