Effect of an artificial intelligence–powered programmed death-ligand 1 combined positive score analyzer in urothelial cancer on inter-observer and inter-site variability.

Abstract

e13546 Background: Programmed death-ligand 1 (PD-L1) is a predictive marker for immune checkpoint inhibitors treatment response in urothelial carcinoma (UC). The combined positive score (CPS) is a representative method to evaluate the expression level of PD-L1 in UC. However, inter-observer and inter-institute variations can disrupt accurate CPS evaluation. The purpose of this study is to assess the role of an artificial intelligence (AI)-powered PD-L1 CPS analyzer on UC in reducing inter-observer and inter-institute variability. Methods: Lunit SCOPE PD-L1 CPS was developed with 4.94 x 105 tumor cells and 4.17 x 105 immune cells from 360 PD-L1 stained whole-slide images (WSIs) of UC from multiple institutions. The algorithm consisted of tissue area segmentation and cell detection AI models. The AI models calculated the CPS by detecting tumor cells over the tumor area and immune cells over the tumor and adjacent area. Three uropathologists from different university hospitals evaluated the CPS classification (≥10 or <10) of 543 PD-L1 stained WSIs of UC from each hospital. The result with concordant CPS classification of each slide across ≥ 2 pathologists was considered the consensus. Each pathologist revisited to evaluate WSIs by referencing the AI model inference, after a washout period, if there was a discrepancy between the pathologist and the AI model. Results: Of 543 WSIs, 446 (82.1%) were classified as the same CPS subgroup by all three uropathologists. Also, pathologists had a high degree of concordance with the consensus in WSIs from their own hospitals. They re-evaluated 64, 73, and 75 WSIs with AI assistance, respectively, and changed the CPS classification for 47, 48, and 48 WSIs. After re-evaluation with AI assistance, three uropathologists agreed on the same CPS classification in 510 WSIs (93.9%). The overall percentage agreement (OPA) of each pathologist with the consensus increased from 95.0%, 94.8%, and 92.3% to 98.7%, 98.3%, and 96.9% after AI assistance, and the OPA for WSIs of other institutions increased more compared to the OPA for WSIs of their own hospital. Conclusions: This study shows that an AI-powered PD-L1 CPS analyzer in UC can reduce inter-observer and inter-site variability. This result suggests that the AI model will help evaluate CPS in UC more accurately and reduce variation in situations where pathologists analyze WSIs from unfamiliar institutions.[Table: see text]