Abstract
Artificial intelligence (AI) algorithms continue to rival human performance on a variety of clinical tasks, while their actual impact on human diagnosticians, when incorporated into clinical workflows, remains relatively unexplored. In this study, we developed a deep learning-based assistant to help pathologists differentiate between two subtypes of primary liver cancer, hepatocellular carcinoma and cholangiocarcinoma, on hematoxylin and eosin-stained whole-slide images (WSI), and evaluated its effect on the diagnostic performance of 11 pathologists with varying levels of expertise. Our model achieved accuracies of 0.885 on a validation set of 26 WSI, and 0.842 on an independent test set of 80 WSI. Although use of the assistant did not change the mean accuracy of the 11 pathologists (p = 0.184, OR = 1.281), it significantly improved the accuracy (p = 0.045, OR = 1.499) of a subset of nine pathologists who fell within well-defined experience levels (GI subspecialists, non-GI subspecialists, and trainees). In the assisted state, model accuracy significantly impacted the diagnostic decisions of all 11 pathologists. As expected, when the model’s prediction was correct, assistance significantly improved accuracy (p = 0.000, OR = 4.289), whereas when the model’s prediction was incorrect, assistance significantly decreased accuracy (p = 0.000, OR = 0.253), with both effects holding across all pathologist experience levels and case difficulty levels. Our results highlight the challenges of translating AI models into the clinical setting, and emphasize the importance of taking into account potential unintended negative consequences of model assistance when designing and testing medical AI-assistance tools.
Highlights
The rapid rate of scientific discovery in pathology has prompted a trend toward subspecialization,[1,2,3] which has made it more difficult for general surgical pathologists to maintain expertise across the entire range of diverse specimen subtypes.[4,5,6] This trend has resulted in misalignment of expertise for subspecialists, when reviewing specimens outside of their area of focus.[7,8] Such situations are commonly encountered, for example, during afterhours intraoperative consultations, when pathologists are confronted with a diverse set of cases across a range of specimen subtypes
To develop the deep learning model for our web-based assistant, we used a total of 25,000 non-overlapping image patches of size 512 × 512 pixels (128 × 128 μm), extracted from tumor-containing regions from 70 (35 hepatocellular carcinoma (HCC) and 35 CC) digital whole-slide images (WSI) of hematoxylin and eosin (H&E)-stained slides from formalin-fixed, paraffin-embedded (FFPE) primary hepatic tumor resections
We developed a deep learning algorithm intended to assist pathologists with the task of distinguishing between HCC and CC on H&E-stained WSI of primary liver tumors
Summary
The rapid rate of scientific discovery in pathology has prompted a trend toward subspecialization,[1,2,3] which has made it more difficult for general surgical pathologists to maintain expertise across the entire range of diverse specimen subtypes.[4,5,6] This trend has resulted in misalignment of expertise for subspecialists, when reviewing specimens outside of their area of focus.[7,8] Such situations are commonly encountered, for example, during afterhours intraoperative consultations, when pathologists are confronted with a diverse set of cases across a range of specimen subtypes. Few studies have taken the step of evaluating the impact of model assistance on pathologist diagnostic performance.[19] most recent applications of AI assistance to pathology have focused on models which run on whole-slide images (WSI) in a completely automated fashion, independent of human input, prior to the post-analytical stage. This creates a barrier to access for the majority of pathology practices, in global health settings, which lack the expensive digital slide scanners necessary for generating the WSI on which these models run.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
