A deep learning system to predict origins for cancers of unknown primary using cytological images from hydrothorax and ascites.

Abstract

e13628 Background: Cancer of unknown primary (CUP) origin is still a mystery in the field of tumor diagnosis, which is difficult to be diagnosed by conventional imaging methods. The aim of this study is to develop a deep learning based method to assist pathologists in determining the origin of malignant hydrothorax and ascites using cytological images. Methods: By curating cytology images of hydrothorax and ascites from 42682 cases at three hospitals, we developed a deep-learning-based method -Tumor Origin diffeRentiation for Cytological Histology (TORCH)- for diagnosis of malignancy and prediction of tumor origin. We examined its performance on three individual test sets (n=12799). We compared TORCH with a panel of four pathologists on 496 cases that were randomly selected from test sets. Area under the receiver operating characteristic curve (AUROC) was used as the primary classification metric. Metrics including accuracy, sensitivity, specificity, precision and negative predictive value were used to measure the efficacy of TORCH versus pathologists. A score system was used to evaluate and compare the performance between our AI model and pathologist experts. Attention heatmap was used to interpret model prediction results. Results: TORCH model reached an overall micro-average one versus rest auroc of 0.957(95% CI: 0.955-0.959). On the three test sets, TORCH achieved an AUROC of 0.964 (95% CI 0.957-0.972), 0.947 (0.940-0.953), and 0.988 (0.985-0.993) in the diagnosis of cancer and 0.947 (0.942-0.952), 0.957 (0.953-0.960) and 0.972 (0.968-0.976) in localization of tumor origin, respectively. For primary tumor origin prediction, TORCH achieved an overall top-1 accuracy of 79.2% and top-3 accuracy of 98.5%. On three test sets, the top-1 accuracy was 75.7%, 79.4%, and 85.0% respectively. Compared with pathologists, TORCH demonstrated a better prediction efficacy (1.665 ± 0.677 vs. 1.265 ± 0.555; P <0.001). The mean diagnostic score of junior pathologists assisted with TORCH was also remarkably higher than the previous one (1.326 ± 0.728 vs. 1.101 ± 0.824; P <0.001). The comprehensive accuracy of attention heatmaps capturing main area of isolated tumor cells was 87.3% (95% CI: 81.9-92.8%). Conclusions: TORCH model was able to provide relatively accurate diagnosis of malignancy and prediction of tumor cell origin in hydrothorax and ascites. It can be used as an effective ancillary method to aid clinicians in allocating personalized therapy and improving patients’ prognosis.