Outcome Supervised Deep Learning Model on Pathological Whole Slide Images for Survival Prediction of Immunotherapy in Non-Small Cell Lung Cancer Patients: A Multicenter Study

Abstract

Although PD-(L)1 inhibitors were marked by durable efficacy in non-small cell lung cancer patients (NSCLC), about 60% of patients still suffer from recurrence and metastasis after PD-(L)1 inhibitors treatment. And there were no robust biomarkers of the response of PD-(L)1 inhibitors. Whole slide images (WSIs) of H&E-stained specimens have been found to characterize the tumor microenvironment, and might be the potential prognostic predictors of NSCLC patients. To accurately predict the response to PD-(L)1 inhibitors, we presented the deep learning model based on WSI of H&E-stained specimens of NSCLC patients. Two independent cohorts of NSCLC patients receiving PD-(L)1 inhibitors from two hospitals were enrolled for model training and testing respectively. The WSI images of H&E-stained histological specimens were obtained from these patients, and patched into 1024×1024 pixels. The labels of patched images were determined due to their progression free survival (PFS) with the interval of 4 months. The patch-level model was firstly trained based on Vit to identify the predictive patches in training cohort, and patch-level probability distribution was performed. Then we trained patient-level survival model-based Vit-RNN framework, and tested it in external validation cohort. A total of 291 WSI images of H&E-stained histological specimens from 198 NSCLC patients in primary cohort and 62 WSI images from 30 NSCLC patients in testing cohort were included for model training and external validation. All patients were divided into 4 groups due to their PFS after PD-(L)1 inhibitors. There were 246,318 patches from 291 images in primary cohort after image pre-processing, and all images were randomly divided into train cohort and validation cohort with the proportion of 7:3. The patch-level Vit model with the highest accuracy was saved and the predictive patches were selected after 50 epochs training. All patches were ranked by the probability of correct prediction, and the first 50 top-ranked patches from each WSI image are sequentially passed to the patient-level Vit-RNN model. The Vit-RNN survival achieved an accuracy of 88.6% in the validation cohort, and an accuracy of 81% in the testing cohort. The multivariate cox analysis also indicated the Vit-RNN survival model remained a statistically independent predictor of survival from PD-(L)1 inhibitors (P = 0.0085). The outcome supervised Vit-RNN survival model based on pathological WSIs could be used to predict the efficacy the PD-(L)1 inhibitors in NSCLC patients, laying the foundation for the deployment of computational pathomics in clinical practice of immunotherapy.