Explainable machine learning to inform real-world evidence: A case study on small cell lung cancer survival analysis.

Abstract

113 Background: In recent years, machine learning algorithms for survival analysis have been developed as an alternative to traditional Cox regression. Despite the good performances, these are viewed as black box models that lack interpretability, which may limit their clinical applications. There has been a growing interest in explaining and interpreting machine learning models, which has led to the emergence of the field of explainable artificial intelligence (XAI). However, few studies have focused on how these methods could be of benefit to survival analysis of real-world healthcare data. Methods: In this study, we explored the use of current XAI techniques to analyze the effects of platinum doublet chemotherapy on small cell lung cancer patients in developed machine learning models.The analyzed data included real-world patients treated at Karolinska University Hospital (n=570) and three phase III randomized clinical trials shared by the Project Data Sphere initiative (n=987). The real-world data covariates included were age, sex, TNM staging, ECOG performance status, lab values, brain metastasis, and concomitant radiotherapy. The aggregated dataset including the clinical trials contained the following variables: age, sex, performance status, brain metastasis, and protocol violations. Eight machine learning models were trained and compared with Cox regression. The performance of the models was evaluated using C-index, and the time variation of Brier Score and C/D AUC.Temporal feature importance and partial dependence were used to explore the overall covariate impact on overall survival (Global XAI). Ceteris-paribus and SurvShap(t) were used to investigate the covariate impact for the single patient prediction (local XAI). The models were firstly trained only on real-world data before aggregation with the clinical trials. Results: Ensemble machine learning provided the best performances. XAI techniques showed the potential to increase explainability of survival predictions in function of time. Global XAI showed the time range of the model reliability, trend inversions regarding treatment decisions and covariate importance along the time. Local XAI allowed to test the impact of covariates between long survival patients and the comparison between real-world and clinical trials. Conclusions: Our results demonstrate the potential of XAI techniques applied to survival machine learning and real-world data, thus providing insights into the mechanisms driving model predictions and demonstrate the utility of this approach in clinical research.