Using a Novel Machine Learning Technique to Predict Radiation Oncology Outcomes

Abstract

Machine learning (ML) uses computer algorithms to recognize patterns in complicated databases. The data-driven nature of radiation oncology research and preponderance of data collected for patients make radiation oncology outcomes research particularly well-suited to ML strategies. Despite this, work has tended to rely on traditional statistical models to make predictions. Studies that have used ML to make predictions have been restricted to only a few of the great diversity of ML algorithms available and offer no evidence that their chosen algorithm produces the most accurate predictions. In this proof of concept study, we use a novel ML technique to predict distant brain failure (DBF) or death after stereotactic radiosurgery (SRS) for brain metastases. We train 28 unique ML algorithms to predict the outcomes of interest and then directly compare the predictive abilities of each algorithm, selecting the top performing algorithms for inclusion in an ensemble model. Finally, we compare the discriminative ability of our ML ensemble to a previously described multivariable Cox proportional hazards (PH) regression model. We completed a multi-center, retrospective study of 2590 patients undergoing SRS for brain metastases from 2000 to 2013. 28 ML algorithms were trained on 13 patient characteristics to predict DBF or death at 6 months. Algorithms were ranked by discriminative ability and the top three performers combined to form an ensemble. The final ensemble was validated to demonstrate generalizability. Permutation and partial dependence analysis were performed to understand the impact of individual variables on the final ensemble. An ensemble comprised of a Regularized Logistic Regression, Vowpal Wabbit Classifier, and Naïve Bayes Combiner Classifier best predicted DBF or death at 6 months following SRS for brain metastases. The ensemble had discriminative ability comparable to the previously described PH model (Harrell c-index of .64 vs .63 for the PH model). Model analysis demonstrated that increasing numbers of metastases at time of SRS, melanoma histology, male gender, increasing age, and progressive status of intracranial disease are the strongest risk factors for DBF or death at 6 months following SRS, replicating the findings of the PH model. We use a novel ML technique to predict DBF or death at 6 months following SRS with discriminative ability comparable to an existing PH model. Using direct comparison of many ML algorithms and an existing statistical model, this technique allows researchers to be confident that they are building the most predictive models for their data. This technique also allows for the integration of new data into the model, facilitating dynamic predictive modeling that can adapt to changes in treatment modalities or patient populations. Finally, in replicating previously reported findings, this study reinforces the applicability of ML approaches to radiation oncology outcomes research.