External validation of the SORG 90-day and 1-year machine learning algorithms for survival in spinal metastatic disease

Abstract

BACKGROUND CONTEXTPreoperative survival estimation in spinal metastatic disease helps determine the appropriateness of invasive management. The SORG ML 90-day and 1-year machine learning algorithms for survival in spinal metastatic disease were previously developed in a single institutional sample but remain to be externally validated. PURPOSEThe purpose of this study was to externally validate these algorithms in an independent population from another institution. STUDY DESIGN/SETTINGRetrospective study at a large, tertiary care center. PATIENT SAMPLEPatients 18 years or older who underwent surgery between 2003 and 2016. OUTCOME MEASURESNinety-day and 1-year mortality. METHODSBaseline characteristics of the validation cohort were compared to the developmental cohort for the SORG ML algorithms. Discrimination (c-statistic and receiver operating curve), calibration (calibration slope, intercept, calibration plot, and observed proportions by predicted risk groups), overall performance (Brier score), and decision curve analysis were used to assess the performance of the SORG ML algorithms in the validation cohort. RESULTSOverall, 176 patients underwent surgery for spinal metastatic disease, of which 44 (22.7%) experienced 90-day mortality and 99 (56.2%) experienced 1-year mortality. The validation cohort differed significantly from the developmental cohort on primary tumor histology, metastatic tumor burden, previous systemic therapy, overall comorbidity burden, and preoperative laboratory characteristics. Despite these differences, the SORG ML algorithms generalized well to the validation cohort on discrimination (c-statistic 0.75–0.81 for 90-day mortality and 0.77–0.78 for 1-year mortality), calibration, Brier score, and decision curve analysis. CONCLUSION and RELEVANCEInitial results from external validation of the SORG ML 90-day and 1-year algorithms for survival prediction in spinal metastatic disease suggest potential utility of these digital decision aids in clinical practice. Further studies are needed to validate or refute these algorithms in large patient samples from prospective, international, multi-institutional trials.