Abstract WP182: Machine Learning Models are More Accurate Than Regression-based Models for Predicting Functional Impairment Risk in Acute Ischemic Stroke

Abstract

Background and Purpose: The burden of stroke-related functional impairment remains high among stroke survivors. Clinical prediction models are commonly used to estimate patient functional impairment risk. However, these models have been principally developed based on regression models, which are sensitive to multicollinearity. This study investigates whether there is any advantage in using machine learning models to develop stroke-related functional impairment risk prediction tools. Methods: Using data from a multi-center hospital-based cohort study (n = 614). Modified Rankin Scale (mRS) score was used to assess 90-day functional impairment status. The accuracy of machine learning models was used to predict the risk of patient-specific risk of 90-day functional impairment. Area under the receiver operating characteristic curve (AUC) was used to assess the predictive accuracy of these models via internal cross-validation and external validation in the ESCAPE randomized controlled trial data. Results: Of the 614 patients included in the analyses, 348(56.7%) had some form of functional impairment (i.e., mRS > 1), 313 (50.9%) were males, while the median and interquartile range (IQR) of age and baseline NIHSS scores were 72 years (IQR = 63-80) and 12 (IQR = 6-19), respectively. Internal cross-validation shows that the AUC for regression models were 68.3% (95%CI = [63.9% - 76.5%]) and 70.1% (95%CI = [63.5% - 76.1%]) while the AUC for machine learning models ranged between 62.7% to 68.8%. But when these models were externally validated in the ESCAPE data, the AUC for regression models were 39.6% (95%CI = [36.1% - 47.5%]) and 35.8% (95%CI = [30.4% - 41.5%]) while the AUC for machine learning models ranged between 61.6% (95%CI = [58.2% - 67.3%]) and 66.7% (95%CI = [61.3% - 72.3%]). Conclusions: This study shows that while there were negligible differences between risk prediction models based on machine learning and regression-based models when internally validated, the former are more accurate than the latter in predicting stroke-related functional impairment in externally validated data. Future research will use Monte Carlo methods to develop recommendations for selecting machine learning models under a variety of data characteristics.