Comparing an Artificial Neural Network to Logistic Regression for Predicting ED Visit Risk Among Patients With Cancer: A Population-Based Cohort Study

Abstract

ContextPrior work using symptom burden to predict emergency department (ED) visits among patients with cancer has used traditional statistical methods such as logistic regression (LR). Machine learning approaches for prediction, such as artificial neural networks (ANNs), are gaining attention but are yet to be commonly applied in practice. ObjectivesWe will compare an artificial neural network with logistic regression for predicting ED visit risk among patients with cancer. MethodsThis was a population-based study of patients diagnosed with cancer between 2007 and 2015 in Ontario, Canada. After splitting the cohort into training and test sets, an ANN model and a LR model were developed on the training cohort to predict the risk of an ED visit within seven days after an assessment of symptom burden. The predictive performance of each risk model was assessed on the test cohort and compared with respect to area under the curve and calibration. ResultsThe training cohort consisted of 170,092 patients undergoing 1,015,125 symptom assessments, and the remaining 42,523 patients undergoing 252,169 symptom assessments were set aside as the test cohort. Both models performed similarly with respect to specificity (ANN 67.0%; LR 67.3%) and accuracy (ANN 67.1%; LR 67.2%), and only minor improvement was found with respect to sensitivity (ANN 68.9%; LR 67.1%), discrimination (ANN 74.3%; LR 73.7%), and calibration under the ANN model compared with the LR model. The most notable improvement in calibration was found among patients in the highest ED visit risk percentile. ConclusionAlthough both models were similar in predictive performance using our data, ANNs have an important role in prediction because of their flexible structure and data-driven distribution-free benefits and should thus be considered as a potential modeling approach when developing a prediction tool.