Predicting daily emergency department visits using machine learning could increase accuracy

Abstract

ObjectiveAdministrators and clinicians alike have attempted to predict emergency department visits for many years. The ability to predict or “forecast” ED visit volume can allow for more efficient resource allocation, including up-staffing or down-staffing, changing OR schedules, and predicting the need for significant resources. The goal of this study is to examine combinations of variables via machine learning to increase prediction accuracy and determine the factors that are most predictive of overall ED visits. As compared to a simple univariate time series model, we hypothesize that machine learning models will predict St. Joseph Mercy Ann Arbor's patient visit load for the emergency department (ED) with higher accuracy than a simple univariate time series model. MethodsUnivariate time series models for daily ED visits, including ARIMA, Exponential Smoothing (ETS), and Facebook Inc.'s prophet algorithm were estimated as a baseline comparison. Machine learning models, including random forests and gradient boosted machines (GBM), were trained using data from 2017 to 2018. After final models were created, they were applied to the 2019 data to determine how well these models predicted actual ED patient volumes in data not utilized during the model fitting process. The accuracy of the machine learning and time series models were assessed based on out-of-sample predictive accuracy, compared using root mean squared error (RMSE). ResultsUsing root mean squared error (RMSE) to assess out-of-sample predictive accuracy of the models, the results showed that the random forest model was the most accurate at predicting daily ED visits in the 2019 test set, followed by the GBM model. These performed only slightly better than the simple exponential smoothing model predictions. The ARIMA model performed poorly in comparison. The day of the week (likely capturing differences between weekdays and weekends) was found to be the most important predictor of patient volumes. Weather-related features such as maximum temperature and SFC pressure appeared to capture some of the seasonality trends related to changes in patient volumes. ConclusionsMachine learning models perform better at predicting daily patient volumes as compared to simple univariate time series models, though not by a substantial amount. Further research can help confirm these limited initial results. Gathering more training data and additional feature engineering could also be beneficial to training the models and potentially improving predictive accuracy.