Use of a metalearner to predict emergency medical services demand in an urban setting

Abstract

ObjectiveTo develop and internally validate a metalearner algorithm to predict the hourly rate of emergency medical services (EMS) dispatches in an urban setting. MethodsWe performed an analysis of EMS data from New York City between years 2015-2019. Our outcome was hourly EMS dispatches, expressed as continuous data. Hours were split into derivation (75%) and validation (25%) datasets. Candidate variables included averages of prior rates, temporal and weather characteristics. We used a metalearner to evaluate and aggregate individual learners (generalized linear model, generalized additive model, random forest, multivariable adaptive regression splines, and extreme gradient boost). Four models were investigated: 1) temporal variables, 2) weather and temporal variables, and datasets in which weather data were lagged by 3) six and 4) twelve hours. In exploratory analyses, we constructed learners for high acuity and trauma encounters. Results7,364,275 EMS dispatches occurred during the 43,823-hour period. When using temporal variables, the mean absolute error (MAE) rate was 11.5 dispatches in the validation dataset. These were slightly improved following incorporation of weather variables (MAE 11.3). When using 6- and 12-hour lagged weather variables, learners demonstrated lower accuracy (MAE 11.8 in 6-hour lagged datasets; 12.2 in 12-hour lagged dataset). All models had a coefficient of determination (R2) ≥0.91. The extreme gradient boosting and random forest learners were assigned the highest coefficients. In an investigation of variable importance, hour of day and average EMS dispatches over the previous six hours were the most important variables in both the extreme gradient boosting and random forest learners. The algorithm performed well at predicting frequently occurring peaks, with greater challenges at both extremes. Learners created high-acuity and for trauma-related encounters demonstrated superior MAE, but with lower R2 in the validation cohort (MAE 6.9 and R2 0.84 for high acuity encounters; MAE 5.3 and R2 0.79 for trauma in learners using time and weather variables). ConclusionWe developed an ensemble machine learning algorithm to predict EMS dispatches in an urban setting. These models demonstrated high accuracy, with MAEs <12 per hour in all. These algorithms may carry benefit in the real-time prediction of EMS responses, allowing for improved resource utilization.