288 Predicting Potential Bouncebacks to the Emergency Department: A Machine Learning Approach

Abstract

Listen

Translate article

Review of patients who return to the emergency department (ED) within 72 hours (ie, bounceback) is a common quality assurance process. Current practice of studying bouncebacks, which relies on manual chart extraction and review, is labor-intensive and limited. In this study, we leverage machine learning methods to automatically extract features from electronic health record (EHR) data, learn patterns among patients who “bounceback,” and develop an EHR-driven risk prediction model. This is a retrospective analysis of EHR data and provider notes of all adult patients who presented to a single-center, urban emergency department (ED) over 4.25 years (1/1/2015—4/1/2019). Two different deep learning approaches have been employed to analyze a combination of structured variables (demographics and vital signs at the time of ED admission) as well as unstructured EHR free text (ED provider notes) to predict patients’ risk of bounceback. Both approaches involved training word embeddings using the EHR free text notes. In the first approach each EHR note was converted to a sequence of word embedding vectors and the sequence was used to train a stacked Long Short-Term Memory Recurrent Neural Network (LSTM-RNN), whereas in the second approach each EHR note was abstracted to a single embedding vector using a Doc2Vec algorithm and it was passed through a fully connected (dense) deep network architecture. The resultant vector in each approach was then concatenated to a vector of structured predictors and passed through multiple fully connected network layers to predict the outcome. The accuracy of predicting the bounceback target variable is characterized by sensitivity, specificity, and the area under the receiver operating characteristic curve (AUROC). Of the adult patients (n = 330,407) who presented to the ED during this time period, 4.7% (n = 15,529) subsequently returned within 72 hours. The LSTM model produced a sensitivity and specificity of 66.3% and 59%, respectively (AUROC = 0.686). The sensitivity and specificity improved with the Doc2Vec approach, to 67.9% and 66.8%, respectively. The resulting AUROC was found to be 0.755, indicating the higher distinctive power of the second approach. In this retrospective analysis of EHR data, deep learning models demonstrated promising results as a proof of concept for uncovering predictive local motifs in the ED provider note texts among patients who return within 72 hours. Once fully developed and deployed, these predictive models could aid clinical quality review of bouncebacks in the ED and serve as adjuncts to clinical decision support.