Emergency Severity Index Score Research Articles

Use of a Large Language Model to Assess Clinical Acuity of Adults in the Emergency Department

The introduction of large language models (LLMs), such as Generative Pre-trained Transformer 4 (GPT-4; OpenAI), has generated significant interest in health care, yet studies evaluating their performance in a clinical setting are lacking. Determination of clinical acuity, a measure of a patient's illness severity and level of required medical attention, is one of the foundational elements of medical reasoning in emergency medicine. To determine whether an LLM can accurately assess clinical acuity in the emergency department (ED). This cross-sectional study identified all adult ED visits from January 1, 2012, to January 17, 2023, at the University of California, San Francisco, with a documented Emergency Severity Index (ESI) acuity level (immediate, emergent, urgent, less urgent, or nonurgent) and with a corresponding ED physician note. A sample of 10 000 pairs of ED visits with nonequivalent ESI scores, balanced for each of the 10 possible pairs of 5 ESI scores, was selected at random. The potential of the LLM to classify acuity levels of patients in the ED based on the ESI across 10 000 patient pairs. Using deidentified clinical text, the LLM was queried to identify the patient with a higher-acuity presentation within each pair based on the patients' clinical history. An earlier LLM was queried to allow comparison with this model. Accuracy score was calculated to evaluate the performance of both LLMs across the 10 000-pair sample. A 500-pair subsample was manually classified by a physician reviewer to compare performance between the LLMs and human classification. From a total of 251 401 adult ED visits, a balanced sample of 10 000 patient pairs was created wherein each pair comprised patients with disparate ESI acuity scores. Across this sample, the LLM correctly inferred the patient with higher acuity for 8940 of 10 000 pairs (accuracy, 0.89 [95% CI, 0.89-0.90]). Performance of the comparator LLM (accuracy, 0.84 [95% CI, 0.83-0.84]) was below that of its successor. Among the 500-pair subsample that was also manually classified, LLM performance (accuracy, 0.88 [95% CI, 0.86-0.91]) was comparable with that of the physician reviewer (accuracy, 0.86 [95% CI, 0.83-0.89]). In this cross-sectional study of 10 000 pairs of ED visits, the LLM accurately identified the patient with higher acuity when given pairs of presenting histories extracted from patients' first ED documentation. These findings suggest that the integration of an LLM into ED workflows could enhance triage processes while maintaining triage quality and warrants further investigation.

Racial differences in triage assessment at rural vs urban Maine emergency departments.

Data continue to accumulate demonstrating that those belonging to racialized groups face implicit bias in the emergency care delivery system across many indices, including triage assessment. The Emergency Severity Index (ESI) was developed and widely implemented across the US to improve the objectivity of triage assessment and prioritization of care delivery; however, research continues to support the presence of subjective bias in triage assessment. We sought to assess the relationship between perceived race and/or need for translator and assigned ESI score and whether this was impacted by hospital geography. We performed retrospective EMR-based review of patients presenting to urban and rural emergency departments of a health system in Maine with one of the top ten most common chief complaints (CC) across a 5-year period, excluding psychiatric CCs. We used multivariable regression to analyze the relationships between perceived race, need for translator, and gender with ESI score, wait time, and hallway bed assignments. We found that patients perceived as non-white were more likely to receive lower acuity ESI scores and have longer wait times as compared to patients perceived as white. Patients perceived as female were more likely to receive lower acuity scores and wait longer to be seen than patients perceived as male. The need for an interpreter was associated with increased wait times but not significantly associated with ESI score. After stratification by hospital geography, evidence of subjective bias was limited to urban emergency departments and was not evident in rural emergency departments. Further investigation of subjective bias in emergency departments in Maine, particularly in urban settings, is warranted.

Comparison of Manchester, qSOFA, emergency severity index, and national early warning scores for prognostic estimation and effective triage system in geriatric patients.

Triage refers to classifying and prioritizing patients based on the severity of their injuries or illnesses in the health care setting. The increasing number of elderly patients seeking care in emergency departments (EDs) highlights the need for special attention to the unique needs of this patient population. We aimed to compare the qSOFA, Emergency Severity Index (ESI), National Early Warning Score (NEWS), and Manchester Triage System (MTS) scores to assist ED physicians in assessing the severity of elderly patients' clinical conditions and triaging them appropriately. This cross-sectional study included 1066 patients aged 65 and over who presented to our ED as outpatients or by ambulance between September 1, 2022, and August 30, 2023. Scoring systems at the time of admission to the ED were recorded separately for outpatients and arriving by ambulance. According to the qSOFA, patients with a score of 0 were 0.976 times less likely to arrive by ambulance compared to those scoring 1 and above (OR = 0.976, p = .934). According to the NEWS, patients in the moderate-risk category were 0.447 times less likely to arrive by ambulance (OR = 0.447, p = .054). According to the ESI score, patients requiring high resource use with normal vital signs were 146.758 times more likely to arrive by ambulance (OR = 146.758, p = .001). Significant differences in patients' methods of presentation to the ED were observed based on the MTS, qSOFA, NEWS, and ESI scores.

Additional Value From Free-Text Diagnoses in Electronic Health Records: Hybrid Dictionary and Machine Learning Classification Study.

Physicians are hesitant to forgo the opportunity of entering unstructured clinical notes for structured data entry in electronic health records. Does free text increase informational value in comparison with structured data? This study aims to compare information from unstructured text-based chief complaints harvested and processed by a natural language processing (NLP) algorithm with clinician-entered structured diagnoses in terms of their potential utility for automated improvement of patient workflows. Electronic health records of 293,298 patient visits at the emergency department of a Swiss university hospital from January 2014 to October 2021 were analyzed. Using emergency department overcrowding as a case in point, we compared supervised NLP-based keyword dictionaries of symptom clusters from unstructured clinical notes and clinician-entered chief complaints from a structured drop-down menu with the following 2 outcomes: hospitalization and high Emergency Severity Index (ESI) score. Of 12 symptom clusters, the NLP cluster was substantial in predicting hospitalization in 11 (92%) clusters; 8 (67%) clusters remained significant even after controlling for the cluster of clinician-determined chief complaints in the model. All 12 NLP symptom clusters were significant in predicting a low ESI score, of which 9 (75%) remained significant when controlling for clinician-determined chief complaints. The correlation between NLP clusters and chief complaints was low (r=-0.04 to 0.6), indicating complementarity of information. The NLP-derived features and clinicians' knowledge were complementary in explaining patient outcome heterogeneity. They can provide an efficient approach to patient flow management, for example, in an emergency medicine setting. We further demonstrated the feasibility of creating extensive and precise keyword dictionaries with NLP by medical experts without requiring programming knowledge. Using the dictionary, we could classify short and unstructured clinical texts into diagnostic categories defined by the clinician.

Delays in treatment and disposition attributable to undertriage of pediatric emergency medicine patients

BackgroundTriage, the initial assessment and sorting of patients in the Emergency Department (ED), determines priority of evaluation and treatment. Little is known about the impact of undertriage, the underestimation of disease severity at triage, on clinical care in pediatric ED patients. We evaluate the impact of undertriage on time to disposition and treatment decisions in pediatric ED patients. MethodsThis was a case control study of ED visits for patients <22 years of age, with an assigned Emergency Severity Index (ESI) score of 4 or 5, and associated hospital admission, nebulized treatment, supplemental oxygen, and/or intravenous (IV) line placement, between January 1, 2018, to June 30, 2022. Controls were sampled from a pool of patient visits with an ESI score of 3, matched by intervention, disposition, and date and hour of arrival.Primary outcome measures were time to order of intervention (nebulized treatment, oxygen administration, or IV placement) and time to disposition decision. A secondary outcome measure was return visits requiring admission or emergency intervention within 14 days of the index visit. Continuous variables (time to orders) were analyzed using Wilcoxon rank sum test and dichotomous outcomes (return visits) were compared using odds ratios with 95% confidence intervals. Analysis was performed with Python v3.10. ResultsThe final analysis included 7245 undertriaged patients. Undertriaged patients had longer times to orders for nebulized treatments, (p < 0.001) IV placement, (p < 0.001) and admission (p < 0.001) when compared to controls. There were no significant differences in time to supplemental oxygen delivery and time to discharge compared to controls. Undertriaged patients were more likely to experience a return visit requiring admission or emergency intervention (OR 3.74, 95% CI 3.32,4.22). ConclusionsUndertriage in the pediatric ED is associated with delays in care and disposition decisions and increases likelihood of return visits.

Enhancing Triage Efficiency and Accuracy in Emergency Rooms for Patients with Metastatic Prostate Cancer: A Retrospective Analysis of Artificial Intelligence-Assisted Triage Using ChatGPT 4.0.

Accurate and efficient triage is crucial for prioritizing care and managing resources in emergency rooms. This study investigates the effectiveness of ChatGPT, an advanced artificial intelligence system, in assisting health providers with decision-making for patients presenting with metastatic prostate cancer, focusing on the potential to improve both patient outcomes and resource allocation. Clinical data from patients with metastatic prostate cancer who presented to the emergency room between 1 May 2022 and 30 April 2023 were retrospectively collected. The primary outcome was the sensitivity and specificity of ChatGPT in determining whether a patient required admission or discharge. The secondary outcomes included the agreement between ChatGPT and emergency medicine physicians, the comprehensiveness of diagnoses, the accuracy of treatment plans proposed by both parties, and the length of medical decision making. Of the 147 patients screened, 56 met the inclusion criteria. ChatGPT had a sensitivity of 95.7% in determining admission and a specificity of 18.2% in discharging patients. In 87.5% of cases, ChatGPT made the same primary diagnoses as physicians, with more accurate terminology use (42.9% vs. 21.4%, p = 0.02) and more comprehensive diagnostic lists (median number of diagnoses: 3 vs. 2, p < 0.001). Emergency Severity Index scores calculated by ChatGPT were not associated with admission (p = 0.12), hospital stay length (p = 0.91) or ICU admission (p = 0.54). Despite shorter mean word count (169 ± 66 vs. 272 ± 105, p < 0.001), ChatGPT was more likely to give additional treatment recommendations than physicians (94.3% vs. 73.5%, p < 0.001). Our hypothesis-generating data demonstrated that ChatGPT is associated with a high sensitivity in determining the admission of patients with metastatic prostate cancer in the emergency room. It also provides accurate and comprehensive diagnoses. These findings suggest that ChatGPT has the potential to assist health providers in improving patient triage in emergency settings, and may enhance both efficiency and quality of care provided by the physicians.

Leaving Without Being Seen From the Pediatric Emergency Department: A New Baseline

BackgroundLeft without being seen (LWBS) rates are an important quality metric for pediatric emergency departments (EDs), with high-acuity LWBS children representing a patient safety risk. Since July 2021, our ED experienced a surge in LWBS after the most stringent COVID-19 quarantine restrictions ended. ObjectiveWe assessed changes in LWBS rates and examined associations of system factors and patient characteristics with LWBS. MethodsWe performed a retrospective study in a large, urban pediatric ED for all arriving patients, comparing the following three time-periods: before COVID-19 (PRE, January 2018–February 2020), during early COVID-19 (COVID, March 2020–June 2021), and after the emergence of COVID-19 variants and re-emergence of seasonal viruses (POST, July 2021–December 2021). We compared descriptive statistics of daily LWBS rates, patient demographic characteristics, and system characteristics. Negative binomial (system factors) and logistic regression (patient characteristics) models were developed to evaluate the associations between system factors and LWBS, and patient characteristics and LWBS, respectively. ResultsMean daily LWBS rates changed from 1.8% PRE to 1.4% COVID to 10.7% during POST. Rates increased across every patient demographic and triage level during POST, despite a decrease in daily ED volume compared with PRE. LWBS rates were significantly associated with patients with an Emergency Severity Index score of 2, mean ED census, and staff productivity within multiple periods. Patient characteristics associated with LWBS included lower assigned triage levels and arrival between 8 pm and 4 am. ConclusionsLWBS rates have shown a large and sustained increase since July 2021, even for high-acuity patients. We identified system factors that may provide opportunities to reduce LWBS. Further work should develop strategies to prevent LWBS in at-risk patients.

Palliative care patient emergency department visits at tertiary university-based emergency department in Ireland

IntroductionPalliative care patients often present to the emergency department (ED) for various reasons e.g., acute illness, pain, altered mental status, and complications of therapy. Many visits involve less severe etiologies e.g., dyspnea, constipation, fear as patients approach the end of life, which may be more effectively and efficiently managed outside of the ED. The objective of this study is to identify and assess the frequency of presenting complaints, primary diagnosis, triage acuity, need for admission, in an Irish setting. MethodsA single-center retrospective, observational study of palliative care patients presenting to a tertiary-care university hospital emergency department in Dublin, Ireland. Study subjects were identified using the palliative care database and cross-referencing with the ED electronic patient record system database. The primary objective to identify potential areas to minimize ED visits and improve patient care and quality of life by elucidating reasons for visits. Outcome measures include presenting complaint, primary diagnosis, triage severity score, admission, discharge, death in hospital. Statistical analysis presented as descriptive statistics. ResultsFour-hundred-ninety-nine ED visits, 245 (49%) were male, and 254 (51%) were female with a mean age of 69.3 years-of-age. Most patients, 285 (57.1%) self-referred to the emergency department, with general practitioners and skilled nursing facility referrals 72 (14.4%) and 39 (7.8%), respectively. Primary diagnoses were various cancers, chronic obstructive pulmonary disease, congestive heart failure, and dementia. Major reasons for visits were dyspnea, pain, falls, trauma, fever, and altered mental status. Two-hundred-eighty-nine patients (58%) had an emergency severity index (ESI) score of 1 or 2 demonstrating a higher level of acuity. Three-hundred-fifty-eight (71.7%) were admitted, 141 (28.3%) discharged to home, 64 (12.8%) admitted patients died during their hospital admission. ConclusionsPalliative care patients utilize ED services not uncommonly. Though many of these patients presented with higher acuity triage scores, 42% had lower ESI scores and may be effectively managed outside of the ED. These data suggest developing mechanisms for these patients to be urgently evaluated in their homes or facilities obviating the need for an ED evaluation.

The Association of Emergency Severity Index Score and Patient and Family Experience in a Pediatric Emergency Department.

The study aim was to determine the relationship between a patient's Emergency Severity Index (ESI) score and their or their family's response to the key performance indicator (KPI) question on the post-visit patient and family experience (PFE) survey. Retrospective review of patients presenting to the Pediatric Emergency Department between July 1, 2021, and June 30, 2022, who completed the KPI question on an associated post-visit survey. We performed univariate analyses on all candidate variables; multivariable linear regression identified independent predictors of KPI on the PFE survey. A total of 8136 patients were included in the study. Although ESI score was significantly associated with PFE in univariate analysis, this association was lost in the multivariable model. Independent associations were appreciated with race/ethnicity, time to provider, length of stay, and procedure performance during the visit. Although ESI is not independently associated with PFE in this study, its interaction with factors such as time to provider, length of stay, and procedure performance may be important for emergency department providers creating interventions to impact experience during low acuity visits.

Machine Learning Methods for Predicting Patient-Level Emergency Department Workload

BackgroundWork Relative Value Units (wRVUs) are a component of many compensation models, and a proxy for the effort required to care for a patient. Accurate prediction of wRVUs generated per patient at triage could facilitate real-time load balancing between physicians and provide many practical operational and clinical benefits. ObjectiveWe examined whether deep-learning approaches could predict the wRVUs generated by a patient's visit using data commonly available at triage. MethodsAdult patients presenting to an urban, academic emergency department from July 1, 2016–March 1, 2020 were included. Deidentified triage information included structured data (age, sex, vital signs, Emergency Severity Index score, language, race, standardized chief complaint) and unstructured data (free-text chief complaint) with wRVUs as outcome. Five models were examined: average wRVUs per chief complaint, linear regression, neural network and gradient-boosted tree on structured data, and neural network on unstructured textual data. Models were evaluated using mean absolute error. ResultsWe analyzed 204,064 visits between July 1, 2016 and March 1, 2020. The median wRVUs were 3.80 (interquartile range 2.56–4.21), with significant effects of age, gender, and race. Models demonstrated lower error as complexity increased. Predictions using averages from chief complaints alone demonstrated a mean error of 2.17 predicted wRVUs per visit (95% confidence interval [CI] 2.07–2.27), the linear regression model: 1.00 wRVUs (95% CI 0.97–1.04), gradient-boosted tree: 0.85 wRVUs (95% CI 0.84–0.86), neural network with structured data: 0.86 wRVUs (95% CI 0.85–0.87), and neural network with unstructured data: 0.78 wRVUs (95% CI 0.76–0.80). ConclusionsChief complaints are a poor predictor of the effort needed to evaluate a patient; however, deep-learning techniques show promise. These algorithms have the potential to provide many practical applications, including balancing workloads and compensation between emergency physicians, quantify crowding and mobilizing resources, and reducing bias in the triage process.

312 Racial Differences in Adult Emergency Department Triage Assignment Across An Urban-Rural Health System

Despite attempts to limit triage subjectivity through use of the emergency severity index (ESI) algorithm, it is possible for implicit or explicit bias to impact acuity assignment. Across our health system, ESI is assigned by trained emergency registered nurses (RNs). Documented demographic characteristics, such as race and ethnicity, are often based upon perception rather than direct patient query. We sought to assess the impact of perceived race on emergency department (ED) ESI score assigned at triage across our 9- ED health system.

Association of Race and Ethnicity With Triage Emergency Severity Index Scores and Total Visit Work Relative Value Units for Emergency Department Patients

Association of Race and Ethnicity With Triage Emergency Severity Index Scores and Total Visit Work Relative Value Units for Emergency Department Patients

Do Non-COVID-19 Patients' Behaviour Towards Emergency Changed During the COVID-19 Outbreak? A Severity-Based Approach.

ObjectiveDuring the COVID-19 pandemic, the number of patients presenting to the emergency department (ED) declined. The main goal of this study was to compare and describe the non-COVID-19 patient’s disease severity presentation during the pandemic with its pre-pandemic severity.MethodsWe conducted a retrospective observational study. We selected two samples of visits: one during the first COVID-19 wave of 2020 (pandemic period, PP) and the other during the same months of 2019 (control period, CP). The primary endpoints were the comparison of severity and distribution of the Emergency Severity Index (ESI). Secondary endpoints were comparisons of specific patient characteristics (age, sex, length of the symptoms before the visits, spontaneous visits or not, return home or not).ResultsThe mean ESI of the visits during the PP (3.19) was statistically significantly lower (P = 0.001) than it was in the CP (3.43). These changes were more pronounced during the months of March (3.03 versus 3.33, P = 0.037) and April (2.96 versus 3.48, P < 0.001). The change in ESI was mainly due to an increase in the proportion of visits by patients with an ESI score of 3 (42% versus 28%, P < 0.001). There were no differences in the characteristics of patients except a decline in patients whose symptoms had a duration of more than 30 days (2% during PP versus 4% during CP, P = 0.03).ConclusionThe COVID-19 pandemic caused a change in the pattern of non-COVID-19 visits, with proportionally more severe presentations based on the ESI. To our knowledge, this is the first description of changes in behaviour in ED visits by specifically non-COVID-19 patients.

Racial differences in treatment among patients with acute headache evaluated in the emergency department and discharged home

BackgroundPrior research has demonstrated the widespread presence of racial disparities in emergency department (ED) care and analgesia. We hypothesized that racial disparities continue to exist in ED analgesic prescribing patterns, time to analgesia, and time to provider in the treatment of headache. MethodsWe performed a retrospective cohort study of patients presenting to a large tertiary academic ED with chief complaint of headache. A structured medical record review was conducted to abstract relevant variables of interest. Patient race was categorized as white or Black, Indigenous, or person of color (BIPOC). Descriptive statistics were used to characterize the cohort and stratified analyses were conducted based on patient race and our key outcome measures of analgesic prescribing patterns, time to analgesia, and time to provider in the treatment of headache. ResultsWhite patients were more likely to be assigned an Emergency Severity Index score 2 or 3 and their BIPOC counterparts were more likely to be assigned an ESI score 3 or 4 (p = 0.02). There was no significant difference by race in time to analgesia (p = 0.318), time to provider (p = 0.358), or time to first medication treatment (p = 0.357). However, there were clear differences in prescribing patterns. BIPOC patients were significantly more likely to be treated with acetaminophen (p = 0.042) or ibuprofen (p = 0.015) despite reporting higher pain levels during triage (p < 0.001). White patients were significantly more likely to receive a head CT scan (p < 0.001) or neurology consult (p = 0.003) than their BIPOC counterparts. ConclusionRacial disparities persist in assessment and type of analgesia for patients being treated for headache in a large academic emergency department.

Patient disposition using the Emergency Severity Index: a retrospective observational study at an interdisciplinary emergency department.

ObjectivesEarly patient disposition is crucial to prevent crowding in emergency departments (EDs). Our study aimed to characterise the need of in-house resources for patients treated in the ED according to the Emergency Severity Index (ESI) and the presenting complaint at the timepoint of triage.DesignA retrospective single-centre study was conducted.SettingData of all patients who presented to the interdisciplinary ED of a tertiary care hospital in Munich, Germany, from 2014 to 2017 were analysed.Participantsn=113 694 patients were included.MeasuresESI Score, medical speciality according to the chief complaint, mode of arrival, admission rates and discharge destination from the ED were evaluated.ResultsPatient disposition varied according to ESI scores in combination with the chief complaint. Patients with low ESI scores were more likely to be admitted after treatment in the ED than patients with high ESI scores. Highly prioritised patients (ESI 1) mainly required admission to an intensive care unit (ICU, 27%), intermediate care unit (IMC, 37%) or immediate intervention (11%). In this critical patient group, 30% of patients with neurological or medical symptoms required immediate intensive care, whereas only 17% of patients with surgical problems were admitted to an ICU. A significant number of patients (particularly with neurological or medical problems) required hospital (and in some cases even ICU or IMC) admission despite high ESI scores.ConclusionsOverall, ESI seems to be a useful tool to anticipate the need for specialised in-hospital resources on arrival. Patients with symptoms pointing at neurological or medical problems need particular attention as ESI may fail to sufficiently predict the care facility level for this patient group.

A Clinical Nurse Specialist-Driven Project to Improve Emergency Department Triage Accuracy.

The purpose of this quality improvement project was to examine the use of video-simulated scenarios and mobile technology to improve accuracy of emergency department (ED) nurses' triage using the Emergency Severity Index (ESI). A quality improvement project with a pre/post educational intervention design consisting of a convenience sample (n = 33) of ED registered nurses (RNs) at a large tertiary hospital in the Midwest was used. A retrospective chart review (n = 495) was completed to obtain ESI accuracy for each triage RN. For 12 consecutive weeks, the ED RNs received different video simulations via mobile technology to determine the ESI level. After receiving their scores, the project team provided the RNs the correct ESI score with rationale via mobile technology. Post intervention, a retrospective chart review was conducted to evaluate RNs' ESI accuracy. Results of this ED triage educational intervention to improve the accuracy of ED nurses' ESI scores were not significant; however, this novel approach may be considered in addition to other teaching strategies to improve outcomes. Triage nurses' ESI scoring accuracy can be inconsistent. Therefore, to ensure patients are receiving prompt and appropriate care for their acuity level, it is important to continuously provide education on ESI scoring.

CT Trends After Minor Head Injury Guideline Implementation

Source: Corwin DJ, Durbin DR, Hayes KL, et al. Trends in emergent head computed tomography utilization for minor head trauma after implementation of a clinical pathway. Pediatr Emerg Care. 2021;37(9);437-441; doi:10.1097/PEC.0000000000001728Investigators from the Children’s Hospital of Philadelphia (CHOP), Philadelphia, PA, conducted a retrospective study to assess changes in neuroimaging among children seen in the ED for mild traumatic brain injury (TBI) following implementation of guidelines designed to reduce unnecessary CT scans. The guidelines, implemented in 2011, were adapted from guidelines published by the Pediatric Emergency Care Applied Research Network (PECARN) that included specific clinical criteria for identifying children at low risk.1 Study participants were patients ≤21 years old seen in the ED at CHOP between 2012 and 2016 with TBI (identified using ICD-9 and ICD-10 codes) with a Glasgow Coma Scale score of 13–15 who were discharged home following evaluation. Data abstracted on these children included demographics, time of day, provider type (general pediatrician, pediatrician emergency physician, or nurse practitioner), whether a CT was obtained, and Emergency Severity Index (ESI) score. ESI scores range from 1–5, with 1 being the most acute. The medical records of a random sample of 10% of patients on whom a CT was obtained were reviewed to assess the reason for obtaining the scan and to classify the child’s risk for intracranial injury based on PECARN criteria. The primary outcome was change in rate of CT utilization during the study period, assessed using Wilcoxon rank sum test for trend. Secondary outcomes included rate of ordering CT scans by provider type, using logistic regression and adjusting for ESI scores.Data were analyzed on 21,129 patients with a mean age of 6.6 years; 62% were male. Overall, CT scans were obtained on 6.3% (95% CI, 5.9%, 6.6%) of these patients. During the study period the rate of CT scan utilization fell from 8.3% in 2012 to 4.9% in 2016 (average decrease 0.9% per year; P <0.001 for trend). After adjusting for ESI score, general pediatricians were more likely to obtain a CT scan than pediatric emergency physicians (OR, 1.37; 95% CI, 1.02, 1.83); there was no difference in likelihood of ordering a CT between nurse practitioners and pediatric emergency physicians (OR, 0.63; 95% CI, 0.33, 1.17). In the 10% random sample of patients on whom CT scans were obtained, 30% met PECARN criteria for being low risk. The most common indications for ordering a CT in children <2 years old were concern for non-accidental trauma and vomiting, while for older patients the most common reasons were parental preference and presence of a scalp hematoma.The authors conclude the rate of CT scan ordering in children with mild TBI decreased after implementation of guidelines designed to reduce unnecessary neuroimaging.Dr Bechtel has disclosed no financial relationship relevant to this commentary. This commentary does not contain a discussion of an unapproved/investigative use of a commercial product/device.Clinical pathways (CPWs) are tools to promote evidence-based healthcare delivery and strive to standardize care for a specific clinical problem in a specific population.2 Thus, compliance with a CPW by a diverse swath of healthcare providers is ultimately the best indicator of its relevance and utility. The primary objective of the current well-done study was to assess the pattern of change in CT utilization in discharged children with mild TBI following the implementation of a CPW in a children’s ED.The authors found that for 5 years, the rate of obtaining CT in children discharged with minor TBI declined by 0.9% per year, a statistically significant decrease, without an accompanying increase of LOS in the ED. However, despite these guidelines, 30% of surveyed Electronic Health Records (EHRs) of discharged children were characterized as low risk by the guidelines and still had a CT obtained. When documented in the EHR, the most common indications for obtaining a CT in this population were age specific. For example, in children younger than 2 years, providers were more concerned about the presence of abusive trauma and vomiting, whereas, in children older than 2 years, parental preference and scalp hematomas drove CT utilization. It may be noted that updated PECARN guidelines do not use vomiting as a risk factor, while parental preference is still a consideration in choosing CT over observation.3 The other interesting finding was that general pediatricians were more likely to order a CT compared with emergency medicine providers. Thus, the dissemination of this CPW to general pediatricians would be helpful to reduce CT use further. (See AAP Grand Rounds. 2017;37[6]:68.)4A CPW for the management of minor closed head trauma in children can lead to the reduction in the utilization of head CTs in childrenAbove and beyond decreasing unnecessary use of expensive resources and decreasing length of ED stay, decreased cranial CT frequency in children with minor TBI reduces exposure to potentially harmful ionizing radiation.4

Analysis of time-to-disposition intervals during early and late parts of an emergency department shift.

Time-to-disposition is an important metric for emergency department throughput. We hypothesized that providers view the shift end as a key timepoint and attempt to leave as few dispositions as possible to the oncoming team, thereby making quicker decisions later in the shift. This study evaluates disposition distribution relative to when patients are assigned a provider during the course of a shift. 50,802 cases were analyzed over the one-year study interval. 31,869 patients were seen in the early half of a shift (hours 1-4) and 18,933 were seen in the later half (hours 5+). We ran a linear mixed model that adjusted for age, gender, emergency severity index score, time of day, weekend arrivals, quarter of arrival and shift type. Median time-to-disposition for the early group was 3.25h (IQR 1.90-5.04), and 2.62h (IQR 1.51-4.31) for the late group. From our mixed model, we conclude that in the later parts of the shift, providers take on average 15.1% less time to make a disposition decision than in the earlier parts of the shift. Patients seen during the latter half of a shift were more likely to have a shorter time-to-disposition than similar patients seen in the first half of a shift. This may be influenced by many factors, such as providers spending the early hours of a shift seeing new patients which generate new tasks and delay dispositions, and viewing the end of shift as a landmark with a goal to maximize dispositions prior to sign-out.

Prediction across healthcare settings: a case study in predicting emergency department disposition

Several approaches exist today for developing predictive models across multiple clinical sites, yet there is a lack of comparative data on their performance, especially within the context of EHR-based prediction models. We set out to provide a framework for prediction across healthcare settings. As a case study, we examined an ED disposition prediction model across three geographically and demographically diverse sites. We conducted a 1-year retrospective study, including all visits in which the outcome was either discharge-to-home or hospitalization. Four modeling approaches were compared: a ready-made model trained at one site and validated at other sites, a centralized uniform model incorporating data from all sites, multiple site-specific models, and a hybrid approach of a ready-made model re-calibrated using site-specific data. Predictions were performed using XGBoost. The study included 288,962 visits with an overall admission rate of 16.8% (7.9–26.9%). Some risk factors for admission were prominent across all sites (e.g., high-acuity triage emergency severity index score, high prior admissions rate), while others were prominent at only some sites (multiple lab tests ordered at the pediatric sites, early use of ECG at the adult site). The XGBoost model achieved its best performance using the uniform and site-specific approaches (AUC = 0.9–0.93), followed by the calibrated-model approach (AUC = 0.87–0.92), and the ready-made approach (AUC = 0.62–0.85). Our results show that site-specific customization is a key driver of predictive model performance.

46 External Validation of a Singapore Electronic Health Record-Based Emergency Department Admission Prediction Model in a North American Setting

Models for predicting admission of patients from an emergency department (ED) have been suggested as a device for reducing ED crowding; however, such models are not widely implemented. The objective of this study is to validate an existing ED admission model on data from an outside hospital, as a first step to identifying a generalizable means to develop such models. Previously, an ED admission model was developed on retrospective observational data at a large urban hospital in Singapore. Data from the electronic health record (EHR) of a tertiary medical center in Durham, North Carolina (NC ED) were curated and transformed to align closely with the features of the Singapore model. A stepwise logistic regression model was built on these features to predict the probability of inpatient admission for patients presenting to the NC ED at the time of triage. A randomly selected 30% subset of the data was withheld from training and used to validate the model. Performance metrics were compared to those attained by the Singapore team. A total of 60, 064 encounters from January to December 2019 were included in the analysis; these patients had an admission rate of 28.2%. Variables used for modeling were first identified by the Singapore team and included age, sex, day of week, Emergency Severity Index (ESI), race, shift time, mode of arrival, time of year, postal code, fever status, and number of prior ED visits. The model we trained on the NC ED EHR data, using the features of the Singapore team’s model, had an area under the ROC curve (AUROC) of 0.821 (95% CI, 0.804-0.837), comparable to the AUROC of 0.825 (95% CI, 0.824-0.827) obtained by the Singapore team’s on their data. The most significant features associated with discharge included an ESI Score above 3 and ED arrival by law enforcement; the most significant features associated with admission included ESI Score below 3, fever status, and advanced age. We have been able to map elements from the NC ED EHR to the Singapore triage model elements to develop a model that performs comparably. Thus, a general method for building simple, effective ED admission models at various independent hospitals is attainable and scalable.