The Epic Sepsis Model Falls Short—The Importance of External Validation

Abstract

Sepsis accounts for nearly 1 million hospitalizations annually and is a major contributor to hospital length of stay, health care expenditures, and in-hospital mortality (ranging from 12.5%-15%).1 Early sepsis identification allows care teams to promptly implement goal-directed therapy to mitigate clinical deterioration. In this issue of JAMA Internal Medicine, Wong et al2 report on their external validation of the Epic Sepsis Model (ESM), a prediction tool available within the Epic electronic health record that is designed to generate automated alerts that warn clinicians that patients may be developing sepsis. Based on their examination of 38 455 hospitalizations at the University of Michigan (Ann Arbor) between December 2018 and October 2019, Wong et al2 found that the ESM had a sensitivity of 33%, specificity of 83%, positive predictive value of 12%, and negative predictive value of 95%, with an area under the curve of 0.63 (95% CI, 0.62-0.64). This falls short of the area under the curve of 0.76 to 0.83 that was jointly reported by Epic and University of Colorado Health.3 Despite generating alerts on 18% of all patients, the ESM did not detect sepsis in 67% of patients with sepsis. Identify all potential conflicts of that might be relevant to your comment. Conflicts of comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued. Err on the side of full disclosure. If you have no conflicts of interest, check No potential conflicts of interest in the box below. The information will be posted with your response. Not all submitted comments are published. Please see our commenting policy for details.