135 How Accurate is “My Gut Feeling?”: Comparing the Accuracy of System 1 versus System 2 Decisionmaking for the Acuity Prediction of Patients Presenting to an Emergency Department

Abstract

Two models of decisionmaking have been characterized by cognitive sciences, in which System 1 represents a fast and information-limited model (eg, intuitive), whereas System 2 represents a slow, information-heavy cognitive model (eg, hypothetico-deductive). The aim of the study was to evaluate the accuracy of diagnostic prediction based on brief System 1 diagnostic reasoning compared to a System 2 process. We conducted a prospective observational study of emergency physicians, residents and faculty, to assess the accuracy of their System 1 and System 2 diagnostic reasoning for the diagnosis and acuity prediction of patients presenting to a tertiary care academic ED with 73,000 annual patient visits. Study personnel provided clinicians the following data: demographics, chief complaint, and vital signs. Clinicians were permitted to briefly observe (less than a minute) each patient as they were roomed and they have access to triage vital signs and chief complaint. Physicians were asked to predict if the patient was “sick” based on a standardized operational definition. Two independent clinician investigators, blinded to real-time assessments of “sick” versus “not sick,” ascertained the gold standard definition of “sick” or “not-sick” using previously published criteria. Disagreements were resolved by a third author. Prognostic accuracy estimates for sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and likelihood ratios (LR) were calculated, along with their respective 95% confidence intervals (CIs). We included a total of 662 observations of 289 different patients. Interobserver agreement (kappa) between observers was 0.97 (95% CI 0.95-099). Residents performed 416 (62.8%) of the assessments. For the “sick” versus “not sick” classification, providers overall had a sensitivity of 73.9% (CI 67.7 to 79.5%), specificity 83.3% (CI 79.5 to 86.7%), PPV 70.3%, NPV 85.7%, LR(+) 2.36 and LR(-) 0.17.There was no difference in sensitivity or specificity between attendings and residents in predicting “sick,” with attendings’ sensitivity of 77.5% (CI 66.8 to 86.1%) versus 72.0% (64.1 to 79.0%) for residents (P=.4), and specificity of 83.1% (76.6 to 88.5%) for attendings versus 83.5% (78.4 to 87.7%) for residents (P=.9). When emergency physicians use System 1 diagnostic reasoning for acuity prediction (ie, “sick”) the accuracy seems to be high, with a sensitivity of 74% and specificity of 83%, similar to the diagnostic yield of tests like the rapid influenza test. A brief System 1 decisionmaking process appears to be helpful in providing a cognitive framework for subsequent patient management.