Abstract 9389: Apache Clinical Text Analysis Knowledge Extraction System (ctakes) Identifies One Out of Five Suspicious Symptoms of Acute Coronary Syndrome from Emergency Department Nursing Triage Notes

Abstract

Introduction: Acute coronary syndrome is a time sensitive condition requiring rapid identification, but nurses only identify 40%-50% of high-risk patients at emergency department (ED) triage. Symptoms are the driving force behind prompt ED care of patients with ACS. Symptom data are recorded as free text in nursing narratives. Standardized extraction of symptoms via natural language processing will create usable clinical data that could be used in predictive analytics, to improve early identification of ACS and ensure life-saving treatments. Hypothesis: The Apache Clinical Text Analysis and Knowledge Extraction System (cTAKES) will recognize symptom concepts in nursing narratives with an acceptable F1 score of greater than or equal to 0.9. Methods: A nurse scientist and a physician annotated 1200 ED nursing narratives of patients with suspicion of ACS for the following symptoms predictive of ACS: chest pain, chest discomfort, shortness of breath, fatigue and sweating. Inter-rater reliability (Cohen’s kappa statistic) was calculated with 10% overlap. cTAKES, which is an open-source natural language processing platform, was applied to extract evidence of the 5 annotated symptoms. Recall (sensitivity), precision (positive predictive value) and F1 score (harmonic mean of precision and recall) were calculated to assess extraction quality for each symptom. Results: Recall, precision and F1 score for each symptom are as follows: chest pain: 0.71, 0.97, 0.823; chest discomfort: 0.6, 0.9, 0.72; fatigue: 0.9, 0.26, 0.41; shortness of breath: 0.93, 0.96, 0.95; and sweating: 0.54, 1, 0.7, respectively. Cohen’s kappa was 0.9. Conclusions: cTAKES had fair results across all 5 ACS predictive symptoms, with shortness of breath meeting our hypothesized F1 score goal. Failure to recognize symptoms at a clinically acceptable value might be due to the linguistic and content variation present in ED nursing initial narratives. Customized natural language processing pipelines trained specifically for ED nursing narratives have significant potential for accurately identifying symptoms and improving early identification of ACS.