471 Assessing the Accuracy and Bias of Digital Symptom Checkers with Myocardial Infarction Patients

Abstract

Abstract Aim The accuracy and safety of symptom checkers in diagnosing and triaging patients is of concern; especially those with life-threatening conditions. The study's aims were to: 1. assess the accuracy of symptom checkers in diagnosing and triaging myocardial infarctions (MI) and, 2. determine whether differences in gender or presentation type exist. Method This prospective diagnostic accuracy study assessed 8 symptom checkers using 100 MI patients of various presentations: typical or atypical. The ability of a symptom checker in providing MI as the first diagnosis (D1) and the first 3 (D3) diagnoses were diagnostic accuracy measures. Triage advice was deemed correct if the symptom checker recommended seeking emergency treatment. Results Symptom checkers correctly diagnosed 48.0±31.4% of cases with MI first. D3 accuracy was 72.6±20.2%. Mean triage accuracy was 82.6±12.6%. 24.0±16.2% of atypical cases had a correct primary diagnosis. D3 accuracy for atypical MI was 43.8±20.6%, significantly lower than that of typical MI (p<0.01). Atypical case triage accuracy was 52.7±20.0%, significantly lower than typical cases (84.2±14.7%, p<0.01). 10.0% of the atypical female cases were diagnosed correctly with MI as the first diagnosis. Female atypical cases had significantly lower accuracy than typical female cases for all accuracy measures (p<0.01). Conclusions Symptom checkers generally provide low accuracy for diagnosing MI. Approximately 20% of cases were under-triaged. Results varied between symptom checkers: patients who presented with atypical symptoms tended to be under-diagnosed and under-triaged, especially those who were female. This demonstrated potential gender bias and therefore raises questions regarding symptom checker regulation and safety.