Online Symptom Checker Accuracy for the Complex HIV and Hepatitis C Population: A Real Life Emergency Room Experience

Abstract

Introduction: We previously analyzed online patient symptom checkers for both diagnostic and triage accuracy for a barrage of GI illnesses using traditional provided clinical vignettes. We now aim to see whether symptom checkers are feasible diagnostic and triage tools for the complex HIV or Hepatitis C patient, using real-life emergency room (ER) patient encounters. Methods: An ER chart analysis from Sept 2013-Sept 2015 for a 10-day block each month was performed on 8363 patients, searching for “HIV” and “Hepatitis C”. No distinction of lab data, treatment regimen, or active disease was made. Patients who came as a direct transfer, sent from clinic, or unable to consciously make a decision were excluded. Eligible patients included: 90 HIV, 67 HepC, 11 Both (Fig. 1). 5 online symptom checkers were utilized for diagnosis, 3 with triage capabilities.Figure: Baseline demographics.Results: When combining HIV and HepC patients, only a mere 6.0%-11.3% of the symptom checkers had the initial diagnosis listed as the Top1 diagnosis. Between 8.9% and 32.1% had the diagnosis listed in the Top10, with only 8.9%-39.3% having the diagnosis listed at all (Fig. 2). Only about half the time was the triage advice correct (44.6%-53.6%). Each checker did display various strengths and weaknesses in triage capabilities (Fig. 2). Whether the diagnosis was considered GI-based did not affect triage ranking as emergent or non-emergent. However, there were varying results for GI versus non-GI diagnosis accuracy. There remained significant inconsistencies between the five symptom checkers in regards to all diagnostic parameters. A model was built to help predict which variables (race, age, gender, HIV or HepC) would aid in predicting triage and diagnostic results. We did find that a higher number of orders in the ER (labs, meds, Rx given, Imaging, Consultant) did positively correlate with the symptom checker triage output as “Emergent” (p=0.034), further justifying the ER visit.Figure: A.) Diagnostic accuracy for initial diagnosis. B). Diagnostic accuracy for discharge diagnosis. C.) Sensitivity: % Emergent among all predicted as Emergent; Specificity: % Non-Emergent among all predicted as Non-Emergent; POS pred value: % Emergent correctly predicted as Emergent; NEG Pred Value: % Non-Emergent correctly predicted as Non-Emergent; Accuracy: % Correct triage (Emergent as Emergent, Non-Emergent as Non-Emergent).Conclusion: Symptom checker diagnostic and triage capability for complex HIV and HepC patients is quite inferior to prior study analysis of the general patient population. In fact, the substandard accuracy may even be at such dangerous levels to where patients are not being triaged to a safe venue. As this is the first study to utilize real-life emergency room data on a large-scale, part of the results may be linked to real-life, non-textbook, complex clinical vignette patient presentations, something prior studies have failed to analyze.