Does ChatGPT-4 succeed in the ECG interpretation: friend or foe to cardiologists?

Abstract

Abstract Background/Introduction Artificial intelligence (AI) and large language models (LLMs) are increasingly being tested for medical purposes, especially in clinical settings and one of the most well-known LLMs is OpenAI’s ChatGPT-4. Recent explorations have considered using ChatGPT to answer medical questions, and the new ChatGPT-4 also allows graphical inputs, which could aid medical doctors in interpreting electrocardiograms (ECGs) in everyday practice. AI-powered ECG interpretation has shown promising results, improving the detection of arrhythmias, ST-segment changes, QT prolongation, and other ECG abnormalities. However, whether ChatGPT4’s interpretation of ECGs is comparable to that of experienced cardiologists has yet to be established. Purpose We sought to test whether ChatGPT-4 could analyze and interpret ECGs similarly to experienced cardiologists. Methods We tasked ChatGPT-4 with interpreting, for scientific purposes, 12-lead ECGs, from 150 patients included in our prospective CaRD registry, presenting with various cardiac pathologies, including acute coronary syndrome, arrhythmias (primarily atrial fibrillation), conduction abnormalities, pacemaker rhythms and normal ECGs. Four experienced cardiologists reviewed the ECG interpretations provided by ChatGPT-4. The model provided information on rate, rhythm, axis, P-wave, PR interval, QRS complex, ST segment, T wave, differential diagnoses, most likely diagnosis and further diagnostic step. Results In this observational study, ChatGPT-4 demonstrated low overall accuracy in interpreting ECGs, with only 24% of the most likely diagnosis being correct. ChatGPT-4 performed the best when interpreting normal ECGs with sinus rhythm, achieving 62% accuracy rate, followed by 17%, 10% and 7.3% for ECGs with arrhythmias, acute ischemic changes, and conduction abnormalities or pacing rhythms, respectively. Conclusion ChatGPT-4 exhibited very low accuracy in interpreting different categories of pathology in 12-lead ECG, yielding less than 20% success rate. While the use of AI in medical diagnostics is an appealing concept in theory, the results suggest that, in its current state, ChatGPT-4 would not provide significant aid to experienced cardiologists in clinical settings.