Validation of an automated artificial intelligence system for 12‑lead ECG interpretation

Abstract

BackgroundThe electrocardiogram (ECG) is one of the most accessible and comprehensive diagnostic tools used to assess cardiac patients at the first point of contact. Despite advances in computerized interpretation of the electrocardiogram (CIE), its accuracy remains inferior to physicians. This study evaluated the diagnostic performance of an artificial intelligence (AI)-powered ECG system and compared its performance to current state-of-the-art CIE. MethodsAn AI-powered system consisting of 6 deep neural networks (DNN) was trained on standard 12‑lead ECGs to detect 20 essential diagnostic patterns (grouped into 6 categories: rhythm, acute coronary syndrome (ACS), conduction abnormalities, ectopy, chamber enlargement and axis). An independent test set of ECGs with diagnostic consensus of two expert cardiologists was used as a reference standard. AI system performance was compared to current state-of-the-art CIE. The key metrics used to compare performances were sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and F1 score. ResultsA total of 932,711 standard 12‑lead ECGs from 173,949 patients were used for AI system development. The independent test set pooled 11,932 annotated ECG labels. In all 6 diagnostic categories, the DNNs achieved high F1 scores: Rhythm 0.957, ACS 0.925, Conduction abnormalities 0.893, Ectopy 0.966, Chamber enlargement 0.972, and Axis 0.897. The diagnostic performance of DNNs surpassed state-of-the-art CIE for the 13 out of 20 essential diagnostic patterns and was non-inferior for the remaining individual diagnoses. ConclusionsOur results demonstrate the AI-powered ECG model's ability to accurately identify electrocardiographic abnormalities from the 12‑lead ECG, highlighting its potential as a clinical tool for healthcare professionals.