Development of a Machine Learning Model Using Electrocardiogram Signals to Determine Acute Pulmonary Embolism

Abstract

Background. Patients with acute pulmonary embolism (PE) benefit from immediate initiation of anticoagulation but similar symptoms can come from clinical conditions for which anticoagulation can be detrimental. A method for immediate, bedside diagnosis or exclusion of PE is not available. Objective. To develop an artificial intelligence (AI) model enabling analysis of 12-lead electrocardiogram (ECG) to detect the presence of acute PE and the categories of acute PE with right ventricular strain (RVS). Methods. One cohort of patients was selected by using a newly developed highly accurate natural language processing (NLP) model to analyze radiology reports of CT angiogram (CTA) and ECG within 6 hours before or after CTA from Mayo Clinic enterprise sites consisting of three tertiary referral centers (Rochester, MN, Phoenix, AZ, and Jacksonville, FL) and regional Mayo Clinic Health System sites in MN, WI, and IA. We identified patients with acute PE, acute PE with RVS, acute saddle PE, and patients with no PE on the CTA. Another cohort consisted of consecutive patients with acute PE who were enrolled into the Mayo Thrombophilia Clinic Registry (TCR) between March 1, 2013 and May 31, 2022 and prospectively followed. For this group, AHA criteria were used to classify PE as massive, submassive, and low risk PE category which for this study was described as PE with no right ventricular dysfunction (NRVD). For this cohort, ECG performed within 24 hours before or after CTA was analyzed. We allocated patients with positive PE testing and negative PE to the training, internal validation, and testing datasets in a 7:1:2 ratio to develop and validate an AI-enabled algorithm using a convolutional neural network to detect the ECG signature of any acute PE, PE with RVS or saddle PE (RVS/SPE) and massive/submassive PE. We calculated the area under the curve of the receiver operating characteristic curve (AUROC) for the internal validation dataset to select a probability threshold, which we applied to the testing dataset. We evaluated model performance on the testing dataset by calculating the AUROC, accuracy, sensitivity, and specificity. Results. NLP identified a total of 80,432 patients with ECG performed within 6 hours before or after CTA who were not a part of the TCR. This cohort was subdivided into 79,894 consisting of acute PE and no PE and another group of 73,609 consisting of either acute PE with RVS/SPE, and no PE. In the former group, 7,423 (9.29%) had acute PE (mean age 63.7±15.9, 46.1% female), and in the latter group, 1,138 (1.55%) had acute PE with RVS/SPE (mean age 65.6±14.1, 45.7% female). Both groups had the same number of negative studies (72,471) for any PE (mean age 60.5±17.6, 52.3% female). Within the group of 4,818 TCR patients with acute venous thromboembolism, 1,007 had acute PE and ECG 24 hours before or after CTA (mean age 62.7±14.6, 43.8% female) including 44 with massive, 317 submassive, 508 NRVD, and 138 with subsegmental PE. For the TCR cohort, we created a control group from NLP identified cases with CTA negative for PE by selecting 6,774 patients that were matched (by age, sex) with acute PE cases from the TCR. The deep neural network prediction of acute PE in patients identified by NLP from Mayo Clinic sites to those without any PE was modest (AUROC 0.6929, see Figure 1). The sensitivity was 63.54% and specificity of 64.66%. This was associated with the positive predictive value (PPV) of 15.55% and high negative predictive value (NPV) of 94.54%. The performance of the deep neural network prediction for the presence of RVS/SPE to the group with negative testing for any PE was better (AUROC 0.8041). The sensitivity was 68.42% and specificity of 74.77%. This was associated with the PPV of 4.07% and NPV as high as 99.34%. The deep neural network prediction of any acute PE from TCR patients was similar to the NLP selected cohort from Mayo Clinic sites (AUROC 0.6756). The sensitivity was 54.95% and specificity of 69.70%. This was associated with the PPV of 21.18% and NPV of 91.26%. Prediction of massive or submassive PE from the TCR group was very well demonstrated within the TCR group (Figure 2). Conclusions. The AI-based analysis of 12-lead ECG shows modest detection power for acute PE, but is accurate for high risk PE detection. Moreover, it provides a fast and very reliable way to exclude the presence of severe PE and therefore the need for immediate anticoagulation and fibrinolytic therapy. Figure 1View largeDownload PPTFigure 1View largeDownload PPT Close modal