Spatio-Temporal Hybrid Neural Networks Reduce Erroneous Human “Judgement Calls” in the Diagnosis of Takotsubo Syndrome

Abstract

Background: Distinguishing Takotsubo syndrome (TTS) from acute anterior wall myocardial infarction is often difficult based on clinic characteristics, biomarkers, electrocardiograms and noninvasive images, leading to dilemmas regarding treatment decisions. The aim of this study was to determine whether deep learning (DL) neural networks can reduce erroneous human “judgment calls” on bedside echocardiograms and improve differential diagnostic accuracy. Methods: We developed deep convolution neural networks (DCNNs), including a single-channel (DCNN[2D SCI]), a multi-channel (DCNN[2D MCI]) and a 3-dimensional (DCNN[2D+t]) network, and a recurrent neural network (RNN) based on the same database consisting of 17,280 still-frame images and 540 videos from 2-dimensional (2D) echocardiograms in a 12-year retrospective cohort of 540 patients in the University of Iowa (UI) and eight other medical centers in the United States. The diagnosis of anterior wall ST segment elevation myocardial infarction (STEMI) and TTS were all confirmed by the coronary angiography. Echocardiograms from 450 UI patients were randomly divided into training and testing sets for internal training, testing, and model construction. Echocardiograms of 90 patients from the other medical centers were used for external validation to evaluate the model generalizability. A total of 49 board-certified human readers (22 cardiologists, 11 senior echocardiographers, and 8 point-of-care ultrasound-trained clinicians) performed human-side classification on the same echocardiography dataset to compare the diagnostic performance and help data visualization. Findings: The DCNN (2D SCI), DCNN (2D MCI), DCNN(2D+t), and RNN models established based on UI dataset for the control versus disease prediction showed mean diagnostic accuracy of 78%, 83%, 92%, and 81% respectively. The DCNN (2D SCI), DCNN (2D MCI), DCNN(2D+t), and RNN models established based on UI dataset for TTS versus STEMI prediction showed mean diagnostic accuracy 73%, 75%, 80%, and 75% respectively, and the mean diagnostic accuracy of 74%, 74%, 77%, and 73%, respectively, on the external validation. The area under the receiver operating characteristic curve (AUC) analysis showed that DCNN(2D+t) (0·787 vs. 0·699, P = 0·015) and RNN models (0·774 vs. 0·699, P = 0·033) consistently outperformed human readers in differentiating TTS and STEMI by reducing the erroneous judgement calls on TTS from human readers. Interpretation: Spatio-temporal hybrid DL neural networks reduce erroneous human “judgement calls” in distinguishing TTS from anterior wall STEMI based on bedside echocardiographic videos, demonstrating the potential of DL neural networks to support frontline triage and management of cardiovascular emergencies. Funding: University of Iowa Obermann Center for Advanced Studies Interdisciplinary Research Grant, and Institute for Clinical and Translational Science Grant. Declaration of Interest: We declare no competing interests. Ethical Approval: The research protocols and waiver of informed consent were approved by the human subjects committee of the UI institutional review board.