DELINEATE-Regurgitation: deep learning for automated assessment of aortic, mitral, and tricuspid regurgitation from echocardiography

Abstract

Abstract Background Valvular heart diseases, including aortic regurgitation (AR), mitral regurgitation (MR), and tricuspid regurgitation (TR) are significant contributors to cardiovascular morbidity and mortality worldwide. Artificial intelligence (AI) may improve echocardiographic assessment, but no models to date have successfully evaluated valvular regurgitation. Purpose The goal of this study was to develop an AI system to improve the accuracy of echocardiographic AR, MR, and TR interpretation. Methods Complete transthoracic echocardiogram (TTE) studies performed at a single center were split into train (n=65,301), validation (14,018), and test sets (13,746). The Deep Learning for Echo Analysis, Tracking, and Evaluation of Valvular Regurgitation (DELINEATE-Regurgitation) system intakes entire TTE studies, identifies color Doppler clips showing each valve, and yields a study level classification of AR, MR, and TR on a 6-grade scale (none/trace, mild, mild-moderate, moderate, moderate-severe, severe) using the cardiologist interpretation as a reference standard. This system used a hybrid neural network, leveraging the spatiotemporal feature learning capabilities of convolutional neural networks with the sequential processing strengths of transformer networks. The end-to-end hybrid approach allows the model to learn nuanced representations of the data including features within and across color Doppler videos. Model accuracy was assessed using quadratically weighted Cohen’s kappa (k), area under the receiver operator characteristic curve (AUROC) for the detection of both "moderate or greater" and "severe" regurgitation, and AI-cardiologist agreement the 6-grade scale. Additionally, an analysis was conducted using a 4-grade regurgitation scale (none/trace, mild, moderate, and severe) by rolling up intermediate classes (mild-moderate and moderate-severe) into the next higher class for both cardiologist determinations and model predictions. Results The DELINEATE-Regurgitation system demonstrated a high accuracy in the classification of AR (k=0.857), MR (k=0.867), and TR (k=0.847) (Figure 1). DELINEATE predicted the same severity as cardiologists (AR 91.8%, MR 79.9%, TR 77.7%) or within ±1 grade accuracy (AR 99.5%, MR 98.6%, TR 98.6%). The models had excellent binary discrimination in the detection of moderate or greater (AR AUROC 99.4, MR 98.6, TR 98.1) and severe (AR 99.7, MR 99.3, TR 99.6) regurgitation (Figure 2). In the 4-grade scale, the model had exact agreement with cardiologists in 92.3% of AR, 82.4% of MR, and 80.6% of TR cases. Panel adjudication of AI-cardiologist disagreement and correlation with quantitative metrics at higher levels of regurgitation will be presented. Conclusions The DELINEATE-Regurgitation AI system demonstrates a high level of accuracy in the assessment of AR, MR, and TR using color Doppler TTE videos. Future work will focus on automating quantification and developing strategies for optimal clinical deployment.Figure 1 - Confusion MatricesFigure 2 - ROC Curves