Abstract 4122612: Validation of a Machine Learning Model for Fetal Echocardiographic Prediction of Critical Coarctation of the Aorta

Abstract

Listen

Translate article

Background: Current fetal echocardiographic (F-echo) metrics have inadequate specificity for confident prediction of neonatal critical coarctation of the aorta (CoA). Using single-center data, our machine learning model for the prediction of fetal CoA demonstrated improved accuracy compared to published F-echo metrics for critical CoA assessment. External validation of this model is needed. Aim: Validate a machine learning F-echo predictive model for CoA with an external patient cohort. Methods: Initial model training and testing were performed using retrospective single center data on 9 F-echo measurements for patients with prenatal concern for CoA. A random forest classifier with 80:20 split and 5-fold cross-validation predicted CoA intervention within 30 days of life. A SHapley Additive exPlanations (SHAP) analysis assessed the marginal contribution of each feature. The model was retrained using the 5 most influential F-echo features. External validation for this model was then performed using patients with prenatal concern for CoA retrospectively collected at a partner institution. Results: Inclusion criteria were met by 132 patients in the initial cohort and 64 patients in the external validation cohort, of whom 44% (n=58) and 25% (n=16) respectively had CoA requiring intervention. SHAP analysis for both cohorts demonstrated transverse to descending aorta angle as the most influential feature, followed by ascending to descending aorta angle (Figure 1A). Using internal cross-validation on the initial cohort, the area under the receiver operating characteristic curve (AUC) was 0.93 ± 0.05 (sensitivity 0.97, specificity 1.0) with an F1 score of 0.97 ± 0.03. Validation of the model with the external cohort produced an AUC of 0.87 (sensitivity 0.81, specificity 1.0) and an F1 of 0.90 (Figure 1B). Conclusions: A random forest classifier using F-echo features predicted neonatal critical CoA with higher accuracy than previously published metrics. The model maintained high accuracy when validated with an external patient cohort. Arch angles most significantly impacted the model’s accuracy. Future directions include prospective validation and converting the model to a distributable clinical calculator.