Explainable machine learning model reveals its decision-making process in identifying patients with paroxysmal atrial fibrillation at high risk for recurrence after catheter ablation

Abstract

BackgroundA number of models have been reported for predicting atrial fibrillation (AF) recurrence after catheter ablation. Although many machine learning (ML) models were developed among them, black-box effect existed widely. It was always difficult to explain how variables affect model output. We sought to implement an explainable ML model and then reveal its decision-making process in identifying patients with paroxysmal AF at high risk for recurrence after catheter ablation.MethodsBetween January 2018 and December 2020, 471 consecutive patients with paroxysmal AF who had their first catheter ablation procedure were retrospectively enrolled. Patients were randomly assigned into training cohort (70%) and testing cohort (30%). The explainable ML model based on Random Forest (RF) algorithm was developed and modified on training cohort, and tested on testing cohort. In order to gain insight into the association between observed values and model output, Shapley additive explanations (SHAP) analysis was used to visualize the ML model.ResultsIn this cohort, 135 patients experienced tachycardias recurrences. With hyperparameters adjusted, the ML model predicted AF recurrence with an area under the curve of 66.7% in the testing cohort. Summary plots listed the top 15 features in descending order and preliminary showed the association between features and outcome prediction. Early recurrence of AF showed the most positive impact on model output. Dependence plots combined with force plots showed the impact of single feature on model output, and helped determine high risk cut-off points. The thresholds of CHA2DS2-VASc score, systolic blood pressure, AF duration, HAS-BLED score, left atrial diameter and age were 2, 130 mmHg, 48 months, 2, 40 mm and 70 years, respectively. Decision plot recognized significant outliers.ConclusionAn explainable ML model effectively revealed its decision-making process in identifying patients with paroxysmal atrial fibrillation at high risk for recurrence after catheter ablation by listing important features, showing the impact of every feature on model output, determining appropriate thresholds and identifying significant outliers. Physicians can combine model output, visualization of model and clinical experience to make better decision.