294 Evaluation of the Clinical Utility of ANCHOR (An Artificial Neural Network for Chronic Subdural Hematoma Referral Outcome Prediction): An External Validation Study of 1713 Patient Referrals

Abstract

Abstract Aim Chronic subdural hematoma (CSDH) incidence and referral rates to neurosurgery are rapidly increasing. We have previously shown that ANCHOR, a locally developed deep learning machine learning (ML) model can accurately predict CSDH referral outcome with a high degree of precision. Our objective for this study was to externally validate the accuracy of the ANCHOR model at predicting CSDH referral outcomes in an external tertiary neurosurgical centre. Method Patients referred to the external validation neurosurgical centre for CSDH were retrospectively analysed for 7 patient features previously identified over the same 5-year period (2015-2020). The optimal ML model identified in the previous study, the Artificial Neural network for Chronic subdural HematOma Referral outcome prediction (ANCHOR), https://medmlanalytics.com/neural-analysis-model/, was then evaluated on 6 performance metrics using the new cohort. Results 1713 referrals were identified, of which only 29.5% (505) were accepted referrals. The ANCHOR model demonstrated good discrimination and calibration, with an accuracy of 92.294% [95%CI: 90.952 – 93.520], sensitivity of 82.970 [95%CI: 79.644 – 86.180], specificity of 96.190% [95%CI: 94.951 – 97.202], PPV of 90.108% [95%CI: 87.197 – 92.688], an AUROC of 0.896 [95%CI: 0.878 – 0.912], and a brier score loss of 0.077 [95%CI: 0.065 – 0.091]. Decision curve analysis further demonstrated superior clinical net benefit across all patient risk threshold probabilities. Conclusions This study successfully validated the clinical utility of the ANCHOR ML model in predicting CSDH referral outcome at an external centre. ANCHOR was able to accurately predict referral outcomes and can potentially be used in clinical practice to support CSDH referral decision making.