Development and validation of a prehospital termination of resuscitation (TOR) rule for out - of hospital cardiac arrest (OHCA) cases using general purpose artificial intelligence (AI)

Abstract

BackgroundPrehospital identification of futile resuscitation efforts (defined as a predicted probability of survival lower than 1%) for out-of-hospital cardiac arrest (OHCA) may reduce unnecessary transport. Reliable prediction variables for OHCA 'termination of resuscitation' (TOR) rules are needed to guide treatment decisions. The Universal TOR rule uses only three variables (Absence of Prehospital ROSC, Event not witnessed by EMS and no shock delivered on the scene) has been externally validated and is used by many EMS systems. Deep learning, an artificial intelligence (AI) platform is an attractive model to guide the development of TOR rule for OHCA. The purpose of this study was to assess the feasibility of developing an AI-TOR rule for neurologically favorable outcomes using general purpose AI and compare its performance to the Universal TOR rule. Methods: We identified OHCA cases of presumed cardiac etiology who were 18 years of age or older from 2016 to 2019 in the All-Japan Utstein Registry. We divided the dataset into 2 parts, the first half (2016–2017) was used as a training dataset for rule development and second half (2018–2019) for validation. The AI software (Prediction One®) created the model using the training dataset with internal cross-validation. It also evaluated the prediction accuracy and displayed the ranking of influencing variables. We performed validation using the second half cases and calculated the prediction model AUC. The top four of the 11 variables identified in the model were then selected as prognostic factors to be used in an AI-TOR rule, and sensitivity, specificity, positive predictive value, and negative predictive value were calculated from validation cohort. This was then compared to the performance of the Universal TOR rule using same dataset. ResultsThere were 504,561 OHCA cases, 18 years of age or older, 302,799 cases were presumed cardiac origin. Of these, 149,425 cases were used for the training dataset and 153,374 cases for the validation dataset. The model developed by AI using 11 variables had an AUC of 0.969, and its AUC for the validation dataset was 0.965. The top four influencing variables for neurologically favorable outcome were Prehospital ROSC, witnessed by EMS, Age (68 years old and younger) and nonasystole. The AUC calculated using the 4 variables for the AI-TOR rule was 0.953, and its AUC for the validation dataset was 0.952 (95%CI 0.949 –0.954). Of 80,198 patients in the validation cohort that satisfied all four criteria for the AI-TOR rule, 58 (0.07%) had a neurologically favorable one-month survival. The specificity of AI-TOR rule was 0.990, and the PPV was 0.999 for predicting lack of neurologically favorable survival, both the specificity and PPV were higher than that achieved with the universal TOR (0.959, 0.998). ConclusionsThe accuracy of prediction models using AI software to determine outcomes in OHCA was excellent and the AI-TOR rule’s variables from prediction model performed better than the Universal TOR rule. External validation of our findings as well as further research into the utility of using AI platforms for TOR prediction in clinical practice is needed.