Continental generalization of a human-in-the-loop AI system for clinical seizure recognition

Abstract

Electroencephalogram (EEG) monitoring and objective seizure identification is an essential clinical investigation for some patients with epilepsy. Accurate annotation is done through a time-consuming process by EEG specialists. Computer-assisted systems for seizure detection currently lack extensive clinical utility due to the retrospective, patient-specific, and/or irreproducible studies that result in low sensitivity or high false positives in clinical tests. We aim to significantly reduce the time and resources on data annotation by demonstrating a continental generalization of seizure detection that balances sensitivity and specificity. This is a prospective inference test of artificial intelligence on nearly 14,590 hours of adult EEG data from patients with epilepsy between 2011 and 2019 in a hospital in Sydney, Australia. The inference set includes patients with different types and frequencies of seizures across a wide range of ages and EEG recording hours. Artificial intelligence (AI) is a convolutional long short-term memory network that is trained on a USA-based dataset. The Australian set is about 16 times larger than the US training dataset with very long interictal periods (between seizures), which is way more realistic than the training set and makes our false positives highly reliable. We validated our inference model in an AI-assisted mode with a human expert arbiter and a result review panel of expert neurologists and EEG specialists on 66 sessions to demonstrate achievement of the same performance with over an order-of-magnitude reduction in time. Our inference on 1006 EEG recording sessions on the Australian dataset achieved 76.68% with nearly 56 [0, 115] false alarms per 24 hours on average, against legacy ground-truth annotations by human experts, conducted independently over nine years. Our pilot test of 66 sessions with a human arbiter, and reviewed ground truth by a panel of experts, confirmed an identical human performance of 92.19% with an AI-assisted system, while the time requirements reduced significantly from 90 to 7.62 min on average. Accurate and objective seizure counting is an important factor in epilepsy. An AI-assisted system can help improve efficiency and accuracy alongside human experts, particularly in low and middle-income countries with limited expert human resources.