Relationship between ECG beat detector performance over short time intervals and correct detection of arrhythmic events

Abstract

Introduction Performance of ECG beat detectors are traditionally assessed using gross sensitivity (Se) and positive predictive value (PPV) on human-annotated datasets over large time windows (e.g.: 30 min). Exceptional performance on gross scores (> 99%) can be achieved by same number of incorrect beat detections occurring nearby or faraway over time. However, missing a true arrhythmic event or detecting an arrhythmic event when one is not actually present can be caused by a limited number of incorrect beat detections occurring nearby over a short period. We compared the relationship between performance of ECG beat detectors measured over short time intervals or by traditional approaches with the correct detection of arrhythmic events. Methods We assessed performance (Se and PPV) of 5 beat detector algorithms over short time intervals (10 s epochs with 5 s overlap) on MIT-BIH arrhythmia database signals. Histograms of Se and PPV were calculated with bin size 1%. Using the histogram, estimated cumulative distribution function (ecdf) for “performance over short interval” was constructed and the area under the ecdf (AUecdf) was calculated. ECG signals where reviewer beat annotations confirmed a true/false alarm in the last 20 s were selected from PhysioNet/Computing in Cardiology Challenge (CinC) 2015 training set. Asystole, bradycardia, and tachycardia alarms were considered. A missed alarm was declared if detected beats did not meet the alarm condition during last 20 s epoch on a record annotated as true alarm. A false alarm was declared if the alarm condition was met on a record annotated as false alarm. For each beat detector total number of incorrect events was the sum of missed and false alarms. The monotonicity of association between performance metrics and number of incorrect events was tested using Spearman's rank correlation coefficient (ρ). Results Table shows performance results for each detector. The number of incorrect events on CinC had no significant correlation with MIT-BIH performance reported as Gross Se (ρ = − 0.60, p = 0.28) or PPV (ρ = − 0.40, p = 0.50) or AUecdf PPV (ρ = − 0.10, p = 0.87). However, AUecdf for Se had the strongest relationship (ρ = 0.90, p = 0.04). Detector Beat detector performance on MIT-BIH arrhythmia database Incorrect arrhythmia detections on CinC 2015 training set Gross score AUecdf Missed alarms False alarms Incorrect arrhythmia events Se PPV Se PPV A B T A B T 1 99.15 99.55 1.98 1.46 1 0 6 17 6 3 33 2 98.59 99.67 2.13 1.28 0 1 18 7 4 0 30 3 99.78 98.82 1.64 2.26 1 1 0 5 2 6 15 4 99.72 99.71 1.57 1.54 0 0 3 1 3 1 8 5 98.86 99.31 2.38 1.85 0 2 14 13 6 4 39 A: Asystole; B: Bradycardia; T: Tachycardia. Discussion A significant relationship between detector performance reported as short time interval metrics with the number of incorrect arrhythmia events was found for Se but not for PPV. Considering performance over short intervals may provide insights on real world performance of beat detectors as automated arrhythmia detectors.