Inter-rater Reliability and Misclassification of the ABCD2 Score after Transient Ischemic Attack

Abstract

The ABCD(2) score was initially developed as a simple tool to help first-line clinicians identify patients at highest short-term risk for stroke after transient ischemic attack (TIA). The score is increasingly used for risk stratification of TIA patients, but little is known about its inter-rater reliability. The aim of the present study was to prospectively assess the inter-rater reliability of the ABCD(2) score in patients with TIA, including a comparison among raters of different specialties. Patients presenting to the emergency department with TIA within 48hours of onset were prospectively evaluated. TIA was defined as acute onset of focal cerebral or monocular symptoms lasting less than 24hours and presumed because of a vascular cause. Only patients who were asymptomatic at the time of enrollment were eligible. ABCD(2) scores determined by raters of different specialties were compared with those of a vascular neurology attending. Estimated component and total scores and ABCD(2) risk category were compared between raters. Reliability was assessed using unweighted kappa statistics. A total of 362 evaluations resulting in ABCD(2) scores were performed. In addition to the vascular neurology attending, scores were generated by internal medicine (n=72), emergency medicine (n=37), and neurology junior (n=92) and senior (n=57) residents. Based on attending scores, 35% of patients were categorized as low risk (ABCD(2) score, 0-3), 50% as moderate risk (ABCD(2) score, 4-5), and 16% as high risk (ABCD(2) score, 6-7). Inter-rater reliability was fair for ABCD(2) total score (κ=.26) and category (κ=.29). Raters agreed with the vascular neurology attending 67% (95% confidence interval [CI], 61%-73%) of the time for ABCD(2) category and 52% (95% CI, 46%-58%) of the time for ABCD(2) total score. Disagreement more often resulted in a lower score by the raters as compared with the vascular neurology attending for both ABCD(2) total score and category. Inter-rater reliability of component scores was near perfect for age (κ=.95) and diabetes (κ=.81) and substantial for blood pressure (κ=.67), but only moderate for clinical features (κ=.55) and duration (κ=.48). The inter-rater reliability of the ABCD(2) score is only fair, with rater disagreement of ABCD(2) risk category in nearly one third of patients.