Abstract TMP96: Development and Validation of a Certification Test for Scoring the Modified Rankin Scale Using the Rankin Focused Assessment

Abstract

Background: The modified Rankin Scale is the most common primary endpoint in acute stroke clinical trials and is a new requirement for Comprehensive Stroke Center clinical practice. The Rankin Focused Assessment (RFA) is a simple, operationalized method to obtain a modified Rankin Scale score and has been shown to have excellent inter-rater reliability. For wide deployment in clinical trials and clinical practice, it is desirable to develop a rater testing and certification process for the RFA method to assign a mRS score. Methods: We developed 30 written case vignettes spanning the entire range of the modified Rankin Scale, based on actual patients enrolled in the NIH FAST-MAG Phase 3 trial. Each vignette briefly portrays patient function at 3 months poststroke, based on available sources of information (direct examination, patient report, and family report). The correct mRS score for each patient was determined by two expert physician-investigators (criterion standard). Vignettes were then beta-tested in a group of 15 experienced mRS raters (14 nurses, 1 physician) and 3 were reworded for clarity and retested. Results: Among all 30 individual vignettes, rater scores concurred with the criterion standard in 435/443 (98.2%). The final set of 30 vignettes was divided into 5 alternative test sets of 6 vignettes each. A score of 100% (6/6 correct) was required for rater certification in the RFA method to elicit the mRS. On the test sets of 6 vignettes, the rater pass rate was 84.3% (59/70). Raters missed passing by 1 (5 right out of 6) in 14.3% of sets and by 2 (4 right out of 6) in 1.4% of sets. Conclusion: A written vignette certification test for the Rankin Focused Assessment method to assign a modified Rankin Scale score has been developed and demonstrated good concurrence of experienced raters with criterion standards. A public service website at which individuals may obtain Rankin Scale certification using the RFA will be demonstrated.