Abstract TP468: Inter-rater Reliability Process Optimizes Quality of a Registries Data

Abstract

Background and Purpose: Ensuring the quality of a registry’s data is essential to its credibility and reliability of the information gathered. The Wisconsin Coverdell Stroke Program (WI Coverdell) monitors inter-rater reliability (IRR) through quarterly re-abstraction of five patient records from each of its participating hospitals. The goal for individual sites is 90% or greater concordance rate for each of the 28 data elements measured. This quarterly process ensures two abstractors at participating sites are familiar with the oftentimes difficult stroke data entry process. It also provides added benefit to hospitals by supporting continued quality improvement initiatives, as well as assists in meeting their certification bodies’ requirements. Methods: Participating hospitals gain access to two stroke data entry sites. One site is for entry of stroke abstracts; the second is utilized solely for re-abstraction entry. Quarterly, WI Coverdell performs an analysis of data agreement of the abstractions and re-abstractions. A subsequent quality report is sent to the hospital contact where the data elements that had mismatches are highlighted. Findings: Over nine quarters we have found individual sites agreement rates between two abstractors have varied from 70% to 100%. Whereas, WI Coverdell hospitals aggregated agreement rate has been stable at 92%-94%. Perceived rationales for agreement rates that are <90% include (1) the need for an identified source of truth for certain data elements, and (2) as new abstractors or re-abstractors begin stroke data entry, due to turnover in their roles, educational opportunities exist for understanding or clarification of the stroke coding instructions. Conclusions: A quarterly IRR process ensures two abstractors are proficient at data entry at participating hospitals, demonstrates the need for creating a source of truth document, and identifies learning opportunities for individual data abstractors.