Abstract P100: An Interactive, Training-Based Data Audit Approach Improves Subsequent Data Validation in a Large State-Based Registry

Abstract

Objective: We sought to determine whether an interactive, training-based approach to data validation would effectively reduce data element discrepancy rates in subsequent years. Methods: Fifty-eight Massachusetts hospitals participate in the Coverdell Stroke Registry and collect data on stroke-care processes using national definitions. From 2007 to 2009 auditors visited selected sites annually to assess data accuracy. Initial data element selection was based on performance measure inclusion. Subsequent audits focused on new data elements and those with frequent errors. Full-day site visits included: re-abstraction of approximately 10 charts and discussion with hospital staff to clarify discrepancies. Emphasis was on teaching to promote data quality improvement. Written feedback was sent to the coordinators following each visit. Variable-level discrepancy rates were calculated as the number of errors for a given variable over the total number of cases reviewed system-wide. A “low” discrepancy rate of < 5% was the goal. Discrepancy rates of > 20% were considered “high”. Results: From 2007 through 2009, a total of 78 stroke registry variables were audited in more than 1,300 patient records during 147 unique hospital visits. The mean variable-level discrepancy rate decreased significantly from 11% in 2007 to 5% in 2008 (p = 0.006) and remained at 5% in 2009. Evidence of decline in the “high” discrepancy rate elements were of particular interest as they became the focus of various training opportunities, including the audit visits. The percentage of variables with a “high” range of errors significantly declined from 20% (10 of 51) in 2007 to 3% (1 of 36) in 2008 to 0% (0 of 46) in 2009 (2007 vs. 2009; p-value = 0.001). Conclusion: The on-site audits proved to be teachable moments for hospital staff resulting in highly effective and valuable one-on-one training. Commonly identified errors were also highlighted at periodic Registry trainings, and subsequent revisions to the data collection tool and reference guides may also have facilitated improvement. While various approaches to assess data reliability exist, using the chart audit as a vehicle for abstractor training was effective in improving discrepancy rates. Further research is warranted.