The Success and Failure of a Radiology Quality Metric: The Case of OP-10

Abstract

PurposeMedical quality metrics can serve various functions, including promoting quality improvement efforts within a medical system, and providing a basis for comparing quality among institutions. OP-10, an imaging efficiency quality measure based on the number of CT scans of the abdomen performed both with and without contrast is broadly used and publically reported, but it has not been investigated in actual practice. MethodsIn this project, we report on both a successful quality improvement effort built around measurement of OP-10, and on the identified potential limitations of OP-10 itself for comparing among institutions. We performed two quality improvement interventions in 2012 and 2014 directed at OP-10, encompassing building of institutional practice standards via creating multidisciplinary consensus, educating radiologists and clinicians, revising CT protocols, and providing performance measurement and feedback. Results were extracted from the radiology information system and analyzed using interrupted time series segmented regression and statistical process control charts. ResultsThe proportion of inappropriate abdominal CT scans with and without contrast under OP-10 decreased, from 3,041 of 13,855 (21.9%) to 691 of 6,006 (11.5%) (P < .0001). However, 262 of 691 (37.8%) of the CT scans labeled as potential overuse by OP-10 could be considered appropriate under national guidelines. These discordant cases clustered in specific clinical areas (eg, urology and hepatology), indicating potential for bias against centers that serve referral populations in these areas. ConclusionsWe conclude that OP-10 can be useful to drive internal quality improvement efforts but is potentially biased when used for interinstitutional comparisons.