Do Experts Understand Performance Measures? A Mixed-Methods Study of Infection Preventionists.

Abstract

Cross-sectional survey PARTICIPANTS. Members of the Society for Healthcare Epidemiology of America (SHEA) Research Network (SRN) METHODS. We administered a 10-item test of CLABSI data comprehension. The primary outcome was percent correct of attempted questions pertaining to the CLABSI data. We also assessed expert perceptions of CLABSI reporting. RESULTS The response rate was 51% (n=67).Among experts, the average proportion of correct responses was 73% (95% confidence interval [CI], 69%-77%). Expert performance on unadjusted data was significantly better than risk-adjusted data (86% [95% CI, 81%-90%] vs 65% [95% CI, 60%-70%]; P<.001). Using a scale of 1 to 100 (0, never reliable; 100, always reliable), experts rated the reliability of CLABSI data as 61. Perceived reliability showed a significant inverse relationship with performance (r=-0.28; P=.03), and as interpretation of data improved, perceptions regarding reliability of those data decreased. Experts identified concerns regarding understanding and applying CLABSI definitions as barriers to care. CONCLUSIONS Significant variability in the interpretation of CLABSI data exists among experts. This finding is likely related to data complexity, particularly with respect to risk-adjusted data. Improvements appear necessary in data sharing and public policy efforts to account for this complexity. Infect Control Hosp Epidemiol 2018;39:71-76.