1823. Signal or Noise? A Comparison of Methods to Identify Outliers in Antimicrobial Use (AU)

Abstract

BackgroundAntimicrobial Stewardship Programs (ASPs) use AU benchmarking data to help identify areas in need of investigation. The high frequency and wide variation in AU make statistical tests frequently significant.MethodsWe compared four statistical methods of analyzing AU data to quantify how often statistically significant outliers occur. We analyzed days of therapy (DOT) per 1,000 days present (dp) from 2017 in medical and surgical adult wards and three NHSN AU antibiotic groups: anti-MRSA agents (anti-MRSA), broad agents for community-onset infections (CO), and broad agents for hospital-onset multidrug-resistant organisms (HO/MDRO). Outliers were defined as follows: (1) Units ≥90th or ≤10th percentiles. (2) Units with Standardized Antimicrobial Administration Ratios (SAARs) outside 95% confidence intervals (CI). (3) Units with observed rates outside 95% CI predicted by a generalized estimating equation (GEE) negative binomial regression model. (4) Units with observed rate outside 95% CI predicted by mixed effects negative binomial regression model with hospital as a random effect. Adjustment in method 2 included hospital teaching status and location type. Methods 3 and 4 included adjustment for teaching status, location type, average age, average hospital length of stay, surgical volume, percent sepsis admissions, and average DRG weight.ResultsFifty-five units and 628,358 dp were included in the 1-year sample. Each method identified both positive and negative outliers. SAAR and GEE methods identified the largest number of outliers; percentiles identified the least (table). The four methods identified different individual units as outliers (figure).Table:Number (%) of Outlier Units Identified Using Four Statistical MethodsGroupAU in DOT/1,000 dp median (IQR)1. Percentile2. SAAR3. GEE model4. Mixed modelAnti-MRSA84 (73–103)10 (18%)42 (76%)30 (55%)14 (26%)CO132 (106–184)10 (18%)50 (91%)22 (40%)14 (26%)HO/MDRO132 (118–151)12 (22%)38 (69%)31 (56%)14 (26%) ConclusionOverly sensitive statistical methods may produce more signals than are clinically meaningful. Investments of ASP resources to investigate such signals may vary widely depending on statistical method used. Additional research is required to develop AU analysis methods with high positive predictive value.Disclosures All authors: No reported disclosures.