A-196 Using Aggregated Proficiency Testing Results to Identify Systematic Error

Abstract

Abstract Background Proficiency testing (PT) should identify systematic errors, which are likely to recur. The Clinical Laboratory Improvement Amendments of 1988 (CLIA) acceptance limits (ALs) include 3SD and concentration limits. We investigated the ability of PT to detect systematic error, especially as affected by the AL type. Methods Ungradable, duplicate, and irregular scores were removed from 2008 to 2018 CLIA laboratory PT results. For analytes that had at least 200 events and at least 200 participating laboratories, we calculated the overall sample miss rate, the unsatisfactory event rate, i.e., less than 80, and event rates for Score 100 to 0. We used paired t-tests and the Wilcoxon Signed-Rank test to compare miss rates and unsatisfactory event rates between short- and long-term participants. We used the binomial distribution to model the expected event scores under the assumption that all errors may be independent (random); see Figure. For each analyte and each of the possible PT scores, we compared observed event scores with their expected values as a ratio. Results 40,596 laboratories produced 15,140,128 event scores for 75 analytes. The distribution of event scores was over-weighted with higher-order event errors (Score 0 - Score 60) compared to the predicted distribution. Miss rates and unsatisfactory rates were significantly higher for short-term laboratories. Plotting the log ratio of observed versus expected rates for event scores showed that the degree of systematic effect increased progressively for each analyte. The magnitude of the effect was less for 3SD ALs. Conclusions In an event, PT errors are often dependent. Concentration and 3SD ALs were similarly sensitive to detect systematic error; both were more sensitive than ALs for qualitative or dilution assays. Expressing systematic error using PT data could help to identify remediable analytical issues.