Detecting reagent lot shifts using proficiency testing data

Abstract

Clinically significant systematic analytical shifts can evade detection despite between-lot reagent verification, quality control and proficiency testing systems practiced by most laboratories. Through numerical simulations, we present two methods to determine whether there has been a shift in the proficiency testing peer group of interest, peer group i, using the measurements from peer group i and J other peer groups. In method 1 ('group mean'), the distance of peer group i from the mean of the other J peer groups is used to determine whether a shift occurs. In method 2 ('inter-peer group' method), the distances of peer group i from each of the means of the other J peer groups are used to determine whether a shift has occurred. The power of detection for both methods increases with the magnitude of systematic shift, the number of peer groups, the number of laboratories within the peer groups and the proportion of laboratories within the affected peer group, and a smaller analytical imprecision. When the number of peer groups is low, the power of detection for the group mean method is comparable to the inter-peer group method, using the m= 1 criterion (a single inter-peer group comparison that exceeds the control limit is considered a flag). At larger peer groups, the inter-peer group method using the same (m= 1) criterion outperforms the group mean method. The proposed methods can elevate the professional role of the proficiency testing program to that of monitoring the peer group method on top of the performance of individual laboratories.