Reply to: The use of statistical process control methods in monitoring clinical performance

Abstract

To the Editor: I couldn’t agree more with Dr Morton that the system within which monitoring is implemented is fundamental. Quality improvement in clinical practice cannot simply be realized by mechanically applying correct techniques. No techniques, however sophisticated, can survive an environment where such tools are used to punish rather than to improve. Concerning SPC in clinical practice, I wish to emphasize that one cannot have both ‘greater sensitivity and specificity’. There is an inherent trade-off between these two fundamental properties of any monitoring scheme. The amount of tradeoff between sensitivity and specificity that is acceptable clearly depends on the nature of what is being monitored. For example, if a procedure entails life-threatening complications, one would want a highly sensitive scheme and be prepared to tolerate low specificity (high false alarm rate). Ignoring this trade-off will result in a poorly designed scheme that will quickly discredit this technique in the eyes of users. Already, I have seen unthinking use of conventions such as alpha error of 5% and beta of 90% in the SPC in health care literature. These conventional error rates are obviously ‘borrowed’ from hypotheses testing in the experimental or clinical trial setting, and have no place at all in the monitoring setting. Every monitoring situation deserves careful consideration of how much sensitivity is desired and how much false alarm can be tolerated. For CUSUM, there is a simple and intuitive way of specifying these [1]. Instead of alpha and beta error rates, we specify the in-control (IC) and out-of-control (OC) average run length (ARL) for the design of the CUSUM chart in monitoring performance. The IC-ARL is the average number of consecutive procedures performed during a period of acceptable performance and the CUSUM chart signal indicates an adverse trend in performance. This is a measure of the propensity to raise a false alarm. The OC-ARL is the average number of procedures performed before the CUSUM chart signal during a period when an individual is performing with an unacceptable failure rate. This is a measure of sensitivity of the scheme.