Performance Metrics for Surveillance Schemes

Abstract

ABSTRACT We review various statistical performance metrics that have been used with prospective surveillance schemes. We consider scenarios under which the metrics are most useful and discuss some of their advantages and disadvantages. A contrast is made between the approaches and metrics used in industrial process monitoring and those used in public health surveillance. The in-control average time between signal events (ATBSE) and the in-control average signal event length (ASEL) are introduced as performance metrics that are useful when a monitoring procedure is not reset to its initial state after a signal. We give particular attention to the recurrence interval, defined as the fixed length of time (measured in number of time periods) for which the expected number of false alarms is one. The recurrence interval is used in public health surveillance whereas in-control time-to-signal measures are used in industrial statistical process control. We compare the recurrence interval and measures based on the time-to-signal properties for the temporal monitoring case using exponentially weighted moving average (EWMA) charts, cumulative sum (CUSUM) charts, and Markov dependent signaling processes. We recommend that measures based on the time-to-signal properties be used when possible to evaluate the performance of surveillance schemes for ongoing monitoring.