Evaluating Real-Time Event Detection Algorithms Using Synthetic Data

Abstract

Contamination warning systems (CWSs) are designed to identify intentional or accidental contamination of drinking water distribution systems, using strategically placed water quality sensors, and real-time algorithms that distinguish anomalous water quality characteristics from noisy sensor data. These real-time algorithms are called event detection (ED) algorithms. While ED systems are available commercially, they have never been objectively evaluated due to a lack of data from contamination events in real distribution systems. Until ED performance is critically evaluated, water utilities will likely remain skeptical of their value in protecting the quality of potable drinking water. This paper presents an approach for generating simulated data appropriate for the evaluation of ED algorithms. The approach uses network hydraulic and water quality models, incorporating realistic contaminant reaction kinetics and important sources of randomness, such as random water demands, chlorine source boundary conditions, and contaminant source location and characteristics. The authors employ a recently developed multi-species extension of the well known EPANET distribution system hydraulic and water quality model, coupled with random process models of water usage and chlorine source concentration, both estimated from field data. A very simple ED algorithm is used to illustrate how the simulated data can be used to evaluate ED algorithms. The advantage of using such an event detection algorithm for illustrative purposes is that its design can be fully described and its performance controlled. This algorithm is evaluated against a simulated event data set where an organophosphate compound is added that forms an Oxon via oxidation by free chlorine. Results are presented in the form of receiver operating characteristics (ROC) curves — graphs of the hit rate versus the false positive rate.