A data quality assessment framework for drinking water distribution system water quality time series datasets

Abstract

Abstract The derivation of information from monitoring drinking water quality at high spatiotemporal resolution as it passes through complex, ageing distribution systems is limited by the variable data quality from the sensitive scientific instruments necessary. A framework is developed to overcome this. Application to three extensive real-world datasets, consisting of 92 multi-parameter water quality time series of data taken from different hardware configurations, shows how the algorithms can provide quality-assured data and actionable insight. Focussing on turbidity and chlorine, the framework consists of three steps to bridge the gap between data and information; firstly, an automated rule-based data quality assessment is developed and applied to each water quality sensor, then, cross-correlation is used to determine spatiotemporal relationships and finally, spatiotemporal information enables multi-sensor data quality validation. The framework provides a method to achieve automated data quality assurance, applicable to both historic and online datasets, such that insight and actionable insight can be gained to help ensure the supply of safe, clean drinking water to protect public health.