Unsupervised anomaly detection in hourly water demand data using an asymmetric encoder–decoder model

Abstract

Water demand forecasting (WDF) is essential for the design and optimal management of water distribution systems (WDS). Historical water demand data contribute significantly to WDF. Yet the obtained water demand data contain anomalies on occasions due to failures in WDS or monitoring systems. The contaminated water demand data are invalid to describe the actual water demand, thus degrading the performance of WDF models. However, the importance of anomaly detection in water demand data is underestimated, or at least not explicitly described in many published papers. To fill the gap, we propose an unsupervised anomaly detection method based on an asymmetric encoder–decoder (asyED) model. Different from the symmetric structure of a traditional autoencoder where a signal is reproduced from itself, asyED is asymmetric where a signal is reproduced from the upstream and downstream information of the signal, while the signal itself does not participate in the reconstruction. In light of this feature, asyED is powerful in identifying anomalies. The proposed method is employed to detect anomalies in hourly water demand data which exhibit trend and seasonality. The results show the superiority of the proposed method over the other four commonly used anomaly detection methods: Z-score, isolation forest, local outlier factor, and seasonal hybrid ESD (Extreme Studentized Deviate test). • An anomaly detection method based on an asymmetric encoder-decoder model is proposed. • The method applies to hourly water demand data which exhibit trend and seasonality. • A cluster-based metric is proposed for model evaluation. • The metric considers the continuity mechanism of anomalies found in the data. • Measures to quantify the strengths of different components in the data are proposed.