Detecting Pattern Anomalies in Hydrological Time Series with Weighted Probabilistic Suffix Trees

Yufeng Yu,Qun Zhao,Dingsheng Wan,Huan Liu

doi:10.3390/w12051464

Yufeng Yu, Qun Zhao + Show 2 more

Open Access

https://doi.org/10.3390/w12051464

Copy DOI

Abstract

Anomalous patterns are common phenomena in time series datasets. The presence of anomalous patterns in hydrological data may represent some anomalous hydrometeorological events that are significantly different from others and induce a bias in the decision-making process related to design, operation and management of water resources. Hence, it is necessary to extract those “anomalous” knowledge that can provide valuable and useful information for future hydrological analysis and forecasting from hydrological data. This paper focuses on the problem of detecting anomalous patterns from hydrological time series data, and proposes an effective and accurate anomalous pattern detection approach, TFSAX_wPST, which combines the advantages of the Trend Feature Symbolic Aggregate approximation (TFSAX) and weighted Probabilistic Suffix Tree (wPST). Experiments with different hydrological real-world time series are reported, and the results indicate that the proposed methods are fast and can correctly detect anomalous patterns for hydrological time series analysis, and thus promote the deep analysis and continuous utilization of hydrological time series data.

Highlights

In the era of Big Data, new satellite, space, airborne, shipborne and ground-based remote sensing systems, as well as Internet of Things (IoT) devices, are ubiquitous, producing data rapidly and continuously, which lead to hydrological time series being acquired at a breathless pace, both in size and variety [1,2]
We propose a novel weighted Probabilistic Suffix Tree (wPST) model to better descript and accurately distinguish different time series sequences; we give here a formal definition for pattern anomaly based on wPST to define the detection boundary and detection target for our anomaly detection algorithm
Trend Feature Symbolic Aggregate approximation (TFSAX), we propose a novel TFSAX_wPST algorithm to detect those patterns meet our definition given propose a novel algorithm to detect those patterns that meet our definition within time series

Summary

Introduction

In the era of Big Data, new satellite, space, airborne, shipborne and ground-based remote sensing systems, as well as Internet of Things (IoT) devices, are ubiquitous, producing data rapidly and continuously, which lead to hydrological time series being acquired at a breathless pace, both in size and variety [1,2]. Due to measurement/manual operation errors, instrument failure, changes in natural laws caused by human activities or hydrological evolution, there is a large number of “anomalous” data in hydrological time series. Those “anomalous” data will significantly affect the models related to flood forecasting and hydrological analysis, and lead to potentially incomplete or inaccurate results [3]. Detecting those “anomalies” in hydrological datasets is becoming an important and urgent task for hydrology and information researchers [4].

Results

Discussion

Conclusion