Abstract
Water level data obtained from telemetry stations typically contains large number of outliers. Anomaly detection and a data imputation are necessary steps in a data monitoring system. Anomaly data can be detected if its values lie outside of a normal pattern distribution. We developed a median-based statistical outlier detection approach using a sliding window technique. In order to fill anomalies, various interpolation techniques were considered. Our proposed framework exhibited promising results after evaluating with F1-score and root mean square error (RMSE) based on our artificially induced data points. The present system can also be easily applied to various patterns of hydrological time series with diverse choices of internal methods and fine-tuned parameters. Specifically, the Spline interpolation method yielded a superior performance on non-cyclical data while the long short-term memory (LSTM) outperformed other interpolation methods on a distinct tidal data pattern.
Highlights
An anomaly detection is an identification of data points that behave differently from normal patterns, see [1]
We proposed a combination of median absolute deviation (MAD) and the sliding window technique to capture irregular data behaviors in a specific time frame
The median and MAD are explored in the anomaly detection part while the linear interpolation, the spline method and the long short-term memory (LSTM)-based model are utilized for the data filling step
Summary
An anomaly detection is an identification of data points that behave differently from normal patterns, see [1]. This process is essential as anomalous data can indicate changes in typical behaviors or technical malfunctions. In the context of time series, input data are generally either univariate, which is our focus in this work, or a multivariate type. Both statistical and machine learning approaches were regularly applied for detecting anomalies as broadly reviewed in [9,10,11,12]
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
