Anomalies In Time Series Data Research Articles

DeepQC: A deep learning system for automatic quality control of in-situ soil moisture sensor time series data

Amidst changing climate, real-time soil moisture monitoring is vital for the development of in-season decision support tools to help farmers manage weather-related risks in agriculture. Precision Sustainable Agriculture (PSA) recently established a real-time soil moisture monitoring network across the central, Midwest, and eastern U.S., but continuous field-scale sensor observations often come with data gaps and anomalies. To maintain the data quality and continuity needed for developing decision tools, a quality control system is necessary.The International Soil Moisture Network (ISMN) introduced the Flagit module for anomaly detection in soil moisture time series observations. However, under certain conditions, Flagit's threshold and spectral based quality control approaches may underperform in identifying anomalies. Recently, deep learning methods have been successfully applied to detect time series anomalies in time series data in various disciplines. However, their use in agriculture for anomaly detection in time series datasets has not been yet investigated. This study focuses on developing a Bi-directional Long Short-Term Memory (LSTM) model, referred to as DeepQC, to identify anomalies in soil moisture time series data. Manual flagged PSA observations were used for training, validation, and testing the model, following an 80:10:10 split. The study then compared the DeepQC and Flagit-based estimates to assess their relative performance.Flagit correctly flagged 95.8 % of the correct observations and 50.3 % of the anomaly observations, indicating its limitations in identifying anomalies, particularly at sites consists of more than 30 % anomalies. On the other hand, the DeepQC correctly flagged 89.8 % of the correct observations and 99.5 % of the anomalies, with overall accuracy of 95.6 %, in significantly less time, demonstrating its superiority over Flagit approach. Importantly, the performance of the DeepQC remained consistent regardless of the number of anomalies in site observations. Given the promising results obtained with the DeepQC, future studies will focus on implementing and finetuning this model on national and global soil moisture networks.

Обзор open-source библиотек для решения задач прогнозирования временных рядов

<p>An overview of various open-source Python libraries for time series analysis and forecasting is presented. It covers such tools as Prophet, Kats, Merlion, as well as ARIMA, LSTM algorithms, which allow to study seasonality, trends and anomalies in time series data. The capabilities of each library, their advantages and applications in time series data analysis are discussed in detail.</p>

Using the method of nonlinear recursive analysis for detecting DDOS anoma-lies in time series data

This research endeavors to address this gap by determining a qualitative char-acteristic for server network traffic and use it to construct the corresponding recur-rence plot (RP). The goal of this study is to develop and assess a novel technique based on nonlinear recursive analysis to detect Distributed Denial of Service (DDoS) anomalies in network traffic time series data. With the increasing frequency of DDoS attacks on modern digital infrastructures, there is a pressing need for more efficient and accurate detection methods. There has been some attempts to apply nonlinear analysis to network traffic [2-4], but those studies lack critical steps in determining parameters for embedding space dimension and delay time . More recent studies have explored machine learning and deep learning approaches [7], which offer improved accuracy but can be computationally intensive and require extensive training data. Despite the advance-ments, there remains a need for a method that is both accurate and efficient, espe-cially in real-time detection scenarios. The researchers employed nonlinear recursive analysis by estimating RQA pa-rameters and determining a qualitative characteristic of data points of DDoS attack contained in CIC-IDS2017 dataset. A technique for determining hidden information for this series and its use for constructing the corresponding recurrence diagram (RP) at the points of information retrieval are described. It is shown that the use of RP has significant drawbacks associated with the visualization of information on a computer monitor screen, so another way of research is proposed - the calculation of numerical indicators of RP The given calculated RP indicators made it possible to typify the received data and determine the type, which was named "DDOS-RP", which makes it possible to distinguish some types of DoS/DDoS type attacks. The study concludes by recom-mending further exploration of this method in diverse network environments and against more complex DDoS attack patterns.

Enhancing Time Series Anomaly Detection: A Hybrid Model Fusion Approach

Exploring and Identifying Anomalies in time-series data is very crucial in today’s world revolve around data. These data are being used to make important decisions; hence, an efficient and reliable anomaly detection system should be involved in this process to ensure that the best decisions are being made. The paper explores other types of anomalies and proposes efficient detection methods which can be used. Anomalies are patterns that deviate from usual expected behavior. These can come from system failures or unexpected activity. This research paper explores the vulnerabilities of commonly used anomaly detection algorithms such as the Z-Score and static threshold approach. Each method used in this paper has its unique capabilities and limitations. These methods range from using statistical methods and machine learning approaches to detecting anomalies in a time-series dataset. Furthermore, this paper explores other open-source libraries that can be used to detect anomalies, such as Greykite and Prophet Python library. This paper serves as a good source for anyone new to anomaly detection and willing to explore.

Anomaly detection in time-series data using evolutionary neural architecture search with non-differentiable functions

Deep neural networks have become the benchmark in diverse fields such as energy consumption forecasting, speech recognition, and anomaly detection, owing to their ability to efficiently process and analyse data. However, they face challenges in managing the complexity and variability in time series data, often leading to increased model complexity and prolonged search duration during parameter tuning. This paper proposes a novel anomaly detection approach through evolutionary neural architecture search (AD-ENAS), which is specifically designed for time series data. The proposed approach focuses on the search for the optimal and minimal neural network architecture. The AD-ENAS method consists of two main phases: architecture evolution and weight adjustment. The architecture evolution phase highlights the importance of neural network architecture by evaluating the fitness of each network agent using shared weight values. Subsequently, the convolutional matrix adaptation technique is used in the next phase for optimal weight adjustment of the neural network. The proposed AD-ENAS method operates without relying on differentiable functions, thus expanding the scope of neural network design beyond traditional backpropagation-based approaches. Various non-differentiable loss functions are explored to facilitate effective architecture search and weight adjustment. Comparative experiments are conducted with five baseline anomaly detection methods on three well-known datasets from reputable sources such as NASA SMAP, NASA MSL and Yahoo S5-A1. The results demonstrate that the AD-ENAS approach effectively evolves neural network architectures, outperforming baseline methods with F1 scores across the three datasets (MSL: 0.942, SMAP: 0.961, Yahoo S5-A1: 0.988) with non-differentiable loss functions, showcasing its efficacy in detecting anomalies in time series data.

Graph spatiotemporal process for multivariate time series anomaly detection with missing values

The detection of anomalies in multivariate time series data is crucial for various practical applications, including smart power grids, traffic flow forecasting, and industrial process control. However, real-world time series data is usually not well-structured, posting significant challenges to existing approaches: (1) The existence of missing values in multivariate time series data along variable and time dimensions hinders the effective modeling of interwoven spatial and temporal dependencies, resulting in important patterns being overlooked during model training; (2) Anomaly scoring with irregularly-sampled observations is less explored, making it difficult to use existing detectors for multivariate series without fully-observed values. In this work, we introduce a novel framework called GST-Pro, which utilizes a graph spatiotemporal process and anomaly scorer to tackle the aforementioned challenges in detecting anomalies on irregularly-sampled multivariate time series. Our approach comprises two main components. First, we propose a graph spatiotemporal process based on neural controlled differential equations. This process enables effective modeling of multivariate time series from both spatial and temporal perspectives, even when the data contains missing values. Second, we present a novel distribution-based anomaly scoring mechanism that alleviates the reliance on complete uniform observations. By analyzing the predictions of the graph spatiotemporal process, our approach allows anomalies to be easily detected. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods, regardless of whether there are missing values present in the data. Our code is available: https://github.com/huankoh/GST-Pro.

Anomaly detection and prediction evaluation for discrete nonlinear dynamical systems

Anomalies in dynamical systems mostly occur as deviations between measurement and prediction. Current anomaly detection methods in multivariate time series often require prior clustering, training data, or cannot distinguish local and global anomalies. Furthermore, no generalized metric exists to evaluate and compare different prediction functions regarding their amount of anomalous behavior. We propose a novel methodology to detect local and global anomalies in time series data of dynamical systems. For this purpose, a theoretical density distribution is derived assuming that only noise conceals the time series. If the theoretical and the empirical density distribution yield significantly different entropies, an anomaly is assumed. For a local anomaly detection, the Mahalanobis distance using the theoretical noise distribution’s covariance is applied to evaluate sequences of predictions and measurements. In addition, the Wasserstein metric enables a comparison of predictions using the distance between the noise and empirical distribution as a measure for selecting the best prediction function. The proposed method performs well on nonlinear time series such as logistic growth and enables a useful selection of a prediction model for satellite orbits. Thus, the proposed method improves anomaly detection in time series and model selection for nonlinear systems.

An Experimental Evaluation of Anomaly Detection in Time Series

Anomaly detection in time series data has been studied for decades in both statistics and computer science. Various algorithms have been proposed for different scenarios, such as fraud detection, environmental monitoring, manufacturing, and healthcare. However, there is a lack of comparative evaluation of these state-of-the-art approaches, especially in the same test environment and with the same benchmark, making it difficult for users to select an appropriate method for real-world applications. In this paper, we present a taxonomy of anomaly detection methods based on the main features, i.e., data dimension, processing technique, and anomaly type and six inner classes. We perform systematic intra- and inter-class comparisons of seventeen state-of-the-art algorithms on real and synthetic datasets with a point metric commonly used in classification problems and a range metric specifically designed for subsequence anomalies in time series data. We analyze the properties of these algorithms and test them in terms of effectiveness, efficiency, and robustness to anomaly rates, data sizes, number of dimensions, anomaly patterns, and threshold settings. We also test their performance in different use cases. Finally, we provide a practical guide for detecting anomalies in time series and discussions.

CL-TAD: A Contrastive-Learning-Based Method for Time Series Anomaly Detection

Anomaly detection has gained increasing attention in recent years, but detecting anomalies in time series data remains challenging due to temporal dynamics, label scarcity, and data diversity in real-world applications. To address these challenges, we introduce a novel method for anomaly detection in time series data, called CL-TAD (Contrastive-Learning-based method for Times series Anomaly Detection), which employs a contrastive-learning-based representation learning technique. Inspired by the successes of reconstruction-based approaches and contrastive learning approaches, the proposed method seeks to leverage these approaches for time series anomaly detection. The CL-TAD method is comprised of two main components: positive sample generation and contrastive-learning-based representation learning. The former component generates positive samples by trying to reconstruct the original data from masked samples. These positive samples, in conjunction with the original data, serve as input for the contrastive-learning-based representation learning component. The representations of input original data and their masked data are used to detect anomalies later on. Experimental results have demonstrated that the CL-TAD method achieved the best performance on five datasets out of nine benchmark datasets over 10 other recent methods. By leveraging the reconstruction learning and contrastive learning techniques, our method offers a promising solution for effectively detecting anomalies in time series data by handling the issues raised by label scarcity and data diversity, delivering high performance.

Detection of Anomalies in Time Series Data

The detection of Anomaly in time series data has gained a significant amount of popularity due to its indulgence in various domains of industry such as healthcare, the financial sector, industrial services, and other several reasons. This paper represents an elaborate survey of varied anomaly detection techniques that are tested on time series data. The aim of the paper is to identify various anomalous outlines in the dataset through a number of different approaches, algorithms, and techniques. At the beginning of the paper, the importance of detecting anomalies in real-world scenarios is explained followed by describing the efficient techniques capable of dealing with it. The paper covers all the aspects of anomaly detection starting from traditional analytical and statistical models to modern-day advanced machine learning methods. The moving averages, exponential smoothening, and process control which are considered to be the traditional approaches for detecting anomalies in Machine learning-based models such as one-class Support Vector Machine (SVM), deep learning-based models, and isolation forests are the distinct topics that are covered in the paper. Moving ahead, the paper revolves around addressing different challenges and limitations associated with the detection of anomalies in time series data. The challenges circle around topics such as selecting appropriate evaluation metrics, addressing class imbalance, handling high-dimensional data, and how to deal with concept drift. Through a systematic analysis of the existing works, this paper aims to provide data scientists, practitioners, and researchers with summarized information for understanding the background of anomaly detection methods. Finally, this paper delves into a number of methodologies used in advancing anomaly detection in time series data and justifying their impact on real-world applications.

Assessing the Performance of Transformer for Time Series Anomaly Detection

This study aims to assess the effectiveness of the Transformer-based reconstruction approach for detecting anomalies in time series data. The reconstruction error-based anomaly detection method was applied to both multivariate time series from NASA SMAP/MSL and univariate time series from UCR. Four deep learning models, including Transformer, Dilated CNN, LSTM, and MLP, were compared in terms of their ability to reconstruct input data. Dilated CNN outperformed the other models in almost all experimental results, achieving a 25% higher score than Transformer on the UCR dataset when trained with random masking, and a 60% higher score when trained with middle masking. These results suggest that the Transformer did not perform as well as expected for anomaly detection based on time series reconstruction errors, and its inferiority to Dilated CNN may be attributed to the characteristics of the time series and the limited training data. Future research should focus on developing Transformer models that can better capture the properties of time series data and investigate the relationship between the model’s performance, data volume, and model complexity.

Singular spectral and control chart analysis of soil radon and thoron time series for forecasting seismic activities

Anomalies in soil radon (222Rn) and thoron (220Rn) time series data (TSD) have been reported in many studies prior to earthquakes, suggesting both radioisotopes may serve as earthquake precursors. In the present study, periodic oscillations are being observed in measured time series of soil radon, thoron, and meteorological parameters viz., pressure, temperature, and humidity. Each time series consists of 15,692 measurements taken over a 40-min sampling period. Singular spectrum analyses (SSA) have been used to identify and split the periodic and aperiodic components. Each of the time series has been tested for the normal distribution using Kolmogorov-Smirnov (K–S) test, the Anderson-Darling (A-D) test, and the Shapiro-Wilk (S–W) test. Results show the non-uniformity of the soil radon and thoron time series.Periodicities in each of the TSD have been studied using periodograms. Control charts have been drawn for radon and thoron time series data (TTSD) to check if any anomalous behavior exists in the time series. Anomalies in TSD have been analyzed for the seismic events that occurred during the study time. The finding of study show that TTSD exhibited unusual behavior prior to the occurrence of the earthquakes.

어텐션 메커니즘을 활용한 철도차량 공기압축기의 이상탐지

Condition-based diagnostic analysis of air compressors of railway vehicles is essential for passenger safety and maintenance cost reduction. When conventional long short-term memory (LSTM) autoencoders are used for detecting anomalies in time series data, the input data cannot be reconstructed appropriately when the sequence length is large. Recently, the attention mechanism was proposed to solve this problem encountered in LSTM autoencoders. In the current study, the anomaly detection performance of railway vehicle air compressors was improved using the attention mechanism. The data reconstruction performance of the LSTM autoencoder model was compared with that of the LSTM autoencoder model with the attention mechanism and the transformer model comprising only the attention mechanism by using the sensor data of an air compressor of the Airport Railroad Express train in Seoul, Korea. For artificially generated abnormal data, three anomaly scenarios were successfully detected using the transformer model, which exhibited the best data reconstruction performance among the three models. The results confirmed that transformer models with attention mechanisms can detect anomalies in the air compressors of railway vehicles in a timely manner.

M3GAN: A masking strategy with a mutable filter for multidimensional anomaly detection

With the advent of the big data era, the detection of anomalies in time series data, especially multidimensional time series data, has received a great deal of attention from researchers in many fields, e.g., industry, medicine, and aerospace. Multidimensional time series anomaly detection (MTSAD) methods aim to discover anomalies. They play an important early warning role in protecting people from possible property damage and personal injury. Many current GAN-based anomaly detection methods suffer from poor robustness, poor generalization, and low accuracy. The M2GAN (a GAN framework based on a masking strategy for multidimensional anomaly detection) proposed in this paper uses the masking method to diversify the data and improve the robustness of the model. This paper further proposes the M3GAN (M2GAN with a mutable filter), which can utilize the different characteristics of different filters to select a suitable model for the features of the data, thus enhancing the generalization ability and improving the accuracy. The experiments prove the effectiveness of the proposed method.

Is It Worth It? Comparing Six Deep and Classical Methods for Unsupervised Anomaly Detection in Time Series

Detecting anomalies in time series data is important in a variety of fields, including system monitoring, healthcare and cybersecurity. While the abundance of available methods makes it difficult to choose the most appropriate method for a given application, each method has its strengths in detecting certain types of anomalies. In this study, we compare six unsupervised anomaly detection methods of varying complexity to determine whether more complex methods generally perform better and if certain methods are better suited to certain types of anomalies. We evaluated the methods using the UCR anomaly archive, a recent benchmark dataset for anomaly detection. We analyzed the results on a dataset and anomaly-type level after adjusting the necessary hyperparameters for each method. Additionally, we assessed the ability of each method to incorporate prior knowledge about anomalies and examined the differences between point-wise and sequence-wise features. Our experiments show that classical machine learning methods generally outperform deep learning methods across a range of anomaly types.

CNTS: Cooperative Network for Time Series

The use of deep learning techniques in detecting anomalies in time series data has been an active area of research with a long history of development and a variety of approaches. In particular, reconstruction-based unsupervised anomaly detection methods have gained popularity due to their intuitive assumptions and low computational requirements. However, these methods are often susceptible to outliers and do not effectively model anomalies, leading to suboptimal results. This paper presents a novel approach for unsupervised anomaly detection, called the Cooperative Network Time Series (CNTS) approach. The CNTS system consists of two components: a detector and a reconstructor. The detector is responsible for directly detecting anomalies, while the reconstructor provides reconstruction information to the detector and updates its learning based on anomalous information received from the detector. The central aspect of CNTS is a multi-objective optimization problem, which is solved through a cooperative solution strategy. Experiments on three real-world datasets demonstrate the state-of-the-art performance of CNTS and confirm the effectiveness of the detector and reconstructor. The source code for this study is publicly available on GitHub <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><i>a</i></sup> .

MTEDS: Multivariant Time Series-Based Encoder-Decoder System for Anomaly Detection.

Intrusion detection systems examine the computer or network for potential security vulnerabilities. Time series data is real-valued. The nature of the data influences the type of anomaly detection. As a result, network anomalies are operations that deviate from the norm. These anomalies can cause a wide range of device malfunctions, overloads, and network intrusions. As a result of this, the network's normal operation and services will be disrupted. The paper proposes a new multi-variant time series-based encoder-decoder system for dealing with anomalies in time series data with multiple variables. As a result, to update network weights via backpropagation, a radical loss function is defined. Anomaly scores are used to evaluate performance. The anomaly score, according to the findings, is more stable and traceable, with fewer false positives and negatives. The proposed system's efficiency is compared to three existing approaches: Multiscaling Convolutional Recurrent Encoder-Decoder, Autoregressive Moving Average, and Long Short Term Medium-Encoder-Decoder. The results show that the proposed technique has the highest precision of 1 for a noise level of 0.2. Thus, it demonstrates greater precision for noise factors of 0.25, 0.3, 0.35, and 0.4, and its effectiveness.

Anomaly detection-based condition monitoring

The impact of an anomaly is domain-dependent. In a dataset of network activities, an anomaly can imply an intrusion attack. Other objectives of anomaly detection are industrial damage detection, data leak prevention, identifying security vulnerabilities or military surveillance. Anomalies are observations or a sequence of observations in which the distribution deviates remarkably from the general distribution of the whole dataset. A large majority of the dataset consists of normal (healthy) data points. The anomalies form only a very small part of the dataset. Anomaly detection is the technique used to find these observations and its methods are specific to the type of data. While there is a wide spectrum of anomaly detection approaches available today, it becomes increasingly difficult to keep track of all the techniques. In fact, it is not clear which of the three categories of detection methods, ie statistical approaches, machine learning approaches or deep learning approaches, is more appropriate to detect anomalies in time-series data, which are mainly used in industry. A typical industrial device has multi-dimensional characteristics. It is possible to measure voltage, current, active power, vibrations, rotational speed, temperature, pressure difference, etc, on such a device. Early detection of the anomalous behaviour of industrial devices can help reduce or prevent serious damage, which could lead to significant financial loss. This paper presents a summary of the methods used to detect anomalies in condition monitoring applications.

MTV: Visual Analytics for Detecting, Investigating, and Annotating Anomalies in Multivariate Time Series

Detecting anomalies in time-varying multivariate data is crucial in various industries for the predictive maintenance of equipment. Numerous machine learning (ML) algorithms have been proposed to support automated anomaly identification. However, a significant amount of human knowledge is still required to interpret, analyze, and calibrate the results of automated analysis. This paper investigates current practices used to detect and investigate anomalies in time series data in industrial contexts and identifies corresponding needs. Through iterative design and working with nine experts from two industry domains (aerospace and energy), we characterize six design elements required for a successful visualization system that supports effective detection, investigation, and annotation of time series anomalies. We summarize an ideal human-AI collaboration workflow that streamlines the process and supports efficient and collaborative analysis. We introduce MTV (MultivariateTime SeriesVisualization), a visual analytics system to support such workflow. The system incorporates a set of novel visualization and interaction designs to support multi-faceted time series exploration, efficient in-situ anomaly annotation, and insight communication. Two user studies, one with 6 spacecraft experts (with routine anomaly analysis tasks) and one with 25 general end-users (without such tasks), are conducted to demonstrate the effectiveness and usefulness of MTV.

Methods of Generating Time Series Image for detecting Anomalies in Time Series data

To detect anomalies in time series data, statistical techniques such as PCA and autoencoder are used, or anomalies are detected based on deep learning models such as RNN. It is difficult to expect good performance only with simple statistical techniques or RNN-based deep learning models because the environment and causes in which anomalies are recorded are not simple and various variables affect them. In this paper, we proposed a method of detecting anomalies using CNN-based deep learning model, a binary classification model of representative images, by imaging time series data. RP, GASF, GADF, and MAF algorithms were used as methods for imaging. All of time series image-based models showed equal or higher accuracy than conventional LSTM-based models, and among imaging-based CNN models, the method of imaging with MTF algorithm derived the highest accuracy.