Time Series Of Anomalies Research Articles

A novel daily runoff forecasting model based on global features and enhanced local feature interpretation

The development of artificial intelligence has introduced new perspectives to the field of hydrological forecasting. However, there is still a lack of research on efficiently identifying the physical characteristics of runoff sequences and developing prediction models that consider global and local sequence features. This study proposes a parallel computing prediction model called IMCAEN (Integrated Multi-Feature Causal Dilated Convolutional Attention Encoder Network) to address these issues. Unlike existing models, this model can monitor fluctuations and anomalies in time series. Incorporating the CDC-AA (Causal Dilated Convolutional Network with Aggregation Attention) and encoder structure captures both local sequence variations and global abrupt anomalies, allowing for comprehensive attention to sequence features. When predicting runoff data from three different hydrological conditions, the IMCAEN model achieved NSEC (Nash-Sutcliffe Efficiency Coefficient) values of 0.98, 0.97, and 0.88, respectively, and outperformed benchmark models in other evaluation indicators as well. Given the opacity of the feature distribution process in AI models, SHAP (SHapleyAdditive exPlanations) analysis and spatial expression of feature distribution are used to assess the contribution of each feature variable to the long-term trend of runoff and to verify the distribution of features trained in each module. The proposed IMCAEN model efficiently captures local and global information in the runoff evolution process through parallel computing and shared features, enabling accurate runoff forecasting and providing critical references for timely warnings and predictions.

An Updated Analysis of Long-Term Sea Level Change in China Seas and Their Adjacent Ocean with T/P: Jason-1/2/3 from 1993 to 2022

This study analyzes sea level changes (SLCs) in China seas and their adjacent ocean (CSO) using data from the TOPEX/Poseidon and Jason-1/2/3 satellite altimetry missions from 1993 to 2022. A 30-year time series of sea level anomalies (SLAs) is established, with trends, spatial distribution, and periodicities analyzed through least squares linear fitting, Kriging interpolation, and wavelet analysis. The average yearly sea level rise in the CSO is 3.87 mm, with specific rates of 4.15 mm/yr in the Bohai Sea, 3.96 mm/yr in the Yellow Sea, 3.54 mm/yr in the East China Sea, and 4.09 mm/yr in the South China Sea. This study examines the spatiotemporal variations in SLAs and identifies an annual primary cycle, along with a new periodicity of 11 years. Utilizing 30 years of satellite observation data, particularly the newer Jason-3 satellite data, this reanalysis reveals new findings related to cycles. Overall, the research updates previous studies and provides valuable insights for further investigations into China’s marine environment.

Feature Bagging with Nested Rotations (FBNR) for anomaly detection in multivariate time series

Detecting anomalies in multivariate time series poses a significant challenge across various domains. The infrequent occurrence of anomalies in real-world data, as well as the lack of a large number of annotated samples, makes it a complex task for classification algorithms. Deep Neural Network approaches, based on Long Short-Term Memory (LSTMs), Autoencoders, and Variational Autoencoders (VAEs), among others, prove effective with handling imbalanced data. However, the same does not follow when such algorithms are applied on multivariate time-series, as their performance degrades significantly. Our main hypothesis is that the above is due to anomalies stemming from a small subset of the feature set. To mitigate the above issues in the multivariate setting, we propose forming an ensemble of base models by combining different feature selection and transformation techniques. The proposed processing pipeline includes applying a Feature Bagging techniques on multiple individual models, which considers separate feature subsets for each specific model. These subsets are then partitioned and transformed using multiple nested rotations derived from Principal Component Analysis (PCA). This approach aims to identify anomalies that arise from only a small portion of the feature set while also introduces diversity by transforming the subspaces. Each model provides an anomaly score, which are then aggregated, via an unsupervised decision fusion model. A semi-supervised fusion model was also explored, in which a Logistic Regressor was applied on the individual model outputs. The proposed methodology is evaluated on the Skoltech Anomaly Benchmark (SKAB), containing multivariate time series related to water flow in a closed circuit, as well as the Server Machine Dataset (SMD), which was collected from a large Internet company. The experimental results reveal that the proposed ensemble technique surpasses state-of-the-art algorithms. The unsupervised approach demonstrated a performance improvement of 2% for SKAB and 3% for SMD, compared to the baseline models. In the semi-supervised approach, the proposed method achieved a minimum of 10% improvement in terms of anomaly detection accuracy.

Machine learning-based real-time anomaly detection using data pre-processing in the telemetry of server farms

Fast and accurate anomaly detection is critical in telemetry systems because it helps operators take appropriate actions in response to abnormal behaviours. However, recent techniques are accurate but not fast enough to deal with real-time data. There is a need to reduce the anomaly detection time, which motivates us to propose two new algorithms called AnDePeD (Anomaly Detector on Periodic Data) and AnDePed Pro. The novelty of the proposed algorithms lies in exploiting the periodic nature of data in anomaly detection. Our proposed algorithms apply a variational mode decomposition technique to find and extract periodic components from the original data before using Long Short-Term Memory neural networks to detect anomalies in the remainder time series. Furthermore, our methods include advanced techniques to eliminate prediction errors and automatically tune operational parameters. Extensive numerical results show that the proposed algorithms achieve comparable performance in terms of Precision, Recall, F-score, and MCC metrics while outperforming most of the state-of-the-art anomaly detection approaches in terms of initialisation delay and detection delay, which is favourable for practical applications.

CARROT: Simultaneous prediction of anomalies from groups of correlated cryptocurrency trends

Cryptocurrencies are virtual currencies that exploit cryptography to perform secure financial transactions. They gained widespread popularity in recent years due to their decentralized nature, (pseudo-)anonymity, and ability to facilitate cross-border transactions without the need for intermediaries. However, their price on the market exhibits a huge volatility, that makes them prone to market anomalies. Therefore, predicting anomalies in cryptocurrency time series can be considered an important task for financial institutions, traders, and investors, to maximize their profit or minimize losses.In this paper, we propose a novel approach for predicting anomalies in cryptocurrency time series by exploiting temporal correlations among different cryptocurrencies. Our approach, called CARROT, is based on the idea that groups of cryptocurrencies exhibit similar trends, possibly due to common influencing factors. CARROT analyzes the temporal correlation between different cryptocurrencies, and identifies clusters showing similar patterns that can be useful for gaining insights into future anomalies. Subsequently, CARROT exploits multiple (i.e., one for each cluster) multi-target LSTM models to predict anomalies.Our experiments, performed on a dataset of 17 cryptocurrencies, proved that CARROT outperforms single-target LSTM models of up to 20%, as well as other approaches based on neural networks, i.e., MLP and CNN, in terms of macro F1-score. Therefore, the proposed approach can be considered as a promising tool for predicting anomalies in cryptocurrency time series data and can potentially be used to improve risk management and trading strategies in the cryptocurrency market.

A General Framework for the Assessment of Detectors of Anomalies in Time Series

A General Framework for the Assessment of Detectors of Anomalies in Time Series

Industrial Environmental Impact Assessment Method Based on Detection of Complex Anomalies in Time Series

Industrial Environmental Impact Assessment Method Based on Detection of Complex Anomalies in Time Series

Enhancing the temporal resolution of water levels from altimetry using D-InSAR: A case study of 10 Swedish Lakes

Lakes provide societies and natural ecosystems with valuable services such as freshwater supply and flood control. Water level changes in lakes reflect their natural responses to climatic and anthropogenic stressors; however, their monitoring is costly due to installation and maintenance requirements. With its advanced hardware and computational capabilities, altimetry has become a popular alternative to conventional in-situ gauging, although subject to the temporal availability of altimetric observations. To further improve the temporal resolution of altimetric measurements, we here combine radar altimetry data with Differential Interferometric Synthetic Aperture Radar (D-InSAR), using ten lakes in Sweden as a testing platform. First, we use Sentinel-1A and Sentinel-1B SAR images to generate consecutive six-day baseline interferograms across 2019. Then, we accumulate the phase change of coherent pixels to construct the time series of InSAR-derived water level anomalies. Finally, we retrieve altimetric observations from Sentinel-3, estimate their mean and standard deviation, and apply them to the D-InSAR standardized anomalies. In this way, we build a water-level time series with more temporal observations. In general, we find a strong agreement between water level estimates from the combination of D-InSAR and Satellite Altimetry (DInSAlt) and in-situ observations in eight lakes (Concordance Correlation Coefficient - CCC >0.8) and moderate agreement in two lakes (CCC >0.57). The applicability of DInSAlt is limited to lakes with suitable conditions for double-bounce scattering, such as the presence of trees or marshes. The accuracy of the water level estimates depends on the quality of the altimetry observations and the lake's width. These findings are important considering the recently launched Surface Water and Ocean Topography (SWOT) satellite, whose capabilities could expand our methodology's geographical applicability and reduce its reliance on ground measurements.

Mutation-Based Multivariate Time-Series Anomaly Generation on Latent Space with an Attention-Based Variational Recurrent Neural Network for Robust Anomaly Detection in an Industrial Control System

Anomaly detection involves identifying data that deviates from normal patterns. Two primary strategies are used: one-class classification and binary classification. In Industrial Control Systems (ICS), where anomalies can cause significant damage, timely and accurate detection is essential, often requiring analysis of time-series data. One-class classification is commonly used but tends to have a high false alarm rate. To address this, binary classification is explored, which can better differentiate between normal and anomalous data, though it struggles with class imbalance in ICS datasets. This paper proposes a mutation-based technique for generating ICS time-series anomalies. The method maps ICS time-series data into a latent space using a variational recurrent autoencoder, applies mutation operations, and reconstructs the time-series, introducing plausible anomalies that reflect multivariate correlations. Evaluations of ICS datasets show that these synthetic anomalies are visually and statistically credible. Training a binary classifier on data augmented with these anomalies effectively mitigates the class imbalance problem.

Univariate Time Series Anomaly Detection Based on Hierarchical Attention Network

Univariate Time Series Anomaly Detection Based on Hierarchical Attention Network

Интеллектуальная технология оценки остаточного ресурса сложных технических систем

The task of estimating the residual life of complex technical systems has recently become increasingly important. For the pre-trustworthy estimation of this indicator it is required to process large arrays of data on the current state of the system under study. At the same time the task of reconstruction of the model of degradation processes development leading to the occurrence of failures requires solving a number of problems. In this regard, there is a need to use intelligent methods of data processing, which include methods of time series anomaly analysis. Purpose of the study: development of a method for detecting contextual anomalies of time series, allowing to determine the degree of development of degradation processes that lead to the occurrence of failures. Methods. An analogy was established between machine learning methods in supervised, semi-supervised and unsupervised modes and groups of anomaly detection methods differing depending on the degree of availability of labels that characterize the properties and attributes of the corresponding time series, which made it possible to evaluate the features of simple and complex anomalies from unified positions. Results. A spectral method for the analysis of contextual anomalies of time series has been developed, which, in contrast to the known spectral methods involving frequency analysis of time series, uses a special basis of exponential functions; the methodology for calculating the spectral coefficients of the investigated time series, on the basis of which a generalized attribute is calculated, allowing to attribute the investigated case to a normal or anomalous group, is outlined. Conclusion. The proposed method of estimating the residual life of complex technical systems based on the analysis of time series anomalies allows timely detection of the occurrence and development of degradation processes leading to the occurrence of failures, which increases the reliability and safety of technical systems, as well as reduces costs and time for their maintenance.

A scattering transform for graphs based on heat semigroups, with an application for the detection of anomalies in positive time series with underlying periodicities

A scattering transform for graphs based on heat semigroups, with an application for the detection of anomalies in positive time series with underlying periodicities

DeepQC: A deep learning system for automatic quality control of in-situ soil moisture sensor time series data

Amidst changing climate, real-time soil moisture monitoring is vital for the development of in-season decision support tools to help farmers manage weather-related risks in agriculture. Precision Sustainable Agriculture (PSA) recently established a real-time soil moisture monitoring network across the central, Midwest, and eastern U.S., but continuous field-scale sensor observations often come with data gaps and anomalies. To maintain the data quality and continuity needed for developing decision tools, a quality control system is necessary.The International Soil Moisture Network (ISMN) introduced the Flagit module for anomaly detection in soil moisture time series observations. However, under certain conditions, Flagit's threshold and spectral based quality control approaches may underperform in identifying anomalies. Recently, deep learning methods have been successfully applied to detect time series anomalies in time series data in various disciplines. However, their use in agriculture for anomaly detection in time series datasets has not been yet investigated. This study focuses on developing a Bi-directional Long Short-Term Memory (LSTM) model, referred to as DeepQC, to identify anomalies in soil moisture time series data. Manual flagged PSA observations were used for training, validation, and testing the model, following an 80:10:10 split. The study then compared the DeepQC and Flagit-based estimates to assess their relative performance.Flagit correctly flagged 95.8 % of the correct observations and 50.3 % of the anomaly observations, indicating its limitations in identifying anomalies, particularly at sites consists of more than 30 % anomalies. On the other hand, the DeepQC correctly flagged 89.8 % of the correct observations and 99.5 % of the anomalies, with overall accuracy of 95.6 %, in significantly less time, demonstrating its superiority over Flagit approach. Importantly, the performance of the DeepQC remained consistent regardless of the number of anomalies in site observations. Given the promising results obtained with the DeepQC, future studies will focus on implementing and finetuning this model on national and global soil moisture networks.

Anomaly detection in hydrological network time series via multiresolution analysis

The territorial extension of Brazil imposes logistic challenges on hydrologic monitoring and increases the chances of systematic observational errors, gauge displacements, downsampling during observation, and low density of streamflow samples on high water stage. Consequently, time series would exhibit artifacts such as gaps, high-frequency anomalies, inhomogeneity, and trends. An innovative anomaly detection system based on multiresolution analysis (MADE) was developed and used to detect synthetic anomalies in raw water stage time series. MADE uses both time and frequency as contextual attributes to identify anomalies. The balance between temporal and frequency resolution allows for capturing low-frequency coarse information and high-frequency details. On MADE, Generalized Extreme Studentized Deviate (GESD) detected short time anomalies on water stage wavelet faster modes (2-, 4-, and 8-days periods). For periods greater than 16 days, Wavelet Coherence and Multiple Wavelet Coherence detected similar water stage time series anomalies. MADE outperformed a Long Short-Term Memory model, a time regression neural network k-Nearest Neighbor model and an Empirical Data Analytics model coupled with clustering techniques on detecting both fast and slow time-scales synthetic anomalies introduced in a São Francisco River water stage time series, using time and frequency domain to pursuit point, collective and contextual anomalies.

Time series anomaly detection using generative adversarial network discriminators and density estimation for infrastructure systems

Efficient data-driven defect detection techniques are crucial for maintaining service quality and providing early warnings for infrastructure systems. To this end, we proposed an effective unsupervised anomaly detection framework (DEGAN) using Generative Adversarial Networks (GANs). The framework relies solely on normal time series data as input to train well-configured discriminators into standalone anomaly predictors by leveraging repeatedly collected data from an infrastructure system. Expected normal patterns in data are identified by generators, and well-configured discriminators are extracted to evaluate anomalies in unseen time series. Kernel density estimation (KDE) is coupled with discriminators for probabilistic anomaly detection. Through a Class I railroad track case study, we evaluated the performance of a convolutional DEGAN in detecting anomalies identified by operators, achieving recall and precision of 80% and 86%, respectively. We also investigated the influence of GAN architectures and parameters, model validation scheme (supervised vs. unsupervised), clustering, and the KDE parameters.

STFT-TCAN: A TCN-attention based multivariate time series anomaly detection architecture with time-frequency analysis for cyber-industrial systems

Networks and industrial systems play a pivotal role in modern society, and their security has garnered increasing attention. Anomalies within industrial equipment may propagate through fault transmission, leading to a cascade of failures. Additionally, cyberattacks on equipment can result in significant losses. Therefore, in the realm of industrial and cyberspace domains, an effective multivariate time series anomaly detection system for monitoring equipment is instrumental in ensuring the healthy operation of the machinery. Nevertheless, detecting anomalies in numerous time series remains challenging, stemming from the absence of anomaly labels and the complexity of the data patterns. Existing algorithms predominantly concentrate on modeling within the time domain, falling short in fully leveraging the informative features present in frequency domain data, resulting in diminished detection performance. This paper introduces STFT-TCAN, a model for anomaly detection in time series that seamlessly integrates information from both time and frequency domains for extracting data features. Sliding windows and the Short Time Fourier Transform (STFT) are utilized to construct a frequency matrix, effectively amalgamating the characteristics of both time and frequency domains within the time series. Furthermore, the model employs Temporal Convolutional Networks (TCN) and Transformer attention mechanisms (which combined to form the TCAN module) to capture the features of multivariate time series, thereby resulting in heightened detection accuracy. The proposed model undergoes validation on six publicly available datasets, showcasing the superior performance of the STFT-TCAN model in comparison to current baseline methods. It adeptly extracts features from both frequency and time domains in sequential data, thereby achieving state-of-the-art performance in tasks related to anomaly detection in multivariate time series.

External factors driving surface temperature changes above geothermal systems: answers from deep learning

Introduction: The surface expression of enhanced geothermal heat fluxes above an active hydrothermal system causes a surface thermal anomaly (ΔT). The thermal anomaly is expressed by the difference between the temperature within the heated zone (Th) and the temperature of non-heated surfaces (T0). Given that the resulting thermal anomaly at the surface is of extremely low magnitude (1°C–5°C at Vulcano, Italy), it is extremely sensitive to overprinting by external factors, namely, meteorological influences on surface temperature variation, such as solar heating, wind and rain.Methods: To test the sensitivity of the surface to external drivers, we installed two surface temperature measurement stations within the Vulcano’s Fossa crater, one inside the thermal anomaly and one outside (separation = 50 m), with a weather station co-located with the T0 station. Time series of Th and T0 were collected for 2020, when the Vulcano Fossa hydrothermal system was at a low and stable level of activity so that external drivers would have been the main influences on Th and T0, and hence also ΔT. To test for divergence from normality in terms of diurnal and seasonal variations in Th and T0, and the role of external factors in causing abnormality, we used the deep learning engine DITAN: a domain-agnostic framework to detect and interpret anomalies in time-series data.Results: During the year, DITAN found 16 cases of two types of meteorological events: intense low-pressure systems and high-intensity rainstorms (cloudbursts). Passage of 13 abnormal low-pressure systems were detected (10 between February and May, and three in December), with three abnormal rainstorm events (all in December); all three being coincident with the abnormal low pressure events. We find just two abnormalities in the time series for of Th and T0, both of which coincide with passage of abnormal low-pressure systems, and neither of which coincide with abnormal rain events. We conclude that diurnal and annual heating and cooling cycles, subject to normal meteorological inputs and at a surface above a geothermal-heated source, are immune to anomalous behaviour to the external (meteorological) variations.

ADAPTATION OF THE ALGORITHM FOR DETECTING ANOMALIES IN TIME SERIES FOR NON-STATIONARY STREAMING DATA

Detection of anomalies in streaming time series data has become an important research topic due to the wide range of possible applications, including the detection of extreme weather conditions, malicious attacks on protected facilities, monitoring unauthorised gas and oil leaks, illegal pipeline connections, power cable faults, and water and other environmental pollution. Rapid detection of abnormal conditions and identifying these critical events is essential to protect lives and assets. Therefore, developing appropriate systems and detection methods is an urgent task. Anomaly detection in streaming data is challenging due to its large volume and high speed, presence of noise, and non-stationarity of the signal (or ‘concept drift’). The latter significantly complicates the identification of differences between new ‘typical’ behaviour and abnormal events. Solving this problem requires the algorithm for processing such data to learn and adapt to changing conditions. The paper proposes a modification of the algorithm for detecting anomalies in time series. This algorithm provides early detection of abnormal series in a massive collection of non-stationary streaming time series data. Anomalies are observations that are highly improbable given the previous time series values. The proposed approach is based on the primary detection of the predictive limit for the typical system behaviour using the theory of extreme values, followed by checking for the abnormality of the following series using the sliding window technique. The time series parameters are used as input data and compared by density distribution to detect any significant changes in the distribution of the characteristics. It allows the decision-making model to automatically adapt to the changing environment according to detected changes. Since anomalies are, by definition, exceptions to the typical behaviour of a system, most of the available stored data should reflect that typical behaviour of the system in question. It is unnecessary to have representative samples of all possible types of the standard behaviour of a given system for the algorithm to work well. The basic idea is to have a warm-up data set to obtain initial values for the decision model parameters. It makes it possible to determine if there is any significant difference between the last typical behaviour and the new typical behaviour. The proposed algorithm demonstrates its performance under conditions of noisy non-stationary data in several time series classes. Keywords: concept drift, extreme value theory, multivariate time series, outlier detection.

Evaluation of reanalysis soil moisture products using cosmic ray neutron sensor observations across the globe

Abstract. Reanalysis soil moisture products are valuable for diverse applications, but their quality assessment is limited due to scale discrepancies when compared to traditional in situ point-scale measurements. The emergence of cosmic ray neutron sensors (CRNSs) with field-scale soil moisture estimates (∼ 250 m radius, up to 0.7 m deep) is more suitable for the product evaluation owing to their larger footprint. In this study, we perform a comprehensive evaluation of eight widely used reanalysis soil moisture products (ERA5-Land, CFSv2, MERRA2, JRA55, GLDAS-Noah, CRA40, GLEAM and SMAP L4 datasets) against 135 CRNS sites from the COSMOS-UK, COSMOS-Europe, COSMOS USA and CosmOz Australia networks. We evaluate the products using six metrics capturing different aspects of soil moisture dynamics. Results show that all reanalysis products generally exhibit good temporal correlation with the measurements, with the median temporal correlation coefficient (R) values spanning 0.69 to 0.79, though large deviations are found at sites with seasonally varying vegetation cover. Poor performance is observed across products for soil moisture anomalies time series, with R values varying from 0.46 to 0.66. The performance of reanalysis products differs greatly across regions, climate, land covers and topographic conditions. In general, all products tend to overestimate data in arid climates and underestimate data in humid regions as well as grassland. Most reanalysis products perform poorly in steep terrain. Relatively low temporal correlation and high bias are detected in some sites from the west of the UK, which might be associated with relatively low bulk density and high soil organic carbon. Overall, ERA5-Land, CRA40, CFSv2, SMAP L4 and GLEAM exhibit superior performance compared to MERRA2, GLDAS-Noah and JRA55. We recommend that ERA5-Land and CFSv2 could be used in humid climates, whereas SMAP L4 and CRA40 perform better in arid regions. SMAP L4 has good performance for cropland, while GLEAM is more effective in shrubland regions. Our findings also provide insights into directions for improvement of soil moisture products for product developers.

Unsupervised detecting anomalies in multivariate time series by Robust Convolutional LSTM Encoder–Decoder (RCLED)

Monitoring modern industrial systems generates a large amount of multivariate time series data. One of the critical tasks of monitoring these systems is anomaly detection. The auto-encoder has emerged as a promising solution for detecting anomalies in systems lacking explicit anomaly data in their historical records; however, its performance can be sensitive to noisy data. This work aims at developing a robust anomaly detection model based on the autoencoder for multivariate time series. The autoencoder architecture consists of convolution, long short-term memory, and self-attention layers for better extraction of complex features from multivariate time series. In addition, a novel training technique, inherited from the idea of robust principal component analysis, was developed to efficiently train the proposed model when the input data is affected by noise. To evaluate the model’s performance, we tested it on two data sets: synthetic, real-industrial, and three public data. We also compare the performance of our model to that of the other state-of-the-art models. The results show that the proposed model outperforms all the latest models, especially when the noise level is considerable.