MemMambaAD: Memory-augmented state space model for multivariate time series anomaly detection

In this article, we present an overview of recent advances in Bayesian modeling and analysis of multivariate time series of counts. We discuss basic modeling strategies including integer valued autoregressive processes, multivariate Poisson time series and dynamic latent factor models. In so doing, we make a connection with univariate modeling frameworks such as dynamic generalized models, Poisson state space models with gamma evolution and present Bayesian approaches that extend these frameworks to multivariate setting. During our development, recent Bayesian approaches to the analysis of integer valued autoregressive processes and multivariate Poisson models are highlighted and concepts such as “decouple/recouple” and “common random environment” are presented. The role that these concepts play in Bayesian modeling and analysis of multivariate time series are discussed. Computational issues associated with Bayesian inference and forecasting from these models are also considered.This article is categorized under:Statistical and Graphical Methods of Data Analysis > Bayesian Methods and TheoryStatistical Models > Time Series Models

Continuous and uncontrolled extraction of groundwater often creates tremendous pressure on groundwater levels (GWLs). As a part of sustainable planning and effective management of water resources, it is crucial to assess the existing and forecasted GWL conditions. In this study, an attempt was made to model and forecast GWL using artificial neural networks (ANNs) and multivariate time series models. Autoregressive integrated moving average (ARIMA) and ARIMA models incorporating exogenous variables (ARIMAX) were adopted as the time series models. GWL data from five monitoring wells from the study area of the Kushtia District in Bangladesh were used to demonstrate the modeling exercise. Rainfall (RF) was taken as the exogenous variable to explore whether its inclusion enhanced the performance of GWL forecasting using the developed models. It was evident from the results that the multivariate ARIMAX model (with the sum of squared errors, SSE, of 15.143) performed better than the univariate ARIMA model with an SSE of 16.585 for GWL forecasting. This demonstrates the fact that the multivariate time series models generated enhanced forecasting of GWL compared to the univariate time series models. When comparing the models, it was found that the ANN-based model outperformed the time series models with enhanced forecasting accuracy (SSE of 9.894). The results also exhibit a significant correlation coefficient (R) of 0.995 (model ANN 6-8-1) for the existing and predicted data. The current study conclusively proves the superiority of ANN over the time series models for the enhanced forecasting of GWL in the study area.

JCCMTM: Joint channel-independent and channel-dependent strategy for masked multivariate time-series modeling.

This paper studies an effective unsupervised deep learning model for multivariate time series anomaly detection. Since multivariate time series usually have problems of insufficient labeling and highly-complex temporal correlation, effectively detecting anomalies in multivariate time series data is particularly challenging. To solve this problem, we propose a model named Wasserstein-GAN with gradient Penalty and effective Scoring (WPS). In this model, Wasserstein Distance with Gradient Penalty helps to capture the data regularities between generator output and real data, thus improving the training stability. Meanwhile, an effective scoring function that consists of reconstruction error, discrimination error, and prediction error is designed to evaluate the accuracy of the abnormal prediction and recall. The experimental results show that compared with the suboptimal baseline model, our proposed WPS obtains 17.68% and 10.41% improvement in prediction precision and F1 score, respectively.

Pipe material and labor costs constitute about 70% of pipeline construction costs. Pipe and labor costs are subject to considerable fluctuations over time. These fluctuations are problematic for cost estimation and bid preparation in pipeline projects, which are mostly large and long-term projects. The accurate prediction of pipe and labor costs is invaluable for cost estimators to prepare accurate bids and manage the cost contingencies. However, the existing literature does not take advantage of the leading indicators of pipeline construction cost time series to accurately forecast cost fluctuations in pipeline projects. The objective of this research is to identify the leading indicators of pipeline construction costs and develop multivariate time series models for forecasting cost fluctuations in pipeline projects. Nineteen potential leading indicators of pipe and labor costs were initially selected based on a comprehensive review of construction cost forecasting literature. The leading indicators were identified from this pool of potential leading indicators based on unit root tests and Granger causality tests. Multivariate time series models were developed based on the results of cointegration tests. Vector error correction (VEC) models were developed for the cointegrated variables, while vector autoregressive (VAR) models were developed for the non-cointegrated variables. Since multivariate time series models include information from the identified leading indicators, multivariate time series models are often expected to deliver more accurate forecasts than univariate time series models. The forecasting accuracies of multivariate time series models were compared with those of univariate time series models based on three common error measures: mean absolute prediction error (MAPE), root-mean-squared error (RMSE), and mean average error (MAE). The results show that multivariate time series models outperform univariate models for forecasting cost fluctuations in pipeline projects. The findings of this research contribute to the state of knowledge by identifying leading indicators of pipe and labor costs and developing multivariate time series models to forecast them. The multivariate time series models with leading indicators are more accurate than univariate models for forecasting cost fluctuations in pipeline projects. It is expected that the proposed multivariate time series forecasting models contribute to the enhancement of the theory and practice of pipeline construction cost forecasting and help cost engineers and investment planners to prepare more accurate bids, cost estimates, and budgets for pipeline projects.

A new Kalman filter approach for the estimation of high-dimensional time-variant multivariate AR models and its application in analysis of laser-evoked brain potentials

It is important to understand the behavior within a social network, particularly excessive communications between nodes. Such excessive activities in a network provide an insight into the pattern of communication between nodes, which, in some cases, could lead to a fraudulent behavior. Scan statistics have been applied before to detect the excessive communications in email networks. However, they alone are not effective in revealing the dynamic relationships and progression of chatter as the scan statistics relate to the maximum of locality statistics. Here a multivariate time series model, vector autoregressive (VAR) model, has been developed and applied to the metadata of organization e-mails as a case study to detect a group of influential nodes and their dynamic relationship. Furthermore, we devise a new methodology, which utilizes the probabilistic topic model obtained from the e-mail content, scan statistics, and time series of maximum information flow. We demonstrate how the influential vertices obtained from the VAR model are connected with the anomalous topic activities. These methodologies would be highly useful in studying the excessive communications and anomalous topic activities in other dynamic networks, such as, twitter networks, telephone calls, scientific collaborations, and other social networks.

In OCT angiography (OCTA), decorrelation computation has been widely used as a local motion index to identify dynamic flow from static tissues, but its dependence on SNR severely degrades the vascular visibility, particularly in low-SNR regions. To mathematically characterize the decorrelation-SNR dependence of OCT signals, we developed a multi-variate time series (MVTS) model. Based on the model, we derived a universal asymptotic linear relation of decorrelation to inverse SNR (iSNR), with the variance in static and noise regions determined by the average kernel size. Accordingly, with the population distribution of static and noise voxels being explicitly calculated in the iSNR and decorrelation (ID) space, a linear classifier is developed by removing static and noise voxels at all SNR, to generate a SNR-adaptive OCTA, termed as ID-OCTA. Then, flow phantom and human skin experiments were performed to validate the proposed ID-OCTA. Both qualitative and quantitative assessments demonstrated that the ID-OCTA offers a superior visibility of blood vessels, particularly in the deep layer. Finally, the implications of this work on both system design and hemodynamic quantification are further discussed.

In real-world scenarios, partial information losses of multivariate time series degrade the time series analysis. Hence, the time series imputation technique has been adopted to compensate for the missing values. Existing methods focus on investigating temporal correlations, cross-variable correlations, and bidirectional dynamics of time series, and most of these methods rely on recurrent neural networks (RNNs) to capture temporal dependency. However, the RNN-based models suffer from the common problems of slow speed and high complexity when dealing with long-term dependency. While some self-attention-based models without any recurrent structures can tackle long-term dependency with parallel computing, they do not fully learn and utilize correlations across the temporal and cross-variable dimensions. To address the limitations of existing methods, we propose a novel so-called dual-branch cross-dimensional self-attention-based imputation (DCSAI) model for multivariate time series, which is capable of performing global and auxiliary cross-dimensional analyses when imputing the missing values. In particular, this model contains masked multi-head self-attention-based encoders aligned with auxiliary generators to obtain global and auxiliary correlations in two dimensions, and these correlations are then combined into one final representation through three weighted combinations. Extensive experiments are presented to show that our model performs better than other state-of-the-art benchmarkers on three real-world public datasets under various missing rates. Furthermore, ablation study results demonstrate the efficacy of each component of the model.

Time series modeling and analysis is central to most financial and econometric data modeling. With increased globalization in trade, commerce and finance, national variables like gross domestic productivity (GDP) and unemployment rate, market variables like indices and stock prices and global variables like commodity prices are more tightly coupled than ever before. This translates to the use of multivariate or vector time series models and algorithms in analyzing and understanding the relationships that these variables share with each other. Autocorrelation is one of the fundamental aspects of time series modeling. However, traditional linear models, that arise from a strong observed autocorrelation in many financial and econometric time series data, are at times unable to capture the rather nonlinear relationship that characterizes many time series data. This necessitates the study of nonlinear models in analyzing such time series. The class of bilinear models is one of the simplest nonlinear models. These models are able to capture temporary erratic fluctuations that are common in many financial returns series and thus, are of tremendous interest in financial time series analysis. Another aspect of time series analysis is homoscedasticity versus heteroscedasticity. Many time series data, even after differencing, exhibit heteroscedasticity. Thus, it becomes important to incorporate this feature in the associated models. The class of conditional heteroscedastic autoregressive (ARCH) models and its variants form the primary backbone of conditional heteroscedastic time series models. Robustness is a highly underrated feature of most time series applications and models that are presently in use in the industry. With an ever increasing amount of information available for modeling, it is not uncommon for the data to have some aberrations within itself in terms of level shifts and the occasional large fluctuations. Conventional methods like the maximum likelihood and least squares are well known to be highly sensitive to such contaminations. Hence, it becomes important to use robust methods, especially in this age with high amounts of computing power readily available, to take into account such aberrations. While robustness and time series modeling have been vastly researched individually in the past, application of robust methods to estimate time series models is still quite open. The central goal of this thesis is the study of robust parameter estimation of some simple vector and nonlinear time series models. More precisely, we will briefly study some prominent linear and nonlinear models in the time series literature and apply the robust S-estimator in estimating parameters of some simple models like the vector autoregressive (VAR) model, the (0, 0, 1, 1) bilinear model and a simple conditional heteroscedastic bilinear model. In each case, we will look at the important aspect of stationarity of the model and analyze the asymptotic behavior of the S-estimator.

SUMMARY We derive conditions for decomposition and collapsibility of graphical interaction models for multivariate time series. These properties enable us to perform stepwise model selection under certain restrictions. For illustration, we apply the results to a multivariate time series describing the haemodynamic system as monitored in intensive care.

Data protection is the process of backing up data in case of a data loss event. It is one of the most critical routine activities for every organization. Detecting abnormal backup jobs is important to prevent data protection failures and ensure the service quality. Given the large scale backup endpoints and the variety of backup jobs, from a backup-as-a-service provider viewpoint, we need a scalable and flexible outlier detection method that can model a huge number of objects and well capture their diverse patterns. In this paper, we introduce H 2 O, a novel hybrid and hierarchical method to detect outliers from millions of backup jobs for large scale data protection. Our method automatically selects an ensemble of outlier detection models for each multivariate time series composed by the backup metrics collected for each backup endpoint by learning their exhibited characteristics. Interactions among multiple variables are considered to better detect true outliers and reduce false positives. In particular, a new seasonal-trend decomposition based outlier detection method is developed, considering the interactions among variables in the form of common trends, which is robust to the presence of outliers in the training data. The model selection process is hierarchical, following a global to local fashion. The final outlier is determined through an ensemble learning by multiple models. Built on top of Apache Spark, H2O has been deployed to detect outliers in a large and complex data protection environment with more than 600,000 backup endpoints and 3 million daily backup jobs. To the best of our knowledge, this is the first work that selects and constructs large scale outlier detection models for multivariate time series on big data platforms.

This PhD dissertation deals with the world of multivariate time series models where the behaviour of the observed process is described by using a time-varying parameter. In particular, this thesis explore three different dynamic multivariate nonlinear models which are able to deal with multivariate time series gathered from heavy-tailed phenomena. Although the popularity of linear and univariate time series models, empirical evidences have shown that variables generated from complex phenomena are typically inter-related both contemporaneously and across time. This is the case for several fields of science such as economics, finance, biology or physics, where it is widely accepted that with a univariate approach it is difficult to obtain a satisfactory representation of the reality or to make good predictions about the future. For these reasons, the literature of linear multivariate Gaussian time series models has received increasing attention. However, these models are known for their unsatisfactory performances when the collected data are contaminated by outliers, yielding biased estimates and unreliable forecasts. In fact, when departure from the hypothesis of normality is confirmed by the observed data, it is reasonable to switch into the realm of nonlinear or non-Gaussian time series models. Unfortunately, despite the development of recent technologies, the estimation of nonlinear time series models might be really challenging, since they require simulation-based and computer-intensive methods. In addition, statistical properties of such estimators are not always easy to be derived. This thesis contributes to the literature by defining dynamic multivariate and heavy-tailed models that are relatively simple. The emphasis is models which are analytically tractable and can be easily estimated by means of maximum likelihood. For each of the models, a very detailed statistical and asymptotic analysis it is provided. Their practical usefulness is highlighted with several simulation studies and empirical applications.

The univariate time series methods and models described in Chapter 3 are based on one time series containing observations on the exchange rate to be forecast over a particular period of time. This chapter deals with multivariate time series models which, as the name implies, involve more than one time series. The time series may relate to various exchange rates to be forecast jointly or to an exchange rate and its determining variables. These models take the following forms. Single-equation economic models are also known as single-equation econometric models or reduced-form models. These models consist of a single equation that specifies the exchange rate (as the dependent variable) to be a function of some explanatory variables. They are ‘economic’ models because they are based on economic theory, unlike the models described in Chapter 3. They are ‘econometric’ models because they are estimated by using some econometric method such as OLS. They are ‘reduced-form’ models because the single equation explains the dependent (endogenous) variable in terms of other (exogenous) explanatory variables.A single equation structural time series model is similar to Harvey’s (1989) structural time series model, which was described in the previous chapter, except that it contains explanatory variables. In such a model, the exchange rate is determined by its time series components as well as some explanatory variables.

Landslide is a kind of genetic type of slope and has the same characteristics with slope. The major external motivation factor of landslide displacement is groundwater and it is under the control of remedial measures at the same time after its remediation. Chaotic time series of landslide displacement and its influential factors could reflect the history of landslide displacement of dynamic system, the displacement could be predicted by reconstructing landslide displacement of dynamic system according to the observation of multivariate time series and adopting RBF neural network to reflect relationship between variables. Comparative analysis of the results from the forecast show that: multivariable time series model can predict landslide displacement effectively, and the forecast accuracy is higher than the accuracy of a single variable time series model; multivariable time series model is of clearer sense of the physical mechanics and reflects the real evolution of deformation characteristics more effective.