Nonstationary Multivariate Time Series Research Articles

Improving the accuracy of predictions in multivariate time series using dynamic vine copulas

In this work, we deal with non-stationary multivariate time series, proposing a method which uses copulas to produce more accurate forecasting. The idea is to apply a copula-based approach to identify change points and then split the time series into consecutive segments based on these change points. In each segment, we define the best-fitting copula family and forecast values of the time series of each segment using the corresponding fitting copula. We apply our model to a financial data set to test the predictive power of our approach. A simulation study is also presented for a detailed illustration and assessment of our proposed methodology. Based on the results of numerical analysis, we observed that our proposed approach will help us to improve the accuracy of forecasting in comparison with other existing methods such as traditional time series forecasting as well as neural network forecasting.

Inverse covariance operators of multivariate nonstationary time series

Inverse covariance operators of multivariate nonstationary time series

Adaptive Wavelet Domain Principal Component Analysis for Nonstationary Time Series

High-dimensional multivariate nonstationary time series, that is, data whose second order properties vary over time, are common in many scientific and industrial applications. In this article we propose a novel wavelet domain dimension reduction technique for nonstationary time series. By constructing a time-scale adaptive principal component analysis of the data, our proposed method is able to capture the salient dynamic features of the multivariate time series. We also introduce a new time and scale dependent cross-coherence measure to quantify the extent of association between a multivariate nonstationary time series and its proposed wavelet domain principal component representation. Theoretical results establish that our associated estimation scheme enjoys good bias and consistency properties when determining wavelet domain principal components of input data. The proposed method is illustrated using extensive simulations and we demonstrate its applicability on a real-world dataset arising in a neuroscience study. Supplementary materials, with proofs of theoretical results, additional simulations and code, are available online.

Identifying quasi-periodic variability using multivariate empirical mode decomposition: a case of the tropical Pacific

Abstract. A variety of statistical tools have been used in climate science to gain a better understanding of the climate system's variability on various temporal and spatial scales. However, these tools are mostly linear, stationary, or both. In this study, we use a recently developed nonlinear and nonstationary multivariate time series analysis tool – multivariate empirical mode decomposition (MEMD). MEMD is a powerful tool for objectively identifying (intrinsic) timescales of variability within a given spatio-temporal system without any timescale pre-selection. Additionally, a red noise significance test is developed to robustly extract quasi-periodic modes of variability. We apply these tools to reanalysis and observational data of the tropical Pacific. This reveals a quasi-periodic variability in the tropical Pacific on timescales ∼ 1.5–4.5 years, which is consistent with El Niño–Southern Oscillation (ENSO) – one of the most prominent quasi-periodic modes of variability in the Earth’s climate system. The approach successfully confirms the well-known out-of-phase relationship of the tropical Pacific mean thermocline depth with sea surface temperature in the eastern tropical Pacific (recharge–discharge process). Furthermore, we find a co-variability between zonal wind stress in the western tropical Pacific and the tropical Pacific mean thermocline depth, which only occurs on the quasi-periodic timescale. MEMD coupled with a red noise test can therefore successfully extract (nonstationary) quasi-periodic variability from the spatio-temporal data and could be used in the future for identifying potential (new) relationships between different variables in the climate system.

Change points detection for nonstationary multivariate time series

Change points detection for nonstationary multivariate time series

Online evolutionary neural architecture search for multivariate non-stationary time series forecasting

Time series forecasting (TSF) is one of the most important tasks in data science. TSF models are usually pre-trained with historical data and then applied on future unseen datapoints. However, real-world time series data is usually non-stationary and models trained offline usually face problems from data drift. Models trained and designed in an offline fashion can not quickly adapt to changes quickly or be deployed in real-time. To address these issues, this work presents the Online NeuroEvolution-based Neural Architecture Search (ONE-NAS) algorithm, which is a novel neural architecture search method capable of automatically designing and dynamically training recurrent neural networks (RNNs) for online forecasting tasks. Without any pre-training, ONE-NAS utilizes populations of RNNs that are continuously updated with new network structures and weights in response to new multivariate input data. ONE-NAS is tested on real-world, large-scale multivariate wind turbine data as well as the univariate Dow Jones Industrial Average (DJIA) dataset. Results demonstrate that ONE-NAS outperforms traditional statistical time series forecasting methods, including online linear regression, fixed long short-term memory (LSTM) and gated recurrent unit (GRU) models trained online, as well as state-of-the-art, online ARIMA strategies. Additionally, results show that utilizing multiple populations of RNNs which are periodically repopulated provide significant performance improvements, allowing this online neural network architecture design and training to be successful.

Bayesian circular lattice filters for computationally efficient estimation of multivariate time-varying autoregressive models

Nonstationary time series data exist in various scientific disciplines, including environmental science, biology, signal processing, econometrics, among others. Many Bayesian models have been developed to handle nonstationary time series. The time-varying vector autoregressive (TV-VAR) model is a well-established model for multivariate nonstationary time series. Nevertheless, in most cases, the large number of parameters presented by the model results in a high computational burden, ultimately limiting its usage. To address this issue, a computationally efficient multivariate Bayesian Circular Lattice Filter is developed, extending the usage of the TV-VAR model to a broader class of high-dimensional problems. The fully Bayesian framework allows both the autoregressive (AR) coefficients and innovation covariance to vary over time. The proposed estimation method is based on the Bayesian lattice filter (BLF), which is extremely computationally efficient and stable in univariate cases. To illustrate the effectiveness of the proposed approach, a comprehensive comparison with other competing methods is conducted through simulation studies and finds that, in most cases, the proposed approach performs superior in terms of the average squared error between the estimated and true time-varying spectral density. Finally, the methodology is demonstrated through applications to quarterly Gross Domestic Product (GDP) data and Northern California wind data.

Inference in Heavy-Tailed Nonstationary Multivariate Time Series

We study inference on the common stochastic trends in a nonstationary, N-variate time series yt , in the possible presence of heavy tails. We propose a novel methodology which does not require any knowledge or estimation of the tail index, or even knowledge as to whether certain moments (such as the variance) exist or not, and develop an estimator of the number of stochastic trends m based on the eigenvalues of the sample second moment matrix of yt . We study the rates of such eigenvalues, showing that the first m ones diverge, as the sample size T passes to infinity, at a rate faster by O ( T ) than the remaining N – m ones, irrespective of the tail index. We thus exploit this eigen-gap by constructing, for each eigenvalue, a test statistic which diverges to positive infinity or drifts to zero according to whether the relevant eigenvalue belongs to the set of the first m eigenvalues or not. We then construct a randomized statistic based on this, using it as part of a sequential testing procedure, ensuring consistency of the resulting estimator of m. We also discuss an estimator of the common trends based on principal components and show that, up to a an invertible linear transformation, such estimator is consistent in the sense that the estimation error is of smaller order than the trend itself. Importantly, we present the case in which we relax the standard assumption of iid innovations, by allowing for heterogeneity of a very general form in the scale of the innovations. Finally, we develop an extension to the large dimensional case. A Monte Carlo study shows that the proposed estimator for m performs particularly well, even in samples of small size. We complete the article by presenting two illustrative applications covering commodity prices and interest rates data. Supplementary materials for this article are available online.

Novel hybrid multi-head self-attention and multifractal algorithm for non-stationary time series prediction

Traditional time series prediction methods have shown their outstanding capabilities in time series prediction. However, due to essential differences in volatility characteristics among diverse types of non-stationary multivariate time series (NSMTS), it is difficult for traditional methods to maintain robust prediction performance. This study proposes a novel dynamic recurrent neural network to achieve stable and robust prediction performance. First, a multifractal gated recurrent unit (MF-GRU) based on the multifractal method is proposed to extract volatility characteristics. Meanwhile, to strengthen the parameters of the historical hidden layer state that has a more significant impact on the output, a self-attention mechanism is introduced into the MF-GRU, leading to a multifractal gated recurrent unit multi-head self-attention model. The efficiency of the proposed model was verified on public datasets. The experimental results show that the proposed model outperforms the traditional methods, such as long short-term memory (LSTM), the gated recurrent unit (GRU), and the minimal gated unit (MGU). etc.

PRINCIPAL COMPONENT ANALYSIS-VECTOR AUTOREGRESSIVE INTEGRATED (PCA-VARI) MODEL USING DATA MINING APPROACH TO CLIMATE DATA IN THE WEST JAVA REGION

Over a long time, atmospheric changes have been caused by natural phenomena. This study uses the Principal Component Analysis (PCA) model combined with Vector Autoregressive Integrated (VARI) called the PCA-VARI model through the data mining approach. PCA reduces ten variables of climate data into two principal components during ten years (2001-2020) of climate data from NASA Prediction Of Worldwide Energy Resources. VARI is a non-stationary multivariate time series to model two or more variables that influence each other using a differencing process. The Knowledge Discovery in Database (KDD) method was conducted for empirical analysis. Pre-processing is an analysis of raw climate data. The data mining process determines the proportion of each component of PCA and is selected as variables in the VARI process. The postprocessing is by visualizing and interpreting the PCA-VARI model. Variables of solar radiation and precipitation are strongly correlated with each measurement location data. A forecast of the interaction of variables between locations is shown in the results of Impulse Response Function (IRF) visualization, where the climate of the West Java region, especially the Lembang and Bogor areas, has strong response climate locations, which influence each other.

Greedy copula segmentation of multivariate non-stationary time series for climate change adaptation

Non-stationary climate data are often encountered in dealing with natural hazards, climate change and disaster reduction. With drought, for instance, it is common to encounter such non-stationary data sets (time series). The objectives of this work are to formulate a rational data-driven approach that can consider non-stationary and time series on multiple random variables that can have generalized underlying probability distributions and dependence structures. The methodology proposed seeks to divide up the data into non-overlapping segments, each of which is treated as stationary with some underlying probability and dependence structure, while the long time series yields multiple such segments that are mutually independent. The Greedy Copula Segmentation (GCS) algorithm developed employs best-fit probability distributions and copula functions after data-driven time series segmentation. Validation of the proposed methodology is demonstrated using a benchmark problem as well as a single-site realistic drought example. The proposed GCS approach has potential use in climate change adaptation (CCA) and disaster risk reduction (DRR) for any climate-related hazards involving non-stationary time series data.

Change point detection-based simulation of nonstationary sub-hourly wind time series

In this paper, we present a wind speed simulation method by detecting change points in multivariate nonstationary wind speed time series data. The change point detection method identifies changes in the covariance structure and decomposes the nonstationary multivariate time series into stationary segments. Parametric and nonparametric techniques are also provided to model and simulate new time series within each stationary segment. The proposed simulation approach retains the statistical properties of the original time series (as obtained from a Numerical Weather Prediction system) and therefore, can be employed for simulation-based analysis of power systems planning and operations problems. We demonstrate the capabilities of the change point detection method through computational experiments conducted on wind speed time series at five-minute resolution. We also conduct experiments on the economic dispatch problem to illustrate the impact of nonstationarity in wind generation on conventional generation and location marginal prices.

Vector error correction model to analyze energy uses, environmental quality and economic growth during Covid-19 Pandemic

Time series data commonly show are interconnected behaviour and non-stationer interrelated variables, so a model that able to obtain a good forecasting result from a non-stationary multivariate variables time series data are needed. Vector Error Correction Model (VECM) is one of multivariate time series model which is a vector form of Vector Autoregressive Boundary (VAR) for non-stationary time series data and has a cointegration relationship. The purpose of this study is to identify the VECM model in analyzing the relationship between energy use, environmental quality (CO2), and economic growth (GDP) during the Covid-19 pandemic that plagued Indonesia. The results of this study explained energy uses and and environmental quality (CO2) and economic growth (GDP) are interrelated and have a long-term cointegration relationship due to the influence of the Covid-19 pandemic.

Dynamic Networks with Multi-scale Temporal Structure

We describe a novel method for modeling non-stationary multivariate time series, with time-varying conditional dependencies represented through dynamic networks. Our proposed approach combines traditional multi-scale modeling and network based neighborhood selection, aiming at capturing temporally local structure in the data while maintaining sparsity of the potential interactions. Our multi-scale framework is based on recursive dyadic partitioning, which recursively partitions the temporal axis into finer intervals and allows us to detect local network structural changes at varying temporal resolutions. The dynamic neighborhood selection is achieved through penalized likelihood estimation, where the penalty seeks to limit the number of neighbors used to model the data. We present theoretical and numerical results describing the performance of our method, which is motivated and illustrated using task-based magnetoencephalography (MEG) data in neuroscience.

A hybrid feedforward neural network algorithm for detecting outliers in non-stationary multivariate time series

To understand the behavior of complex phenomena, data collection and data analysis are the two basic key issues in this process. The most significant hard problem experimenters may face is the optimality selection of the dataset which provides valuable information about the behavior of the phenomena under the experimentation. An experiment with an optimal dataset allows more significant parameters to be estimated with minimum variance and without bias. Unfortunately, the collected datasets of many real-life experiments are not optimal due to the existence of outliers. An outlier is an observation that deviates significantly from other experimental data points arouse suspicions that it was generated by a different mechanism. The presence of even a few outliers leads to misspecification of model, biased estimation of parameters, and poor forecasts. Therefore, removing the outliers from the collected datasets is the critical and significant step before analyzing the data. This paper gives a hybrid feedforward neural network algorithm for detecting outliers as single points as well as small and large clusters in non-stationary multivariate time series using robust measures of location and dispersion matrix. From various perspectives, the performance of the proposed algorithm is compared with the existing algorithms under different scenarios using simulated datasets.

Long-term forecasting of multivariate time series in industrial furnaces with dynamic Gaussian Bayesian networks

Many of the data sets extracted from real-world industrial environments are time series that describe dynamic processes with characteristics that change over time. In this paper, we focus on the fouling process in an industrial furnace, which corresponds to a non-stationary multivariate time series with a seasonal component, non-homogeneous cycles and sporadic human interventions. We aim to forecast the evolution of the temperature inside the furnace over a long span of time of two and a half months. To accomplish this, we model the time series with dynamic Gaussian Bayesian networks (DGBNs) and compare their performance with convolutional recurrent neural networks. Our results show that DGBNs are capable of properly treating seasonal data and can capture the tendency of a time series without being distorted by the effect of interventions or by the varying length of the cycles.

Moving dynamic principal component analysis for non-stationary multivariate time series

This paper proposes an extension of principal component analysis to non-stationary multivariate time series data. A criterion for determining the number of final retained components is proposed. An advance correlation matrix is developed to evaluate dynamic relationships among the chosen components. The theoretical properties of the proposed method are given. Many simulation experiments show our approach performs well on both stationary and non-stationary data. Real data examples are also presented as illustrations. We develop four packages using the statistical software R that contain the needed functions to obtain and assess the results of the proposed method.

Vector Error Correction Model to Forecasting Spot Prices for Coffee Commodities During Covid-19 Pandemic

Forecasting time series commonly shows non-stationer behavior and involves interrelated variables, so a method that able to obtain a good forecasting result from a non-stationary multivariate time series data is needed. Vector Error Correction Model (VECM) as one of multivariate time series model is a vector form of restricted Vector Autoregressive (VAR) for non-stationary data which has Cointegrity relationship. Prices of agricultural commodities futures usually show variability and unsystematic behavior, especially in times of facing the Covid-19 pandemic as it is today, so getting complete information about data requires data handling while still considering the concept data-driven. Coffee, as one of agricultural commodity data, is often not known for its stationarity. The purpose of this study is to identify the reliability of VECM to predict the spot price of commodities futures coffee during the Covid-19 pandemic in Indonesia. The used data is several observed variables, there are: spot price of Robusta Coffee, futures prices of Robusta Coffee, exchange rates and daily case positive Covid-19 in Indonesia. The results of this study described the Robusta Coffee data in Indonesia as non-stationary and there is a long-term cointegration relationship to the Robusta Coffee futures price movement due to the influence of the Covid-19 pandemic.

Adaptive Bayesian Spectral Analysis of High-Dimensional Nonstationary Time Series

This article introduces a nonparametric approach to spectral analysis of a high-dimensional multivariate nonstationary time series. The procedure is based on a novel frequency-domain factor model that provides a flexible yet parsimonious representation of spectral matrices from a large number of simultaneously observed time series. Real and imaginary parts of the factor loading matrices are modeled independently using a prior that is formulated from the tensor product of penalized splines and multiplicative gamma process shrinkage priors, allowing for infinitely many factors with loadings increasingly shrunk toward zero as the column index increases. Formulated in a fully Bayesian framework, the time series is adaptively partitioned into approximately stationary segments, where both the number and locations of partition points are assumed unknown. Stochastic approximation Monte Carlo techniques are used to accommodate the unknown number of segments, and a conditional Whittle likelihood-based Gibbs sampler is developed for efficient sampling within segments. By averaging over the distribution of partitions, the proposed method can approximate both abrupt and slowly varying changes in spectral matrices. Performance of the proposed model is evaluated by extensive simulations and demonstrated through the analysis of high-density electroencephalography. Supplementary materials for this article are available online.

A wavelet-based approach for imputation in nonstationary multivariate time series

Many multivariate time series observed in practice are second order nonstationary, i.e. their covariance properties vary over time. In addition, missing observations in such data are encountered in many applications of interest, due to recording failures or sensor dropout, hindering successful analysis. This article introduces a novel method for data imputation in multivariate nonstationary time series, based on the so-called locally stationary wavelet modelling paradigm. Our methodology is shown to perform well across a range of simulation scenarios, with a variety of missingness structures, as well as being competitive in the stationary time series setting. We also demonstrate our technique on data arising in a health monitoring application.