Stationary Multivariate Time Series Research Articles

A nonparametrically corrected likelihood for Bayesian spectral analysis of multivariate time series

A novel approach to Bayesian nonparametric spectral analysis of stationary multivariate time series is presented. Starting with a parametric vector-autoregressive model, the parametric likelihood is nonparametrically adjusted in the frequency domain to account for potential deviations from parametric assumptions. A proof of mutual contiguity of the nonparametrically corrected likelihood, the multivariate Whittle likelihood approximation and the exact likelihood for Gaussian time series is given. A multivariate extension of the nonparametric Bernstein-Dirichlet process prior for univariate spectral densities to the space of Hermitian positive definite spectral density matrices is specified directly on the correction matrices. An infinite series representation of this prior is then used to develop a Markov chain Monte Carlo algorithm to sample from the posterior distribution. The code is made publicly available for ease of use and reproducibility. With this novel approach, a generalisation of the multivariate Whittle-likelihood-based method of Meier et al. (2020) as well as an extension of the nonparametrically corrected likelihood for univariate stationary time series of Kirch et al. (2019) to the multivariate case is presented. It is demonstrated that the nonparametrically corrected likelihood combines the efficiencies of a parametric with the robustness of a nonparametric model. Its numerical accuracy is illustrated in a comprehensive simulation study. Its practical advantages are illustrated by a spectral analysis of two environmental time series data sets: a bivariate time series of the Southern Oscillation Index and fish recruitment and a multivariate time series of windspeed data at six locations in California.

Kernel-based joint independence tests for multivariate stationary and non-stationary time series.

Multivariate time-series data that capture the temporal evolution of interconnected systems are ubiquitous in diverse areas. Understanding the complex relationships and potential dependencies among co-observed variables is crucial for the accurate statistical modelling and analysis of such systems. Here, we introduce kernel-based statistical tests of joint independence in multivariate time series by extending the d-variable Hilbert-Schmidt independence criterion to encompass both stationary and non-stationary processes, thus allowing broader real-world applications. By leveraging resampling techniques tailored for both single- and multiple-realization time series, we show how the method robustly uncovers significant higher-order dependencies in synthetic examples, including frequency mixing data and logic gates, as well as real-world climate, neuroscience and socio-economic data. Our method adds to the mathematical toolbox for the analysis of multivariate time series and can aid in uncovering high-order interactions in data.

Estimation of prediction error in time series

Summary The accurate estimation of prediction errors in time series is an important problem, which has immediate implications for the accuracy of prediction intervals as well as the quality of a number of widely used time series model selection criteria such as the Akaike information criterion. Except for simple cases, however, it is difficult or even impossible to obtain exact analytical expressions for one-step and multi-step predictions. This may be one of the reasons that, unlike in the independent case (see Efron, 2004), up to now there has been no fully established methodology for time series prediction error estimation. Starting from an approximation to the bias-variance decomposition of the squared prediction error, a method for accurate estimation of prediction errors in both univariate and multivariate stationary time series is developed in this article. In particular, several estimates are derived for a general class of predictors that includes most of the popular linear, nonlinear, parametric and nonparametric time series models used in practice, with causal invertible autoregressive moving average and nonparametric autoregressive processes discussed as lead examples. Simulations demonstrate that the proposed estimators perform quite well in finite samples. The estimates may also be used for model selection when the purpose of modelling is prediction.

Factor modeling of multivariate time series: A frequency components approach

A frequency domain factor model method for multivariate stationary time series is proposed. The dimension reduction framework aims to find a lower-dimensional multivariate stationary factor series. Frequency components of the observed series are assumed to be linearly generated by the corresponding frequency components of a latent factor series using frequency-specific factor loadings matrices. These loadings matrices are then estimated using an eigendecomposition of symmetric non-negative definite matrices involving the real and imaginary parts of the spectral matrix. The factor dimension is estimated using nonparametric bootstrap tests. Consistency results concerning the estimation of eigenvalues, eigenvectors, factor loadings matrices and the factor dimension are provided. The numerical performance of the proposed method is illustrated through simulation examples. An application to modeling resting-state fMRI time series from autism individuals is demonstrated where a frequency-specific factor analysis helps understand functional connectivity.

Efficient Variational Bayes Learning of Graphical Models With Smooth Structural Changes.

Estimating a sequence of dynamic undirected graphical models, in which adjacent graphs share similar structures, is of paramount importance in various social, financial, biological, and engineering systems, since the evolution of such networks can be utilized for example to spot trends, detect anomalies, predict vulnerability, and evaluate the impact of interventions. Existing methods for learning dynamic graphical models require the tuning parameters that control the graph sparsity and the temporal smoothness to be selected via brute-force grid search. Furthermore, these methods are computationally burdensome with time complexity O(NP3) for P variables and N time points. As a remedy, we propose a low-complexity tuning-free Bayesian approach, named BASS. Specifically, we impose temporally dependent spike and slab priors on the graphs such that they are sparse and varying smoothly across time. An efficient variational inference algorithm based on natural gradients is then derived to learn the graph structures from the data in an automatic manner. Owing to the pseudo-likelihood and the mean-field approximation, the time complexity of BASS is only O(NP2). To cope with the local maxima problem of variational inference, we resort to simulated annealing and propose a method based on bootstrapping of the observations to generate the annealing noise. We provide numerical evidence that BASS outperforms existing methods on synthetic data in terms of structure estimation, while being more efficient especially when the dimension P becomes high. We further apply the approach to the stock return data of 78 banks from 2005 to 2013 and find that the number of edges in the financial network as a function of time contains three peaks, in coincidence with the 2008 global financial crisis and the two subsequent European debt crisis. On the other hand, by identifying the frequency-domain resemblance to the time-varying graphical models, we show that BASS can be extended to learning frequency-varying inverse spectral density matrices, and further yields graphical models for multivariate stationary time series. As an illustration, we analyze scalp EEG signals of patients at the early stages of Alzheimer's disease (AD) and show that the brain networks extracted by BASS can better distinguish between the patients and the healthy controls.

Spectral Subsampling MCMC for Stationary Multivariate Time Series with Applications to Vector ARTFIMA Processes

A multivariate generalisation of the Whittle likelihood is used to extend spectral subsampling MCMC to stationary multivariate time series by subsampling matrix-valued periodogram observations in the frequency domain. To assess the performance of the methodology in challenging problems, a multivariate generalisation of the autoregressive tempered fractionally integrated moving average model (ARTFIMA) is introduced and some of its properties derived. Bayesian inference based on the Whittle likelihood is demonstrated to be a fast and accurate alternative to the exact time domain likelihood. Spectral subsampling is shown to provide up to two orders of magnitude additional speed-up, while retaining MCMC sampling efficiency and accuracy, compared to spectral methods using the full dataset.

Fast Bayesian inference on spectral analysis of multivariate stationary time series

Spectral analysis discovers trends, periodic and other characteristics of a time series by representing these features in the frequency domain. However, when multivariate time series are considered, and the number of components increases, the size of the spectral density matrix grows quadratically, making estimation and inference rather challenging. The proposed novel Bayesian framework considers a Whittle likelihood-based spectral modeling approach and imposes a discounted regularized horseshoe prior on the coefficients that define a spline representation of each of the components of a Cholesky factorization of the inverse spectral density matrix. The proposed prior structure leads to a model that provides higher posterior accuracy when compared to alternative currently available approaches. To achieve fast inference that takes advantage of the massive power of modern hardware (e.g., GPU), a stochastic gradient variational Bayes approach is proposed for the highly parallelizable posterior inference that provides computational flexibility for modeling high-dimensional time series. The accurate empirical performance of the proposed method is illustrated via extensive simulation studies and the analysis of two datasets: a wind speed data from 6 locations in California, and a 61-channel electroencephalogram data recorded on two contrasting subjects under specific experimental conditions.

On sparse high-dimensional graphical model learning for dependent time series

We consider the problem of inferring the conditional independence graph (CIG) of a sparse, high-dimensional stationary multivariate Gaussian time series. A sparse-group lasso-based frequency-domain formulation of the problem based on frequency-domain sufficient statistic for the observed time series is presented. We investigate an alternating direction method of multipliers (ADMM) approach for optimization of the sparse-group lasso penalized log-likelihood. We provide sufficient conditions for convergence in the Frobenius norm of the inverse PSD estimators to the true value, jointly across all frequencies, where the number of frequencies are allowed to increase with sample size. This result also yields a rate of convergence. We also empirically investigate selection of the tuning parameters based on the Bayesian information criterion, and illustrate our approach using numerical examples utilizing both synthetic and real data.

Detecting departures from meta-ellipticity for multivariate stationary time series

Abstract A test for detecting departures from meta-ellipticity for multivariate stationary time series is proposed. The large sample behavior of the test statistic is shown to depend in a complicated way on the underlying copula as well as on the serial dependence. Valid asymptotic critical values are obtained by a bootstrap device based on subsampling. The finite-sample performance of the test is investigated in a large-scale simulation study, and the theoretical results are illustrated by a case study involving financial log returns.

OLS Estimation of Markov switching VAR models: asymptotics and application to energy use

We show that the ordinary least squares (OLS) estimates of population parameters for Markov switching vector autoregressive (MS VAR) models coincide with the maximum likelihood estimates. Then, we propose an algorithm in matrix form for the estimation of model parameters, and derive an explicit expression in closed-form for the asymptotic covariance matrix of the OLS estimator of such models. The obtained characterization of the asymptotic variance is new to our knowledge. It is easier to program than the usual approach based on second derivatives, and more accurate. Our theorems generalize the classical results known for a linear VAR process, and complete those existing in the literature on the estimation of the asymptotic covariance matrix for multivariate stationary time series. Numerical simulations are provided to illustrate the obtained theoretical results. Finally, an application on energy use and economic growth in the Euro area gives some insights on the nonlinear nature of the corresponding time series, and reproduces the major stylized facts.

New HSIC-based tests for independence between two stationary multivariate time series

This paper proposes some novel one-sided omnibus tests for independence between two multivariate stationary time series. These new tests apply the Hilbert-Schmidt independence criterion (HSIC) to test the independence between the innovations of both time series. Under regular conditions, the limiting null distributions of our HSIC-based tests are established. Next, our HSIC-based tests are shown to be consistent. Moreover, a residual bootstrap method is used to obtain the critical values for our HSIC-based tests, and its validity is justified. Compared with the existing cross-correlation-based tests for linear dependence, our tests examine the general (including both linear and non-linear) dependence to give investigators more complete information on the causal relationship between two multivariate time series. The merits of our tests are illustrated by some simulation results and a real example.

Intrinsic Wavelet Regression for Curves of Hermitian Positive Definite Matrices

Intrinsic wavelet transforms and wavelet estimation methods are introduced for curves in the non-Euclidean space of Hermitian positive definite matrices, with in mind the application to Fourier spectral estimation of multivariate stationary time series. The main focus is on intrinsic average-interpolation wavelet transforms in the space of positive definite matrices equipped with an affine-invariant Riemannian metric, and convergence rates of linear wavelet thresholding are derived for intrinsically smooth curves of Hermitian positive definite matrices. In the context of multivariate Fourier spectral estimation, intrinsic wavelet thresholding is equivariant under a change of basis of the time series, and nonlinear wavelet thresholding is able to capture localized features in the spectral density matrix across frequency, always guaranteeing positive definite estimates. The finite-sample performance of intrinsic wavelet thresholding is assessed by means of simulated data and compared to several benchmark estimators in the Riemannian manifold. Further illustrations are provided by examining the multivariate spectra of trial-replicated brain signal time series recorded during a learning experiment. Supplementary materials for this article are available online.

Edge Exclusion Tests for Graphical Model Selection: Complex Gaussian Vectors and Time Series

We consider the problem of inferring the conditional independence graph (CIG) of a stationary multivariate Gaussian time series. A $p$ -variate Gaussian time series graphical model associated with an undirected graph with $p$ vertices is defined as the family of time series that obey the conditional independence restrictions implied by the edge set of the graph. In some existing methods, partial coherence has been used as a test statistic for graphical model selection. To test inclusion/exclusion of a given edge in the graph, the test is applied at distinct frequencies, requiring multiple tests and leading to a loss in test power. The nonparametric method of Matsuda uses the Kullback-Leibler divergence measure to define a test statistic which does not need multiple testing to test between two competing models. In this paper we propose a generalized likelihood ratio test (GLRT) based edge exclusion test statistic that also does not need multiple testing. It is computationally significantly faster than the method of Matsuda, and simulations show that we achieve comparable power levels. Our proposed approach is based on a novel formulation of a GLRT based edge exclusion test for $p$ -variate complex Gaussian graphical models (CGGMs); this result is also of independent interest. The computational complexity of the proposed statistic is $\mathcal {O}(p^3)$ compared to $\mathcal {O}(p^5)$ for the existing result. We also apply our time series graphical model selection method to a foreign exchange data set consisting of monthly trends of foreign exchange rates of 10 countries.

Prediction intervals and regions for multivariate time series models with sieve bootstrap

In the paper, the construction of unconditional bootstrap prediction intervals and regions for some class of second order stationary multivariate linear time series models is considered. Our approach uses the sieve bootstrap procedure introduced by Kreiss 1992 and Bühlmann 1997. Basic theoretical results concerning consistency of the bootstrap replications and the bootstrap prediction regions are proved. We present a simulation study comparing the proposed bootstrap methods with the Box–Jenkins approach.

Reconstructing missing data sequences in multivariate time series: an application to environmental data

Missing data arise in many statistical analyses, due to faults in data acquisition, and can have a significant effect on the conclusions that can be drawn from the data. In environmental data, for example, a standard approach usually adopted by the Environmental Protection Agencies to handle missing values is by deleting those observations with incomplete information from the study, obtaining a massive underestimation of many indexes usually used for evaluating air quality. In multivariate time series, moreover, it may happen that not only isolated values but also long sequences of some of the time series’ components may miss. In such cases, it is quite impossible to reconstruct the missing sequences basing on the serial dependence structure alone. In this work, we propose a new procedure that aims to reconstruct the missing sequences by exploiting the spatial correlation and the serial correlation of the multivariate time series, simultaneously. The proposed procedure is based on a spatial-dynamic model and imputes the missing values in the time series basing on a linear combination of the neighbor contemporary observations and their lagged values. It is specifically oriented to spatio-temporal data, although it is general enough to be applied to generic stationary multivariate time-series. In this paper, the procedure has been applied to the pollution data, where the problem of missing sequences is of serious concern, with remarkably satisfactory performance.

Multivariate integer-valued time series with flexible autocovariances and their application to major hurricane counts

This paper examines a bivariate count time series with some curious statistical features: Saffir–Simpson Category 3 and stronger annual hurricane counts in the North Atlantic and eastern Pacific Ocean Basins. As land and ocean temperatures on our planet warm, an intense climatological debate has arisen over whether hurricanes are becoming more numerous, or whether the strengths of the individual storms are increasing. Recent literature concludes that an increase in hurricane counts occurred in the Atlantic Basin circa 1994. This increase persisted through 2012; moreover, the 1994–2012 period was one of relative inactivity in the eastern Pacific Basin. When Atlantic activity eased in 2013, heavy activity in the eastern Pacific Basin commenced. When examined statistically, a Poisson white noise model for the annual severe hurricane counts is difficult to resoundingly reject. Yet, decadal cycles (longer term dependence) in the hurricane counts is plausible. This paper takes a statistical look at the issue, developing a stationary multivariate count time series model with Poisson marginal distributions and a flexible autocovariance structure. Our auto- and cross-correlations can be negative and have long-range dependence; features that most previous count models cannot achieve in tandem. Our model is new in the literature and is based on categorizing and super-positioning multivariate Gaussian time series. We derive the autocovariance function of the model and propose a method to estimate model parameters. In the end, we conclude that severe hurricane counts are indeed negatively correlated across the two ocean basins. Some evidence for long-range dependence is also presented; however, with only a 49-year record, this issue cannot be definitively judged without additional data.

Martingale Difference Divergence Matrix and Its Application to Dimension Reduction for Stationary Multivariate Time Series

ABSTRACTIn this article, we introduce a new methodology to perform dimension reduction for a stationary multivariate time series. Our method is motivated by the consideration of optimal prediction and focuses on the reduction of the effective dimension in conditional mean of time series given the past information. In particular, we seek a contemporaneous linear transformation such that the transformed time series has two parts with one part being conditionally mean independent of the past. To achieve this goal, we first propose the so-called martingale difference divergence matrix (MDDM), which can quantify the conditional mean independence of V ∈ Rp given U ∈ Rq and also encodes the number and form of linear combinations of V that are conditional mean independent of U. Our dimension reduction procedure is based on eigen-decomposition of the cumulative martingale difference divergence matrix, which is an extension of MDDM to the time series context. Interestingly, there is a static factor model representation for our dimension reduction framework and it has subtle difference from the existing static factor model used in the time series literature. Some theory is also provided about the rate of convergence of eigenvalue and eigenvector of the sample cumulative MDDM in the fixed-dimensional setting. Favorable finite sample performance is demonstrated via simulations and real data illustrations in comparison with some existing methods. Supplementary materials for this article are available online.

Copula-Based Factor Models for Multivariate Asset Returns

Recently, several copula-based approaches have been proposed for modeling stationary multivariate time series. All of them are based on vine copulas, and they differ in the choice of the regular vine structure. In this article, we consider a copula autoregressive (COPAR) approach to model the dependence of unobserved multivariate factors resulting from two dynamic factor models. However, the proposed methodology is general and applicable to several factor models as well as to other copula models for stationary multivariate time series. An empirical study illustrates the forecasting superiority of our approach for constructing an optimal portfolio of U.S. industrial stocks in the mean-variance framework.

Estimación Espectral de Series de Tiempo de Absorbancia Uv-Vis para el Monitoreo de Calidad de Aguas

Context: Signals recorded as multivariate time series by UV-Vis absorbance captors installed in urban sewer systems, can be non-stationary, yielding complications in the analysis of water quality monitoring. This work proposes to perform spectral estimation using the Box-Cox transformation and differentiation in order to obtain stationary multivariate time series in a wide sense. Additionally, Principal Component Analysis (PCA) is applied to reduce their dimensionality.Method: Three different UV-Vis absorbance time series for different Colombian locations were studied: (i) El-Salitre Wastewater Treatment Plant (WWTP) in Bogotá; (ii) Gibraltar Pumping Station (GPS) in Bogotá; and (iii) San-Fernando WWTP in Itagüí. Each UV-Vis absorbance time series had equal sample number (5705). The esti-mation of the spectral power density is obtained using the average of modified periodograms with rectangular window and an overlap of 50%, with the 20 most important harmonics from the Discrete Fourier Transform (DFT) and Inverse Fast Fourier Transform (IFFT).Results: Absorbance time series dimensionality reduction using PCA, resulted in 6, 8 and 7 principal components for each study site respectively, altogether explaining more than 97% of their variability. Values of differences below 30% for the UV range were obtained for the three study sites, while for the visible range the maximum differences obtained were: (i) 35% for El-Salitre WWTP; (ii) 61% for GPS; and (iii) 75% for San-Fernando WWTP.Conclusions: The Box-Cox transformation and the differentiation process applied to the UV-Vis absorbance time series for the study sites (El-Salitre, GPS and San-Fernando), allowed to reduce variance and to eliminate ten-dency of the time series. A pre-processing of UV-Vis absorbance time series is recommended to detect and remove outliers and then apply the proposed process for spectral estimation.Language: Spanish.

Equivalent Dynamic Models

ABSTRACTEquivalences of two classes of dynamic models for weakly stationary multivariate time series are discussed: dynamic factor models and autoregressive models. It is shown that exploratory dynamic factor models can be rotated, yielding an infinite set of equivalent solutions for any observed series. It also is shown that dynamic factor models with lagged factor loadings are not equivalent to the currently popular state-space models, and that restriction of attention to the latter type of models may yield invalid results. The known equivalent vector autoregressive model types, standard and structural, are given a new interpretation in which they are conceived of as the extremes of an innovating type of hybrid vector autoregressive models. It is shown that consideration of hybrid models solves many problems, in particular with Granger causality testing.