Autocovariance Matrices Research Articles

Banded spatio-temporal autoregressions

We propose a new class of spatio-temporal models with unknown and banded autoregressive coefficient matrices. The setting represents a sparse structure for high-dimensional spatial panel dynamic models when panel members represent economic (or other type) individuals at many different locations. The structure is practically meaningful when the order of panel members is arranged appropriately. Note that the implied autocovariance matrices are unlikely to be banded, and therefore, the proposal is radically different from the existing literature on the inference for high-dimensional banded covariance matrices. Due to the innate endogeneity, we apply the least squares method based on a Yule–Walker equation to estimate autoregressive coefficient matrices. The estimators based on multiple Yule–Walker equations are also studied. A ratio-based method for determining the bandwidth of autoregressive matrices is also proposed. Some asymptotic properties of the inference methods are established. The proposed methodology is further illustrated using both simulated and real data sets.

A Durbin–Levinson regularized estimator of high-dimensional autocovariance matrices

We consider the problem of estimating the high-dimensional autocovariance matrix of a stationary random process, with the purpose of out of sample prediction and feature extraction. This problem has received several solutions. In the nonparametric framework, the literature has concentrated on banding and tapering the sample autocovariance matrix. This paper proposes and evaluates an alternative approach, based on regularizing the sample partial autocorrelation function, via a modified Durbin-Levinson algorithm that receives as input the banded and tapered partial autocorrelations and returns a sample autocovariance sequence which is positive definite. We show that the regularized estimator of the autocovariance matrix is consistent and its convergence rates is established. We then focus on constructing the optimal linear predictor and we assess its properties. The computational complexity of the estimator is of the order of the square of the banding parameter, which renders our method scalable for high-dimensional time series. The performance of the autocovariance estimator and the corresponding linear predictor is evaluated by simulation and empirical applications.

Spatial Weight Determination of GSTAR(1;1) Model by Using Kernel Function

The stochastic process models with the index parameters such as time and location were investigated in this paper. The model used was GSTAR (1;1), and it was applied to the Gamma ray log data. The important thing to be assessed in this model is the determination of the space weight matrix. Commonly, the spatial weight matrix was determined based on the Euclidean distance, but not based on data. In this work, we use the kernel function approach to determine the spatial weighting function whose domain was in the form of data observation. In addition, we also study the influence of this weight matrix to the stationary condition of GSTAR (1;1) model, and we use the inverse of auto-covariance matrices or IAcM methods. The results showed that the kernel weights matrix approach still being met influence on stationary of this model.

A Jacobi Generalized Orthogonal Joint Diagonalization Algorithm for Joint Blind Source Separation

Joint blind source separation (J-BSS) has emerged as a data-driven technique for multi-set data fusion applications. In this paper, we propose a Jacobi generalized orthogonal joint diagonalization (GOJD) algorithm for J-BSS of multiset signals. By the use of second-order statistics, we can obtain multiple sets of auto-covariance and cross-covariance matrices from the multi-set signals, which together admit a GOJD formulation. For computing the GOJD, we propose a computationally efficient Jacobi algorithm, which uses a sequence of Givens rotations to simultaneously diagonalize the covariance matrices. In comparison with other GOJD algorithms, the proposed algorithm is shown to have fast convergence. Moreover, as the optimal Givens rotation matrix in each update is calculated in closed-form, this algorithm is computationally very efficient. In the application aspect, we have considered the scenario where different data sets in J-BSS may have different number of sources, among which there exist both similar components that are consistently present in multiple data sets, and diverse components that are uniquely present in each data set. We have shown how J-BSS based on the proposed GOJD algorithm can effectively extract both similar and diverse source components. Simulation results are given to show the nice performance of the proposed algorithm, with regards to both speed and accuracy, in comparison with other algorithms of similar type.

Banded Spatio-Temporal Autoregressions

Abstract We propose a new class of spatio-temporal models with unknown and banded autoregressive coefficient matrices. The setting represents a sparse structure for high-dimensional spatial panel dynamic models when panel members represent economic (or other type) individuals at many different locations. The structure is practically meaningful when the order of panel members is arranged appropriately. Note that the implied autocovariance matrices are unlikely to be banded, and therefore, the proposal is radically different from the existing literature on the inference for high-dimensional banded covariance matrices. Due to the innate endogeneity, we apply the least squares method based on a Yule–Walker equation to estimate autoregressive coefficient matrices. The estimators based on multiple Yule–Walker equations are also studied. A ratio-based method for determining the bandwidth of autoregressive matrices is also proposed. Some asymptotic properties of the inference methods are established. The proposed methodology is further illustrated using both simulated and real data sets.

Spectral analysis of sample autocovariance matrices of a class of linear time series in moderately high dimensions

Author(s): Wang, L; Aue, A; Paul, D | Abstract: This article is concerned with the spectral behavior of p-dimensional linear processes in the moderately high-dimensional case when both dimensionality p and sample size n tend to infinity so that p/n→0. It is shown that, under an appropriate set of assumptions, the empirical spectral distributions of the renormalized and symmetrized sample autocovariance matrices converge almost surely to a nonrandom limit distribution supported on the real line. The key assumption is that the linear process is driven by a sequence of pdimensional real or complex random vectors with i.i.d. entries possessing zero mean, unit variance and finite fourth moments, and that the p × p linear process coefficient matrices are Hermitian and simultaneously diagonalizable. Several relaxations of these assumptions are discussed. The results put forth in this paper can help facilitate inference on model parameters, model diagnostics and prediction of future values of the linear process.

Autocovariance-based MPC model mismatch estimation for systems with measurable disturbances

In this paper, we propose a novel autocovariance-based plant-model mismatch estimation approach for linear MPC MIMO control loops with changing setpoints and measurable disturbances. Assuming a noise model is available and that there are of periods of operating data where the active set of the controller is fixed and the plant-model mismatch is invariant, we establish an explicit relation between the autocovariance matrices of the mean-centered process outputs and the plant-model mismatch. We then formulate the mismatch estimation problem as an optimization aimed at minimizing the difference between the theoretical autocovariance, computed from the established relation, and actual output autocovariances, calculated from the plant data. We elaborate our results for step-response models typically used in MPC, as well as for parametric (transfer function models) in both continuous time and discrete time. A simulation case study for an unconstrained MPC controller with measurable disturbances is used to illustrate the theoretical results.

Eigenvalues and eigenvectors of heavy-tailed sample covariance matrices with general growth rates: The iid case

In this paper we study the joint distributional convergence of the largest eigenvalues of the sample covariance matrix of a p-dimensional time series with iid entries when p converges to infinity together with the sample size n. We consider only heavy-tailed time series in the sense that the entries satisfy some regular variation condition which ensures that their fourth moment is infinite. In this case, Soshnikov (2004, 2006) and Auffinger et al. (2009) proved the weak convergence of the point processes of the normalized eigenvalues of the sample covariance matrix towards an inhomogeneous Poisson process which implies in turn that the largest eigenvalue converges in distribution to a Fréchet distributed random variable. They proved these results under the assumption that p and n are proportional to each other. In this paper we show that the aforementioned results remain valid if p grows at any polynomial rate. The proofs are different from those in Auffinger et al. (2009) and Soshnikov (2004, 2006); we employ large deviation techniques to achieve them. The proofs reveal that only the diagonal of the sample covariance matrix is relevant for the asymptotic behavior of the largest eigenvalues and the corresponding eigenvectors which are close to the canonical basis vectors. We also discuss extensions of the results to sample autocovariance matrices.

High-dimensional and banded vector autoregressions

We consider a class of vector autoregressive models with banded coefficient matrices. The setting represents a type of sparse structure for high-dimensional time series, though the implied autocovariance matrices are not banded. The structure is also practically meaningful when the order of component time series is arranged appropriately. The convergence rates for the estimated banded autoregressive coefficient matrices are established. We also propose a Bayesian information criterion for determining the width of the bands in the coefficient matrices, which is proved to be consistent. By exploring some approximate banded structure for the autocovariance functions of banded vector autoregressive processes, consistent estimators for the auto-covariance matrices are constructed.

Blind Source Separation of Event-Related EEG/MEG.

Blind source separation (BSS) can be used to decompose complex electroencephalography (EEG) or magnetoencephalography data into simpler components based on statistical assumptions without using a physical model. Applications include brain-computer interfaces, artifact removal, and identifying parallel neural processes. We wish to address the issue of applying BSS to event-related responses, which is challenging because of nonstationary data. We introduce a new BSS approach called momentary-uncorrelated component analysis (MUCA), which is tailored for event-related multitrial data. The method is based on approximate joint diagonalization of multiple covariance matrices estimated from the data at separate latencies. We further show how to extend the methodology for autocovariance matrices and how to apply BSS methods suitable for piecewise stationary data to event-related responses. We compared several BSS approaches by using simulated EEG as well as measured somatosensory and transcranial magnetic stimulation (TMS) evoked EEG. Among the compared methods, MUCA was the most tolerant one to noise, TMS artifacts, and other challenges in the data. With measured somatosensory data, over half of the estimated components were found to be similar by MUCA and independent component analysis. MUCA was also stable when tested with several input datasets. MUCA is based on simple assumptions, and the results suggest that MUCA is robust with nonideal data. Event-related responses and BSS are valuable and popular tools in neuroscience. Correctly designed BSS is an efficient way of identifying artifactual and neural processes from nonstationary event-related data.

Factor Modelling for High‐Dimensional Time Series: Inference and Model Selection

Analysis of high‐dimensional time series data is of increasing interest among different fields. This article studies high‐dimensional time series from a dimension reduction perspective using factor modelling. Statistical inference is conducted using eigen‐analysis of a certain non‐negative definite matrix related to autocovariance matrices of the time series, which is applicable to fixed or increasing dimension. When the dimension goes to infinity, the rate of convergence and limiting distributions of estimated factors are established. Using the limiting distributions of estimated factors, a high‐dimensional final prediction error criterion is proposed to select the number of factors. Asymptotic properties of the criterion are illustrated by simulation studies and real applications.

Estimation of inverse autocovariance matrices for long memory processes

This work aims at estimating inverse autocovariance matrices of long memory processes admitting a linear representation. A modified Cholesky decomposition is used in conjunction with an increasing order autoregressive model to achieve this goal. The spectral norm consistency of the proposed estimate is established. We then extend this result to linear regression models with long-memory time series errors. In particular, we show that when the objective is to consistently estimate the inverse autocovariance matrix of the error process, the same approach still works well if the estimated (by least squares) errors are used in place of the unobservable ones. Applications of this result to estimating unknown parameters in the aforementioned regression model are also given. Finally, a simulation study is performed to illustrate our theoretical findings.

Optimal trading strategies—a time series approach

Motivated by recent advances in the spectral theory of auto-covariance matrices, we are led to revisit a reformulation of Markowitz’ mean-variance portfolio optimization approach in the time domain. In its simplest incarnation it applies to a single traded asset and allows an optimal trading strategy to be found which—for a given return—is minimally exposed to market price fluctuations. The model is initially investigated for a range of synthetic price processes, taken to be either second order stationary, or to exhibit second order stationary increments. Attention is paid to consequences of estimating auto-covariance matrices from small finite samples, and auto-covariance matrix cleaning strategies to mitigate against these are investigated. Finally we apply our framework to real world data.

A more efficient second order blind identification method for separation of uncorrelated stationary time series

The classical second order source separation methods use approximate joint diagonalization of autocovariance matrices with several lags to estimate the unmixing matrix. Based on recent asymptotic results, we propose a novel unmixing matrix estimator which selects the best lag set from a finite set of candidate sets specified by the user. The theory is illustrated by a simulation study.

Extreme value analysis for the sample autocovariance matrices of heavy-tailed multivariate time series

We provide some asymptotic theory for the largest eigenvalues of a sample covariance matrix of a p-dimensional time series where the dimension p = p_n converges to infinity when the sample size n increases. We give a short overview of the literature on the topic both in the light- and heavy-tailed cases when the data have finite (infinite) fourth moment, respectively. Our main focus is on the heavytailed case. In this case, one has a theory for the point process of the normalized eigenvalues of the sample covariance matrix in the iid case but also when rows and columns of the data are linearly dependent. We provide limit results for the weak convergence of these point processes to Poisson or cluster Poisson processes. Based on this convergence we can also derive the limit laws of various function als of the ordered eigenvalues such as the joint convergence of a finite number of the largest order statistics, the joint limit law of the largest eigenvalue and the trace, limit laws for successive ratios of ordered eigenvalues, etc. We also develop some limit theory for the singular values of the sample autocovariance matrices and their sums of squares. The theory is illustrated for simulated data and for the components of the S&P 500 stock index.

Large sample behaviour of high dimensional autocovariance matrices

The existence of limiting spectral distribution (LSD) of $\hat{\Gamma}_u+\hat{\Gamma}_u^*$, the symmetric sum of the sample autocovariance matrix $\hat{\Gamma}_u$ of order $u$, is known when the observations are from an infinite dimensional vector linear process with appropriate (strong) assumptions on the coefficient matrices. Under significantly weaker conditions, we prove, in a unified way, that the LSD of any symmetric polynomial in these matrices such as $\hat{\Gamma}_u+\hat{\Gamma}_u^*$, $\hat{\Gamma}_u\hat{\Gamma}_u^*$, $\hat{\Gamma}_u\hat{\Gamma}_u^*+\hat{\Gamma}_k\hat{\Gamma}_k^*$ exist. Our approach is through the more intuitive algebraic method of free probability in conjunction with the method of moments. Thus, we are able to provide a general description for the limits in terms of some freely independent variables. All the previous results follow as special cases. We suggest statistical uses of these LSD and related results in order determination and white noise testing.

Optimal Trading Strategies A Time Series Approach

Motivated by recent advances in the spectral theory of auto-covariance matrices, we are led to revisit a reformulation of Markowitz’ mean-variance portfolio optimization approach in the time domain. In its simplest incarnation it applies to a single traded asset and allows to find an optimal trading strategy which — for a given return — is minimally exposed to market price fluctuations. The model is initially investigated for a range of synthetic price processes, taken to be either second order stationary, or to exhibit second order stationary increments. Attention is paid to consequences of estimating auto-covariance matrices from small finite samples, and auto-covariance matrix cleaning strategies to mitigate against these are investigated. Finally we apply our framework to real world data.

Discussion of “Estimating structured high-dimensional covariance and precision matrices: Optimal rates and adaptive estimation”

In this discussion, we present a brief overview of recent works on the behavior of summary statistics for high-dimensional observations that are time-dependent, and the inference on parameters associated with high-dimensional time series, with emphasis on covariance and auto-covariance matrices.

Separation of Uncorrelated Stationary time series using Autocovariance Matrices

In blind source separation, one assumes that the observed p time series are linear combinations of p latent uncorrelated weakly stationary time series. To estimate the unmixing matrix, which transforms the observed time series back to uncorrelated latent time series, second‐order blind identification (SOBI) uses joint diagonalization of the covariance matrix and autocovariance matrices with several lags. In this article, we find the limiting distribution of the well‐known symmetric SOBI estimator under general conditions and compare its asymptotical efficiencies to those of the recently introduced deflation‐based SOBI estimator. The theory is illustrated by some finite‐sample simulation studies.

On the Marčenko–Pastur law for linear time series

This paper is concerned with extensions of the classical Mar\v{c}enko-Pastur law to time series. Specifically, $p$-dimensional linear processes are considered which are built from innovation vectors with independent, identically distributed (real- or complex-valued) entries possessing zero mean, unit variance and finite fourth moments. The coefficient matrices of the linear process are assumed to be simultaneously diagonalizable. In this setting, the limiting behavior of the empirical spectral distribution of both sample covariance and symmetrized sample autocovariance matrices is determined in the high-dimensional setting $p/n\to c\in (0,\infty)$ for which dimension $p$ and sample size $n$ diverge to infinity at the same rate. The results extend existing contributions available in the literature for the covariance case and are one of the first of their kind for the autocovariance case.