Autocovariance Matrices Research Articles

Discussion of “High-dimensional autocovariance matrices and optimal linear prediction”

In this note I provide some discussion on the paper by “High-dimensional auto covariance matrices and optimal linear prediction” by T. McMurry and D. Politis.

Limiting spectral distribution of sample autocovariance matrices

We show that the empirical spectral distribution (ESD) of the sample autocovariance matrix (ACVM) converges as the dimension increases, when the time series is a linear process with reasonable restriction on the coefficients. The limit does not depend on the distribution of the underlying driving i.i.d. sequence and its support is unbounded. This limit does not coincide with the spectral distribution of the theoretical ACVM. However, it does so if we consider a suitably tapered version of the sample ACVM. For banded sample ACVM the limit has unbounded support as long as the number of non-zero diagonals in proportion to the dimension of the matrix is bounded away from zero. If this ratio tends to zero, then the limit exists and again coincides with the spectral distribution of the theoretical ACVM. Finally, we also study the LSD of a naturally modified version of the ACVM which is not non-negative definite.

Inverse moment bounds for sample autocovariance matrices based on detrended time series and their applications

In this paper, we assume that observations are generated by a linear regression model with short- or long-memory dependent errors. We establish inverse moment bounds for kn-dimensional sample autocovariance matrices based on the least squares residuals (also known as the detrended time series), where kn≪n, kn→∞ and n is the sample size. These results are then used to derive the mean-square error bounds for the finite predictor coefficients of the underlying error process. Based on the detrended time series, we further estimate the inverse of the n-dimensional autocovariance matrix, Rn−1, of the error process using the banded Cholesky factorization. By making use of the aforementioned inverse moment bounds, we obtain the convergence of moments of the difference between the proposed estimator and Rn−1 under spectral norm.

ESTIMATION OF AUTOCOVARIANCE MATRICES FOR INFINITE DIMENSIONAL VECTOR LINEAR PROCESS

AbstractConsider an infinite dimensional vector linear process. Under suitable assumptions on the parameter space, we provide consistent estimators of the autocovariance matrices. In particular, under causality, this includes the infinite‐dimensional vector autoregressive (IVAR) process. In that case, we obtain consistent estimators for the parameter matrices. An explicit expression for the estimators is obtained for IVAR(1), under a fairly realistic parameter space. We also show that under some mild restrictions, the consistent estimator of the marginal large dimensional variance–covariance matrix has the same convergence rate as that in case of i.i.d. samples.

Macroeconomic forecasting and structural analysis through regularized reduced-rank regression

This paper proposes a strategy for detecting and imposing reduced-rank restrictions in medium vector autoregressive models. It is known that Canonical Correlation Analysis (CCA) does not perform well in this framework, because inversions of large covariance matrices are required. We propose a method that combines the richness of reduced-rank regression with the simplicity of naïve univariate forecasting methods. In particular, we suggest the usage of a proper shrinkage estimator of the autocovariance matrices that are involved in the computation of CCA, in order to obtain a method that is asymptotically equivalent to CCA, but numerically more stable in finite samples. Simulations and empirical applications document the merits of the proposed approach for both forecasting and structural analysis.

Deflation-based separation of uncorrelated stationary time series

In this paper we assume that the observed p time series are linear combinations of p latent uncorrelated weakly stationary time series. The problem is then to find an estimate for an unmixing matrix that transforms the observed time series back to uncorrelated time series. The so called SOBI (Second Order Blind Identification) estimate aims at a joint diagonalization of the covariance matrix and several autocovariance matrices with varying lags. In this paper, we propose a novel procedure that extracts the latent time series one by one. The limiting distribution of this deflation-based SOBI is found under general conditions, and we show how the results can be used for the comparison of estimates. The exact formula for the limiting covariance matrix of the deflation-based SOBI estimate is given for general multivariate MA(∞) processes. Finally, a whole family of estimates is proposed with the deflation-based SOBI as a special case, and the limiting properties of these estimates are found as well. The theory is widely illustrated by simulation studies.

Distributions for residual autocovariances in parsimonious periodic vector autoregressive models with applications

The asymptotic distribution of the residual autocovariance matrices in the class of periodic vector autoregressive time series models with structured parameterization is derived. Diagnostic checking with portmanteau test statistics represents a useful application of the result. Under the assumption that the periodic white noise process of the periodic vector autoregressive time series model is composed of independent random variables, we demonstrate that the finite sample distributions of the Hosking‐Li‐McLeod portmanteau test statistics can be approximated by those of weighted sums of independent chi‐square random variables. The quantiles of the asymptotic distribution can be computed using the Imhof algorithm or other exact methods. Thus, using the (single) chi‐square distribution for these test statistics appears inadequate in general, although it is often recommended in practice for diagnostic methods of that kind. A simulation study provides empirical evidence.

Macroeconomic Forecasting and Structural Analysis Through Regularized Reduced-Rank Regression

This paper proposes a strategy to detect and impose reduced-rank restrictions in medium vector autoregressive models. In this framework, it is known that Canonical Correlation Analysis (CCA) does not perform well because inversions of large covariance matrices are required. We propose a method that combines the richness of reduced-rank regression with the simplicity of naive univariate forecasting methods. In particular, we suggest to use a proper shrinkage estimator of the autocovariance matrices that are involved in the computation of CCA, thus obtaining a method that is asymptotically equivalent to CCA, but it is numerically more stable in finite samples. Simulations and empirical applications document the merits of the proposed approach both in forecasting and in structural analysis.

Spectra of empirical auto-covariance matrices

We compute spectral densities of large sample auto-covariance matrices of stationary stochastic processes at fixed ratio α = N/M of matrix dimension N and sample size M. We find a remarkable scaling relation which expresses the spectral density ρα(λ) of sample auto-covariance matrices for processes with correlations as a continuous superposition of copies of the spectral density ρ(0)α(λ) for a sequence of uncorrelated random variables at the same value of α, rescaled in terms of the Fourier transform of the true auto-covariance function. We also obtain a closed-form approximation for the scaling function ρ(0)α(λ). Our results are in excellent agreement with numerical simulations using auto-regressive processes, and processes exhibiting a power-law decay of correlations.

Performance Analysis of the Strong Uncorrelating Transformation in Blind Separation of Complex-Valued Sources

The strong uncorrelating transformation (SUT) is an effective tool for blind separation of complex-valued independent sources, commonly applied to the (spatial) sample autocovariance and pseudo-autocovariance matrices of the observed mixtures. In this work we analyze the performance (in terms of the resulting interference to source ratio (ISR)) of SUT-based blind separation of general wide-sense stationary complex-valued sources. Based on a small-errors analysis, we derive explicit expressions for the attainable ISR in terms of the temporal correlations and pseudo-correlations of the sources. We demonstrate by simulation that the empirical performance closely follows our analytic prediction at moderate to long observation lengths. An important implication ensuing from our analysis is that a rectilinear (“maximally improper”) source can be perfectly separated from all the other (nonrectilinear) sources.

Banded Regularization of Autocovariance Matrices in Application to Parameter Estimation and Forecasting of Time Series

SummaryThe paper addresses a ‘large p–small n’ problem in a time series framework and considers properties of banded regularization of an empirical autocovariance matrix of a time series process. Utilizing the banded autocovariance matrix enables us to fit a much longer auto-regressive AR(p) model to the observed data than typically suggested by the Akaike information criterion, while controlling how many parameters are to be estimated precisely and the level of accuracy. We present results on asymptotic consistency of banded autocovariance matrices under the Frobenius norm and provide a theoretical justification on optimal band selection by using cross-validation. Remarkably, the cross-validation loss function for banded prediction is related to the conditional mean-square prediction error and, thus, may be viewed as an alternative model selection criterion. The procedure proposed is illustrated by simulations and application to predicting the sea surface temperature index in the Niño 3.4 region.

Banded and tapered estimates for autocovariance matrices and the linear process bootstrap

We address the problem of estimating the autocovariance matrix of a stationary process. Under short range dependence assumptions, convergence rates are established for a gradually tapered version of the sample autocovariance matrix and for its inverse. The proposed estimator is formed by leaving the main diagonals of the sample autocovariance matrix intact while gradually down‐weighting off‐diagonal entries towards zero. In addition, we show the same convergence rates hold for a positive definite version of the estimator, and we introduce a new approach for selecting the banding parameter. The new matrix estimator is shown to perform well theoretically and in simulation studies. As an application, we introduce a new resampling scheme for stationary processes termed the linear process bootstrap (LPB). The LPB is shown to be asymptotically valid for the sample mean and related statistics. The effectiveness of the proposed methods are demonstrated in a simulation study.

Correlated Wishart ensembles and chaotic time series

We study the correlated Wishart ensembles in the context of time series analysis. We are interested in the statistics of eigenlevels, viz. variables associated with independent eigenmodes in the system. The motivation of this work is to study the effect of time series correlations on the Wishart ensembles. In this connection, we derive the level density and the two-point function for the correlated Wishart ensembles by using the binary correlation method. Using our analytic results we analyze spectra of autocovariance matrices derived from single variable stationary time series. We consider the stochastic time series of Gaussian variables with exponentially decaying correlations and time series of chaotic maps, viz. the Arnold map, the Standard map and the stadium billiard map. In both cases, correlated time series are encountered and analyzed under the framework of random matrix theory. It is shown that the eigenlevel statistics for the chaotic maps follow closely those of correlated Wishart ensembles. It is indicated that the presence of collective modes in the spectra of autocovariance matrices is related to the integrability of the system.

On multiplicative seasonal modelling for vector time series

Many time series encountered in real applications display seasonal behavior. In this paper, we consider multiplicative seasonal vectorial autoregressive moving average (SVARMA) models to describe seasonal vector time series. We discuss conditional maximum likelihood estimation of the model parameters, allowing them to satisfy general linear constraints. Having fitted a model, residual autocovariances (or autocorrelations) have been found useful in checking time series models. Consequently, we obtain the asymptotic distributions of the residual autocovariance matrices. As applications of these results, Portmanteau test statistics are proposed and their asymptotic distributions are studied. The finite-sample properties of the test statistics are evaluated using Monte Carlo experiments.

Estimation and model adequacy checking for multivariate seasonal autoregressive time series models with periodically varying parameters

We introduce a class of multivariate seasonal time series models with periodically varying parameters, abbreviated by the acronym SPVAR. The model is suitable for multivariate data, and combines a periodic autoregressive structure and a multiplicative seasonal time series model. The stationarity conditions (in the periodic sense) and the theoretical autocovariance functions of SPVAR stochastic processes are derived. Estimation and checking stages are considered. The asymptotic normal distribution of the least squares estimators of the model parameters is established, and the asymptotic distributions of the residual autocovariance and autocorrelation matrices in the class of SPVAR time series models are obtained. In order to check model adequacy, portmanteau test statistics are considered and their asymptotic distributions are studied. A simulation study is briefly discussed to investigate the finite‐sample properties of the proposed test statistics. The methodology is illustrated with a bivariate quarterly data set on travelers entering in to Canada.

On modelling and diagnostic checking of vector periodic autoregressive time series models

Abstract. Vector periodic autoregressive time series models (PVAR) form an important class of time series for modelling data derived from climatology, hydrology, economics and electrical engineering, among others. In this article, we derive the asymptotic distributions of the least squares estimators of the model parameters in PVAR models, allowing the parameters in a given season to satisfy linear constraints. Residual autocorrelations from classical vector autoregressive and moving‐average models have been found useful for checking the adequacy of a particular model. In view of this, we obtain the asymptotic distribution of the residual autocovariance matrices in the class of PVAR models, and the asymptotic distribution of the residual autocorrelation matrices is given as a corollary. Portmanteau test statistics designed for diagnosing the adequacy of PVAR models are introduced and we study their asymptotic distributions. The proposed test statistics are illustrated in a small simulation study, and an application with bivariate quarterly West German data is presented.

Do coincident indicators have one-factor structure?

The Stock–Watson coincident index of business cycles and its extensions assume a static linear one-factor model for the component indicators. This paper tests that assumption. Since the factor structure restricts the autocovariance matrices of the component indicators, a distance test, or the Hansen–Sargan test of over-identifying restrictions, is applicable. This also gives a GMM counterpart of the Stock–Watson coincident index, or a new composite index (CI) of coincident indicators, as a by-product. For the four US coincident indicators that currently make up the CI, the test strongly rejects the null hypothesis of static linear one-factor structure.

Diagnostic checking of multivariate nonlinear time series models with martingale difference errors

In this article, we derive the asymptotic distribution of residual autocovariance and autocorrelation matrices for a general class of multivariate nonlinear time series models by assuming only that the error term is a martingale difference sequence. Two types of applications are developed: global test statistics of the portmanteau type and one-lag test statistics, which describe the residual correlation at individual lags. To illustrate the proposed methodology, simulation results are reported for diagnosing multivariate threshold time series models. The following test statistics are compared: the classical test statistics presuming independent errors and the proposed methodology which supposes only martingale difference errors.

Modified [formula omitted] tests for assessing the multiple correlation between one spatial process and several others

The modified t test presented by Dutilleul [1993. Modifying the t test for assessing the correlation between two spatial processes. Biometrics 49, 305–314] to assess the simple correlation between two spatial processes is built on the sample correlation coefficient calculated from partial realizations, and requires the estimation of an effective sample size appropriately defined. The autocovariances of processes are taken into account via the effective sample size, both in the evaluation of the test statistic and in the calculation of the probability of significance. In this article, we present modified F tests to assess the multiple correlation between one spatial process and several (i.e., q ) others, from partial realizations collected at the same sampling locations. An effective sample size is used in all these modified F tests. Its theoretical expression is obtained from the expected value of the coefficient of determination in the absence of multiple correlation, under the bivariate Gaussian and first-order stationarity assumptions for the ‘dependent process’ and the ‘predicted process’ (i.e., the ordinary least-squares (OLS) regression predictor defined from the q other processes). The modified F tests presented differ in the procedure (i.e., model-based versus model-free) followed to estimate the autocovariance matrices appearing in the expression of the effective sample size. The validity and power of the new testing procedures are studied through simulations, for various sampling schemes and combinations of the autocovariance structure and number of sampling locations. An example with real data is given. In closing, we discuss the theoretical and practical aspects of our modified F tests in multiple correlation analysis, in comparison with those used in the repeated measures ANOVA.

On the asymptotic distribution of residual autocovariances in VARX models with applications

In this paper, we derive the asymptotic distribution of residual autocovariance matrices in the class of vector autoregressive models with explanatory variables (VARX). The asymptotic distribution of residual autocorrelation matrices is also obtained. New test statistics are proposed as an application of our main result. Test statistics at individual lags and portmanteau statistics are introduced and their asymptotic distributions are established. The proposed test, statistics are illustrated in a small simulation study.