Autocovariance Matrices Research Articles

Characteristic value decomposition: A unifying paradigm for data-driven modal analysis

Data-driven modal analysis is an indispensable means to understand the dynamic behavior of engineered structures and natural systems. It effectively captures active dynamic behavior and embeds them in a set of identified linear modal subspaces, unveiling complex dynamic behavior via the superposition of simpler hierarchical spatiotemporal dynamics. Despite the existence of many methods, there is no unified perspective connecting these modal analysis techniques. This paper proposes a new modal decomposition method, the characteristic value decomposition (CVD), that provides a unifying paradigm for data-driven modal analysis. CVD identifies identical spatial modes and temporal coordinates from two measurement data matrices. The modal parameters are identified through an eigendecomposition to a quotient of auto-covariance and cross-covariance matrices of the two data matrices. This paper shows that several data-driven modal analysis methods, namely, the state-variable decomposition, the dynamic mode decomposition, the Ibrahim time domain method, and the eigensystem realization algorithm, are special cases of CVD when the two data matrices are appropriately selected. Moreover, the CVD identifies the modal parameters with higher accuracy and is robust to added noise compared to established methods, as demonstrated using numerical experiments. It is also shown that the CVD flexibly utilizes data from multiple experiments and undersampled data sets to unveil system dynamics.

Improved multiple-Toeplitz matrices reconstruction method using quadratic spatial smoothing for coherent signals DOA estimation

PurposeThe purpose of this paper is to exploit the multiple-Toeplitz matrices reconstruction method combined with quadratic spatial smoothing processing to improve the direction-of-arrival (DOA) estimation performance of coherent signals at low signal-to-noise ratio (SNRs).Design/methodology/approachAn improved multiple-Toeplitz matrices reconstruction method is proposed via quadratic spatial smoothing processing. Our proposed method takes advantage of the available information contained in the auto-covariance matrices of individual Toeplitz matrices and the cross-covariance matrices of different Toeplitz matrices, which results in a higher noise suppression ability.FindingsTheoretical analysis and simulation results show that, compared with the existing Toeplitz matrix processing methods, the proposed method improves the DOA estimation performance in cases with a low SNR. Especially for the cases with a low SNR and small snapshot number as well as with closely spaced sources, the proposed method can achieve much better performance on estimation accuracy and resolution probability.Research limitations/implicationsThe study investigates the possibility of reusing pre-existing designs for the DOA estimation of the coherent signals. The proposed technique enables achieve good estimation performance at low SNRs.Practical implicationsThe paper includes implications for the DOA problem at low SNRs in communication systems.Originality/valueThe proposed method proved to be useful for the DOA estimation at low SNR.

Estimation and inference in a high-dimensional semiparametric Gaussian copula vector autoregressive model

This paper develops simple, robust estimation and inference methods for the transition matrix of a high-dimensional semiparametric Gaussian copula vector autoregressive process. Our estimator is based on rank estimators of the large variance and auto-covariance matrices of a transformed latent high-dimensional Gaussian process. We derive rates of convergence of our estimator based on which we develop de-biased inference for Granger causality. Numerical results demonstrate the efficacy of the proposed methods. Although our focus is on the observable process, by the nature of rank estimators, all the methods developed directly apply to the transformed latent process. In technical terms, our analysis relies heavily on newly developed exponential inequalities for (degenerate) U-statistics under α-mixing condition.

Generalized autocovariance matrices for multivariate time series

The paper treats the modeling of stationary multivariate stochastic processes via frequency domain, and extends the notion of generalized autocovariance function, given by Proietti and Luati (2015) for univariate time series, to the multivariate setting. The generalized autocovariance matrices are defined for stationary multivariate stochastic processes as the Fourier transform of the power transformation of the spectral density matrix. Then we prove the consistency and derive the asymptotic distribution of frequency domain non-parametric estimators of the generalized autocovariance matrices, based on the power transformation of the periodogram matrix. Generalized autocovariance matrices are used to construct white noise hypothesis testing, to discriminate stochastic processes, and to introduce a generalized Yule–Walker estimator for the spectrum. A so-called λ–squared distance between two multivariate stochastic processes is also defined by using their generalized autocovariance matrices, and it serves for clustering time series and estimation by feature matching. Another use is in discriminant analysis.

On eigenvalue distributions of large autocovariance matrices

In this article, we establish a limiting distribution for eigenvalues of a class of autocovariance matrices. The same distribution has been found in the literature for a regularized version of these autocovariance matrices. The original nonregularized autocovariance matrices are noninvertible, thus introducing supplementary difficulties for the study of their eigenvalues through Girko’s Hermitization scheme. The key result in this paper is a new polynomial lower bound for a specific family of least singular values associated to a rank-defective quadratic function of a random matrix with independent and identically distributed entries. Another innovation from the paper is that the lag of the autocovariance matrices can grow to infinity with the matrix dimension.

Spectral measure of empirical autocovariance matrices of high-dimensional Gaussian stationary processes

Consider the empirical autocovariance matrices at given non-zero time lags, based on observations from a multivariate complex Gaussian stationary time series. The spectral analysis of these autocovariance matrices can be useful in certain statistical problems, such as those related to testing for white noise. We study the behavior of their spectral measure in the asymptotic regime where the time series dimension and the observation window length both grow to infinity, and at the same rate. Following a general framework in the field of the spectral analysis of large random non-Hermitian matrices, at first the probabilistic behavior of the small singular values of a shifted version of the autocovariance matrix is obtained. This is then used to obtain the asymptotic behavior of the empirical spectral measure of the autocovariance matrices at any lag. Matrix orthogonal polynomials on the unit circle play a crucial role in our study.

Autoregressive mixture models for clustering time series

Clustering time series into similar groups can improve models by combining information across like time series. While there is a well developed body of literature for clustering of time series, these approaches tend to generate clusters independently of model training, which can lead to poor model fit. We propose a novel distributed approach that simultaneously clusters and fits autoregression models for groups of similar individuals. We apply a Wishart mixture model so as to cluster individuals while modelling the corresponding autocovariance matrices at the same time. The fitted Wishart scale matrices map to cluster‐level autoregressive coefficients through the Yule–Walker equations, fitting robust parsimonious autoregressive mixture models. This approach is able to discern differences in underlying autocorrelation variation of time series in settings with large heterogeneous datasets. We prove consistency of our cluster membership estimator, asymptotic distributions of coefficients and compare our approach against competing methods through simulation as well as by fitting a COVID‐19 forecast model.

On the effect of probing noise in optimal control LQR via Q-learning using adaptive filtering algorithms

The probing noise is essential to obtain the persistence of excitation condition for solving Bellman equations in optimal control problems based on reinforcement learning. However, the level of probing noise affects the system state and the input control signal. The Bellman equation solution can be obtained using the classical adaptive filtering algorithms, namely, normalized least-mean-square (NLMS) and recursive-least-squares (RLS); thus, taking into account this kind of solution, in this paper, we present an analysis on the effect of probing noise in the state and input of the dynamic system, considering the linear quadratic regulator (LQR) problem using Q-learning policy iteration. In our analysis, a closed formula for the autocovariance matrices of the system state and the input are obtained, showing that their norms are proportional to the variance of the probing noise during the learning process. Numerical experiments show the performance of Q-learning policy iteration method based on NLMS and RLS for different variances of probing noise.

A nonstationary and non‐Gaussian moving average model for solar irradiance

AbstractHistorically, power has flowed from large power plants to customers. Increasing penetration of distributed energy resources such as solar power from rooftop photovoltaic has made the distribution network a two‐way‐street with power being generated at the customer level. The incorporation of renewables introduces additional uncertainty and variability into the power grid. Distribution network operation studies are being adapted to include renewables; however, such studies require high quality solar irradiance data that adequately reflect realistic meteorological variability. Data from satellite‐based products are spatially complete, but temporally coarse, whereas solar irradiances exhibit high frequency variation at very fine timescales. We propose a new stochastic method for temporally downscaling global horizontal irradiance (GHI) to 1 min resolution, but we do not consider the spatial aspect due to limited availability of the in situ irradiance measurements. Solar irradiance's first and second‐order structures vary diurnally and seasonally, and our model adapts to such nonstationarity. Empirical irradiance data exhibits highly non‐Gaussian behavior; we develop a nonstationary and non‐Gaussian moving average model that is shown to capture realistic solar variability at multiple timescales. We also propose a new estimation scheme based on Cholesky factors of empirical autocovariance matrices, bypassing difficult and inaccessible likelihood‐based approaches. The model is demonstrated for a case study of three locations that are located in diverse climates through the United States. The model is compared against competitors from the literature and is shown to provide better uncertainty and variability quantification on testing data.

Large sample autocovariance matrices of linear processes with heavy tails

We provide asymptotic theory for certain functions of the sample autocovariance matrices of a high-dimensional time series with infinite fourth moment. The time series exhibits linear dependence across the coordinates and through time. Assuming that the dimension increases with the sample size, we provide theory for the eigenvectors of the sample autocovariance matrices and find explicit approximations of a simple structure, whose finite sample quality is illustrated for simulated data. We also obtain the limits of the normalized eigenvalues of functions of the sample autocovariance matrices in terms of cluster Poisson point processes. In turn, we derive the distributional limits of the largest eigenvalues and functionals acting on them. In our proofs, we use large deviation techniques for heavy-tailed processes, point process techniques motivated by extreme value theory, and related continuous mapping arguments.

Estimation of autocovariance matrices for high dimensional linear processes

In this paper under some mild restrictions upper bounds on the rate of convergence for estimators of ptimes p autocovariance and precision matrices for high dimensional linear processes are given. We show that these estimators are consistent in the operator norm in the sub-Gaussian case when p={mathcal {O}}left( n^{gamma /2}right) for some gamma >1, and in the general case when p^{2/beta }(n^{-1} log p)^{1/2}rightarrow 0 for some beta >2 as p=p(n)rightarrow infty and the sample size nrightarrow infty . In particular our results hold for multivariate AR processes. We compare our results with those previously obtained in the literature for independent and dependent data. We also present non-asymptotic bounds for the error probability of these estimators.

On the behavior of large empirical autocovariance matrices between the past and the future

The asymptotic behavior of the distribution of the squared singular values of the sample autocovariance matrix between the past and the future of a high-dimensional complex Gaussian uncorrelated sequence is studied. Using Gaussian tools, it is established that the distribution behaves as a deterministic probability measure whose support [Formula: see text] is characterized. It is also established that the squared singular values are almost surely located in a neighborhood of [Formula: see text].

Generalised cepstral models for the spectrum of vector time series

The paper treats the modeling of stationary multivariate stochastic processes via a frequency domain model expressed in terms of cepstrum theory. The proposed model nests the vector exponential model of [20] as a special case, and extends the generalised cepstral model of [36] to the multivariate setting, answering a question raised by the last authors in their paper. Contemporarily, we extend the notion of generalised autocovariance function of [35] to vector time series. Then we derive explicit matrix formulas connecting generalised cepstral and autocovariance matrices of the process, and prove the consistency and asymptotic properties of the Whittle likelihood estimators of model parameters. Asymptotic theory for the special case of the vector exponential model is a significant addition to the paper of [20]. We also provide a mathematical machinery, based on matrix differentiation, and computational methods to derive our results, which differ significantly from those employed in the univariate case. The utility of the proposed model is illustrated through Monte Carlo simulation from a bivariate process characterized by a high dynamic range, and an empirical application on time varying minimum variance hedge ratios through the second moments of future and spot prices in the corn commodity market.

Joint convergence of sample autocovariance matrices when $p/n\to 0$ with application

Consider a high-dimensional linear time series model where the dimension $p$ and the sample size $n$ grow in such a way that $p/n\to 0$. Let $\hat{\Gamma }_{u}$ be the $u$th order sample autocovariance matrix. We first show that the LSD of any symmetric polynomial in $\{\hat{\Gamma }_{u},\hat{\Gamma }_{u}^{*},u\geq 0\}$ exists under independence and moment assumptions on the driving sequence together with weak assumptions on the coefficient matrices. This LSD result, with some additional effort, implies the asymptotic normality of the trace of any polynomial in $\{\hat{\Gamma }_{u},\hat{\Gamma }_{u}^{*},u\geq 0\}$. We also study similar results for several independent MA processes. We show applications of the above results to statistical inference problems such as in estimation of the unknown order of a high-dimensional MA process and in graphical and significance tests for hypotheses on coefficient matrices of one or several such independent processes.

On testing for high-dimensional white noise

Testing for white noise is a classical yet important problem in statistics, especially for diagnostic checks in time series modeling and linear regression. For high-dimensional time series in the sense that the dimension $p$ is large in relation to the sample size $T$, the popular omnibus tests including the multivariate Hosking and Li–McLeod tests are extremely conservative, leading to substantial power loss. To develop more relevant tests for high-dimensional cases, we propose a portmanteau-type test statistic which is the sum of squared singular values of the first $q$ lagged sample autocovariance matrices. It, therefore, encapsulates all the serial correlations (up to the time lag $q$) within and across all component series. Using the tools from random matrix theory and assuming both $p$ and $T$ diverge to infinity, we derive the asymptotic normality of the test statistic under both the null and a specific VMA(1) alternative hypothesis. As the actual implementation of the test requires the knowledge of three characteristic constants of the population cross-sectional covariance matrix and the value of the fourth moment of the standardized innovations, nontrivial estimations are proposed for these parameters and their integration leads to a practically usable test. Extensive simulation confirms the excellent finite-sample performance of the new test with accurate size and satisfactory power for a large range of finite $(p,T)$ combinations, therefore, ensuring wide applicability in practice. In particular, the new tests are consistently superior to the traditional Hosking and Li–McLeod tests.

Instrumental Variables Estimation of Systems of Simultaneous Equations: Interrelation of Methods

The article is devoted to the interrelation between methods of estimating parameters of simultaneous equations. Simultaneous equations model (SEM) is commonly used to model complex socio-economic phenomena. SEM is a set of linear simultaneous equations in which response variables are among explanatory variables in each equation of regression. This causes the problem of endogeneity and leads to biased and inconsistent estimation of parameters. There is a number of special methods to solve the problem of endogeneity of regressors: method of instrumental variables (IV), indirect least squares method (ILS), two-stage least squares method (2SLS), and three-stage least squares method (3SLS). In this article, the relationship between 2SLS and IV, ILS and 2SLS, ILS and OLS with restrictions on structural parameters, as well as the equivalence of point estimates of parameters and autocovariance matrices, is shown using empirical example.

Large Covariance and Autocovariance Matrices, By Arup Bose and Monika Bhattacharjee. Published by Taylor & Francis Group, LLC, Boca Raton, London, New York, 2019. ISBN: 9781138303867 (HARDBACK)

Large Covariance and Autocovariance Matrices, By Arup Bose and Monika Bhattacharjee. Published by Taylor & Francis Group, LLC, Boca Raton, London, New York, 2019. ISBN: 9781138303867 (HARDBACK)

Moment Bounds for Large Autocovariance Matrices Under Dependence

The goal of this paper is to obtain expectation bounds for the deviation of large sample autocovariance matrices from their means under weak data dependence. While the accuracy of covariance matrix estimation corresponding to independent data has been well understood, much less is known in the case of dependent data. We make a step toward filling this gap and establish deviation bounds that depend only on the parameters controlling the “intrinsic dimension” of the data up to some logarithmic terms. Our results have immediate impacts on high-dimensional time-series analysis, and we apply them to high-dimensional linear VAR(d) model, vector-valued ARCH model, and a model used in Banna et al. (Random Matrices Theory Appl 5(2):1650006, 2016).

Necessary and Sufficient Conditions for the Existence of Robust Whitening Matrices

In blind source separation (BSS), many methods use positive definite symmetric pencil either to achieve the whitening procedure of the data or to estimate the mixing matrix. All these methods consist in finding a positive definite linear combination (PDLC) of several time delayed autocovariance matrices. Many algorithms have been developed to compute the coefficients of such a linear combination. However, in BSS the problem of existence of such a linear combination has never been addressed with a rigorous way. In this letter, we exhibit a necessary and sufficient condition for the existence of such a PDLC. Moreover, we determine sufficient condition for the existence of such a PDLC in order to guide the user to choose efficiently the set of time-lags autcovariance matrices instead of choosing it intuitively. A brief numerical study is performed, in order to validate theoretical results and to show how to fix the set of time-lags.

Large Covariance and Autocovariance Matrices; Patterned Random Matrices

Large Covariance and Autocovariance Matrices; Patterned Random Matrices