Finite Fourth Moment Research Articles

Large sample correlation matrices with unbounded spectrum

In this paper, we demonstrate that the diagonal of a high-dimensional sample covariance matrix stemming from n independent observations of a p-dimensional time series with finite fourth moments can be approximated in spectral norm by the diagonal of the population covariance matrix regardless of the spectral norm of the population covariance matrix. Our assumptions involve p and n tending to infinity, with p/n tending to a constant which might be positive or zero. Consequently, we investigate the asymptotic properties of the sample correlation matrix with a divergent spectrum, and we explore its applications by deriving the limiting spectral distribution for its eigenvalues and analyzing the convergence of divergent and non-divergent spiked eigenvalues under a generalized spiked correlation framework.

Central limit theorem in disordered Monomer‐Dimer model

AbstractWe consider the disordered monomer‐dimer model on general finite graphs with bounded degrees. Under the finite fourth moment assumption on the weight distributions, we prove a Gaussian central limit theorem for the free energy of the associated Gibbs measure with a rate of convergence. The central limit theorem continues to hold under a nearly optimal finite ‐moment assumption on the weight distributions if the underlying graphs are further assumed to have a uniformly subexponential volume growth. This generalizes a recent result by Dey and Krishnan who showed a Gaussian central limit theorem in the disordered monomer‐dimer model on cylinder graphs. Our proof relies on the idea that the disordered monomer‐dimer model exhibits a decay of correlation with high probability. We also establish a central limit theorem for the Gibbs average of the number of dimers where the underlying graph has subexponential volume growth and the edge weights are Gaussians.

On the sieve M-estimation for a special bilinear time series model with time-functional variance noises

. The bilinear model has been frequently used in fields like control theory and economics to model seismic data. In this article, time-functional variance (TFV) noises are embedded into a specific bilinear model. We propose a generalized autoregressive conditional heteroskedasticity-type maximum likelihood estimator (GMLE) with a sieve method and then provide various inferential techniques based on this GMLE. It is shown that under the finite fourth moment of errors, the GMLE is consistent and asymptotically normally distributed. A simulation study and analysis of real data are additionally carried out to evaluate GMLE’s finite sample performance.

On the adaptive Lasso estimator of AR(p) time series with applications to INAR(p) and Hawkes processes

We investigate the consistency and the rate of convergence of the adaptive Lasso estimator for the parameters of linear AR(p) time series with a white noise which is a strictly stationary and ergodic martingale difference. Roughly speaking, we prove that (i) If the white noise has a finite second moment, then the adaptive Lasso estimator is almost sure consistent (ii) If the white noise has a finite fourth moment, then the error estimate converges to zero with the same rate as the regularizing parameters of the adaptive Lasso estimator. Such theoretical findings are applied to estimate the parameters of INAR(p) time series and to estimate the fertility function of Hawkes processes. The results are validated by some numerical simulations, which show that the adaptive Lasso estimator allows for a better balancing between bias and variance with respect to the Conditional Least Square estimator and the classical Lasso estimator.

Local Marchenko–Pastur law at the hard edge of the sample covariance ensemble

Consider an N by N matrix X of complex entries with iid real and imaginary parts. We show that the local density of eigenvalues of X*X converges to the Marchenko–Pastur law on the optimal scale with probability 1. We also obtain rigidity of the eigenvalues in the bulk and near both hard and soft edges. Here we avoid logarithmic and polynomial corrections by working directly with high powers of expectation of the Stieltjes transforms. We work under the assumption that the entries have a finite fourth moment and are truncated at N1/4, or alternatively with exploding moments. In this work we simplify and adapt the methods from prior papers of Götze–Tikhomirov [Probab. Relat. Fields 165(1–2), 163–233 (2016)] and Cacciapuoti–Maltsev–Schlein [Probab. Theory Relat. Fields 163(1–2), 1–59 (2015)] to covariance matrices.

Finite rank perturbations of heavy-tailed Wigner matrices

One-rank perturbations of Wigner matrices have been closely studied: let [Formula: see text] with [Formula: see text] symmetric, [Formula: see text] i.i.d. with centered standard normal distributions, and [Formula: see text] It is well known [Formula: see text] the largest eigenvalue of [Formula: see text] has a phase transition at [Formula: see text]: when [Formula: see text] [Formula: see text] whereas for [Formula: see text] [Formula: see text] Under more general conditions, the limiting behavior of [Formula: see text] appropriately normalized, has also been established: it is normal if [Formula: see text] or the convolution of the law of [Formula: see text] and a Gaussian distribution if [Formula: see text] is concentrated on one entry. These convergences require a finite fourth moment, and this paper considers situations violating this condition. For symmetric distributions [Formula: see text] heavy-tailed with index [Formula: see text] the fluctuations are shown to be universal and dependent on [Formula: see text] but not on [Formula: see text] whereas a subfamily of the edge case [Formula: see text] displays features of both the light- and heavy-tailed regimes: two limiting laws emerge and depend on whether [Formula: see text] is localized, each presenting a continuous phase transition at [Formula: see text] respectively. These results build on our previous work which analyzes the asymptotic behavior of [Formula: see text] in the aforementioned subfamily.

On Estimating Rank-One Spiked Tensors in the Presence of Heavy Tailed Errors

In this paper, we study the estimation of a rank-one spiked tensor in the presence of heavy tailed noise. Our results highlight some of the fundamental similarities and differences in the tradeoff between statistical and computational efficiencies under heavy tailed and Gaussian noise. In particular, we show that, for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> th order tensors, the tradeoff manifests in an identical fashion as the Gaussian case when the noise has finite <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4(p-1)$ </tex-math></inline-formula> th moment. The difference in signal strength requirements, with or without computational constraints, for us to estimate the singular vectors at the optimal rate, interestingly, narrows for noise with heavier tails and vanishes when the noise only has finite fourth moment. Moreover, if the noise has less than fourth moment, tensor SVD, perhaps the most natural approach, is suboptimal even though it is computationally intractable. Our analysis exploits a close connection between estimating the rank-one spikes and the spectral norm of a random tensor with iid entries. In particular, we show that the order of the spectral norm of a random tensor can be precisely characterized by the moment of its entries, generalizing classical results for random matrices. In addition to the theoretical guarantees, we propose estimation procedures for the heavy tailed regime, which are easy to implement and efficient to run. Numerical experiments are presented to demonstrate their practical merits.

Asymptotics of AIC, BIC and Cp model selection rules in high-dimensional regression

Variable selection in multivariate linear regression is essential for the interpretation, subsequent statistical inferences and predictions of the statistical problem at hand. It has a long history of being studied, and many regressor selection criteria have been proposed. Most commonly used criteria include the Akaike information criterion (AIC), Bayesian information criterion (BIC), and Mallow’s Cp and their modifications. It is well-known that if the true model is among the candidate models, then BIC is strongly consistent while AIC is not when only the sample size tends to infinity and the numbers of response variables and regressors remain fixed; a setting often described as large-sample. Increasingly, more and more datasets are viewed as high-dimensional in the sense that the number of response variables (p), the number of regressors (k) and the sample size (n) tend to infinity such that p∕n→c∈(0,1) and k∕n→α∈[0,1) with α+c<1. A few recent works reported that, under high dimension, the asymptotic properties of AIC, BIC and Cp selection rules in the large-sample setting do not necessarily carry over in the high-dimensional setting. In this paper, we clarify their asymptotic properties and provide sufficient conditions for which a selection rule is strongly consistent, almost surely under specify and over specify a true model. We do not assume normality in the errors, and we only require finite fourth moment. The main tool employed is random matrix theory techniques. A consequence of this work states that, under certain mild high-dimensional conditions, if the BIC selection rule is strongly consistent then the AIC selection rule is also strongly consistent, but not vice versa. This result is in stark contrast to the large-sample result.

Empirical spectral processes for stationary state space models

In this paper, we consider function-indexed normalized weighted integrated periodograms for equidistantly sampled multivariate continuous-time state space models which are multivariate continuous-time ARMA processes. Thereby, the sampling distance is fixed and the driving Lévy process has at least a finite fourth moment. Under different assumptions on the function space and the moments of the driving Lévy process we derive a central limit theorem for the function-indexed normalized weighted integrated periodogram. Either the assumption on the function space or the assumption on the existence of moments of the Lévy process is weaker. Furthermore, we show the weak convergence in both the space of continuous functions and in the dual space to a Gaussian process and give an explicit representation of the covariance function. The results can be used to derive the asymptotic behavior of the Whittle estimator and to construct goodness-of-fit test statistics as the Grenander–Rosenblatt statistic and the Cramér–von Mises statistic. We present the exact limit distributions of both statistics and show their performance through a simulation study.

Robust parameter estimation of regression models under weakened moment assumptions

This paper provides some extended results on estimating parameter matrix of several regression models when the covariate or response possesses weaker moment condition. We study the M-estimator of Fan et al. (2021) for matrix completion model with (1+ϵ)-th moment noise. The corresponding phase transition phenomenon is observed. When 1>ϵ>0, the robust estimator possesses a slower convergence rate compared with previous literature. For high dimensional multiple index coefficient model, we propose an improved estimator via applying the element-wise truncation method to handle heavy-tailed data with finite fourth moment. The extensive simulation study validates our theoretical results.

On random matrices arising in deep neural networks: General I.I.D. case

We study the eigenvalue distribution of random matrices pertinent to the analysis of deep neural networks. The matrices resemble the product of the sample covariance matrices, however, an important difference is that the analog of the population covariance matrix is now a function of random data matrices (synaptic weight matrices in the deep neural network terminology). The problem has been treated in recent work [J. Pennington, S. Schoenholz and S. Ganguli, The emergence of spectral universality in deep networks, Proc. Mach. Learn. Res. 84 (2018) 1924–1932, arXiv:1802.09979] by using the techniques of free probability theory. Since, however, free probability theory deals with population covariance matrices which are independent of the data matrices, its applicability in this case has to be justified. The justification has been given in [L. Pastur, On random matrices arising in deep neural networks: Gaussian case, Pure Appl. Funct. Anal. (2020), in press, arXiv:2001.06188] for Gaussian data matrices with independent entries, a standard analytical model of free probability, by using a version of the techniques of random matrix theory. In this paper, we use another version of the techniques to extend the results of [L. Pastur, On random matrices arising in deep neural networks: Gaussian case, Pure Appl. Funct. Anal. (2020), in press, arXiv:2001.06188] to the case where the entries of the data matrices are just independent identically distributed random variables with zero mean and finite fourth moment. This, in particular, justifies the mean field approximation in the infinite width limit for the deep untrained neural networks and the property of the macroscopic universality of random matrix theory in this case.

Robust covariance estimation with noisy high-frequency financial data

We propose consistent and efficient robust different time-scales estimators to mitigate the heavy-tail effect of high-frequency financial data. Our estimators are based on minimising the Huber loss function with a suitable threshold. We show these estimators are guaranteed to be robust to measurement noise of certain types and jumps. With only finite fourth moments of the observation log-price data, we develop the sub-Gaussian concentration of our estimators around the volatility. We conduct the simulation studies to show the finite sample performance of the proposed estimation methods. The simulation studies imply that our methods are also robust to financial data in the presence of jumps. Empirical studies demonstrate the practical relevance and advantages of our estimators.

Annealed inhomogeneities in random ferromagnets.

We consider spin models on complex networks frequently used to model social and technological systems. We study the annealed ferromagnetic Ising model for random networks with either independent edges (Erdős-Rényi) or prescribed degree distributions (configuration model). Contrary to many physical models, the annealed setting is poorly understood and behaves quite differently than the quenched system. In annealed networks with a fluctuating number of edges, the Ising model changes the degree distribution, an aspect previously ignored. For random networks with Poissonian degrees, this gives rise to three distinct annealed critical temperatures depending on the precise model choice, only one of which reproduces the quenched one. In particular, two of these annealed critical temperatures are finite even when the quenched one is infinite because then the annealed graph creates a giant component for all sufficiently small temperatures. We see that the critical exponents in the configuration model with deterministic degrees are the same as the quenched ones, which are the mean-field exponents if the degree distribution has finite fourth moment and power-law-dependent critical exponents otherwise. Remarkably, the annealing for the configuration model with random independent and identically distributed degrees washes away the universality class with power-law critical exponents.

Self-Weighted LSE and Residual-Based QMLE of ARMA-GARCH Models

This paper studies the self-weighted least squares estimator (SWLSE) of the ARMA model with GARCH noises. It is shown that the SWLSE is consistent and asymptotically normal when the GARCH noise does not have a finite fourth moment. Using the residuals from the estimated ARMA model, it is shown that the residual-based quasi-maximum likelihood estimator (QMLE) for the GARCH model is consistent and asymptotically normal, but if the innovations are asymmetric, it is not as efficient as that when the GARCH process is observed. Using the SWLSE and residual-based QMLE as the initial estimators, the local QMLE for ARMA-GARCH model is asymptotically normal via an one-step iteration. The importance of the proposed estimators is illustrated by simulated data and five real examples in financial markets.

Large sample correlation matrices: a comparison theorem and its applications

In this paper, we show that the diagonal of a high-dimensional sample covariance matrix stemming from n independent observations of a p-dimensional time series with finite fourth moments can be approximated in spectral norm by the diagonal of the population covariance matrix. We assume that n,p→∞ with p∕n tending to a constant which might be positive or zero. As applications, we provide an approximation of the sample correlation matrix R and derive a variety of results for its eigenvalues. We identify the limiting spectral distribution of R and construct an estimator for the population correlation matrix and its eigenvalues. Finally, the almost sure limits of the extreme eigenvalues of R in a generalized spiked correlation model are analyzed.

R-estimators in GARCH models: asymptotics and applications

Summary The quasi-maximum likelihood estimation is a commonly-used method for estimating the generalized autoregressive conditional heteroscedastic parameters. However, such estimators are sensitive to outliers and their asymptotic normality is proved under the finite fourth moment assumption on the underlying error distribution. In this paper, we propose a novel class of estimators of the generalized autoregressive conditional heteroscedastic parameters based on ranks of the residuals, called R-estimators, with the property that they are asymptotically normal under the existence of a finite $2+\delta$ moment of the errors and are highly efficient. We propose a fast algorithm for computing the R-estimators. Both real data analysis and simulations show the superior performance of the proposed estimators under the heavy-tailed and asymmetric distributions.

TEST FOR ZERO MEDIAN OF ERRORS IN AN ARMA–GARCH MODEL

Because the ARMA–GARCH model can generate data with some important properties such as skewness, heavy tails, and volatility persistence, it has become a benchmark model in analyzing financial and economic data. The commonly employed quasi maximum likelihood estimation (QMLE) requires a finite fourth moment for both errors and the sequence itself to ensure a normal limit. The self-weighted quasi maximum exponential likelihood estimation (SWQMELE) reduces the moment constraints by assuming that the errors and their absolute values have median zero and mean one, respectively. Therefore, it is necessary to test zero median of errors before applying the SWQMELE, as changing zero mean to zero median destroys the ARMA–GARCH structure. This paper develops an efficient empirical likelihood test without estimating the GARCH model but using the GARCH structure to reduce the moment effect. A simulation study confirms the effectiveness of the proposed test. The data analysis shows that some financial returns do not have zero median of errors, which cautions the use of the SWQMELE.

High-dimensional sphericity test by extended likelihood ratio

Testing sphericity of the covariance matrices has been an active part in contemporary statistics. In this paper, we put forward a new test procedure for high-dimensional sphericity test based on the likelihood ratio test (LRT). The proposed test broadens the applicability of LRT which fails when the dimension is larger than the sample size. Under general population with finite fourth moment, the test statistic is shown to be asymptotically normally distributed under the null hypothesis. When the alternative hypothesis is true, the limiting distribution of the test statistic is derived under the spiked model. Simulation studies reveal that the proposed test controls the Type I error rate very well and outperforms some well-known tests in terms of the empirical power in several examined situations.

Robust covariance estimation for high-dimensional compositional data with application to microbial communities analysis.

Microbial communities analysis is drawing growing attention due to the rapid development fire of high-throughput sequencing techniques nowadays. The observed data has the following typical characteristics: it is high-dimensional, compositional (lying in a simplex) and even would be leptokurtic and highly skewed due to the existence of overly abundant taxa, which makes the conventional correlation analysis infeasible to study the co-occurrence and co-exclusion relationship between microbial taxa. In this article, we address the challenges of covariance estimation for this kind of data. Assuming the basis covariance matrix lying in a well-recognized class of sparse covariance matrices, we adopt a proxy matrix known as centered log-ratio covariance matrix in the literature. We construct a Median-of-Means estimator for the centered log-ratio covariance matrix and propose a thresholding procedure that is adaptive to the variability of individual entries. By imposing a much weaker finite fourth moment condition compared with the sub-Gaussianity condition in the literature, we derive the optimal rate of convergence under the spectral norm. In addition, we also provide theoretical guarantee on support recovery. The adaptive thresholding procedure of the MOM estimator is easy to implement and gains robustness when outliers or heavy-tailedness exist. Thorough simulation studies are conducted to show the advantages of the proposed procedure over some state-of-the-arts methods. At last, we apply the proposed method to analyze a microbiome dataset in human gut.

Empirical likelihood test for the application of swqmele in fitting an arma‐garch model

Fitting an ARMA‐GARCH model has become a common practice in financial econometrics. Because the asymptotic normality of the quasi maximum likelihood estimation (QMLE) requires finite fourth moment for both errors and the sequence itself, self‐weighted quasi maximum exponential likelihood estimation (SWQMELE) has been proposed to reduce the moment constraints but requires the errors to have zero median instead of zero mean. Because changing zero mean to zero median destroys the ARMA‐GARCH structure and has a serious effect on skewed data, this article proposes an efficient empirical likelihood test for zero mean of errors in the application of SWQMELE to ensure that the model still concerns conditional mean. A simulation study confirms the good finite sample performance before applying the test to the US housing price indexes and financial returns for the study of comovement.