Tyler's M-estimator Research Articles

Robust Self-Tuning Semiparametric PCA for Contaminated Elliptical Distribution

Principal component analysis (PCA) is one of the most popular dimension reduction methods. The usual PCA is known to be sensitive to the presence of outliers, and thus many robust PCA methods have been developed. Among them, the Tyler's M-estimator is shown to be the most robust scatter estimator under the elliptical distribution. However, when the underlying distribution is contaminated and deviates from ellipticity, Tyler's M-estimator might not work well. In this article, we apply the semiparametric theory to propose a robust semiparametric PCA. The merits of our proposal are twofold. First, it is robust to heavy-tailed elliptical distributions as well as robust to non-elliptical outliers. Second, it pairs well with a data-driven tuning procedure, which is based on active ratio and can adapt to different degrees of data outlyingness. Theoretical properties are derived, including the influence functions for various statistical functionals and asymptotic normality. Simulation studies and a data analysis demonstrate the superiority of our method.

On robust spectrum sensing using M-estimators of covariance matrix

In this paper, we consider the spectrum sensing in cognitive radio networks when the impulsive noise appears. We propose a class of blind and robust detectors using M-estimators in eigenvalue based spectrum sensing method. The conventional eigenvalue based method uses statistics derived from the eigenvalues of sample covariance matrix(SCM) as testing statistics, which are inefficient and unstable in the impulsive noise environment. Instead of SCM, we can use M-estimators, which have good performance under both impulsive and non-impulsive noise. Among those M-estimators, We recommend the Tyler's M-estimator instead, which requires no knowledge of noise distribution and have the same probability of false alarm under different complex elliptically symmetric distributions. In addition, it performs better than the detector using sample covariance matrix when the noise is highly impulsive. It should be emphasized that this detector does not require knowledge of noise power which is required by the energy detection based methods. Simulations show that it performs better than conventional detector using sample covariance matrix in a highly impulsive noise environment.

Gini covariance matrix and its affine equivariant version

We propose a new covariance matrix called Gini covariance matrix (GCM), which is a natural generalization of univariate Gini mean difference (GMD) to the multivariate case. The extension is based on the covariance representation of GMD by applying the multivariate spatial rank function. We study properties of GCM, especially in the elliptical distribution family. In order to gain the affine equivariance property for GCM, we utilize the transformation–retransformation (TR) technique and obtain an affine equivariant version GCM that turns out to be a symmetrized M-functional. The influence function of those two GCM’s are obtained and their estimation has been presented. Asymptotic results of estimators have been established. A closely related scatter Kotz functional and its estimator are also explored. Finally, asymptotical efficiency and finite sample efficiency of the TR version GCM are compared with those of sample covariance matrix, Tyler-M estimator and other scatter estimators under different distributions.

Robust subspace recovery by Tyler's M-estimator

This paper considers the problem of robust subspace recovery: given a set of N points in RD, if many lie in a d-dimensional subspace, then can we recover the underlying subspace? We show that Tyler’s M-estimator can be used to recover the underlying subspace, if the percentage of the inliers is larger than d/D and the data points lie in general position. Empirically, Tyler’s M-estimator compares favorably with other convex subspace recovery algorithms in both simulations and experiments on real data sets.

Robust Estimation of Structured Covariance Matrix for Heavy-Tailed Elliptical Distributions

This paper considers the problem of robustly estimating a structured covariance matrix with an elliptical underlying distribution with a known mean. In applications where the covariance matrix naturally possesses a certain structure, taking the prior structure information into account in the estimation procedure is beneficial to improving the estimation accuracy. We propose incorporating the prior structure information into Tyler's M-estimator and formulating the problem as minimizing the cost function of Tyler's estimator under the prior structural constraint. First, the estimation under a general convex structural constraint is introduced with an efficient algorithm for finding the estimator derived based on the majorization-minimization (MM) algorithm framework. Then, the algorithm is tailored to several special structures that enjoy a wide range of applications in signal processing related fields, namely, sum of rank-one matrices, Toeplitz, and banded Toeplitz structure. In addition, two types of non-convex structures, i.e., the Kronecker structure and the spiked covariance structure, are also discussed, where it is shown that simple algorithms can be derived under the guidelines of MM. The algorithms are guaranteed to converge to a stationary point of the problems. Furthermore, if the constraint set is geodesically convex, such as the Kronecker structure set, then the algorithm converges to a global minimum. Numerical results show that the proposed estimator achieves a smaller estimation error than the benchmark estimators at a lower computational cost.

Performance Analysis of Tyler's Covariance Estimator

This paper analyzes the performance of Tyler's M-estimator of the scatter matrix in elliptical populations. We focus on the non-asymptotic setting and derive the estimation error bounds depending on the number of samples n and the dimension p. We show that under quite mild conditions the squared Frobenius norm of the error of the inverse estimator decays like p^2/n with high probability.

On Multivariate Rank and Generalized Rank Regression

Multivariate regression estimates based on ranks and generalized ranks are proposed. These estimates are based on a transformation and retransformation technique that uses Tyler's (1987) M-estimator of scatter. The proposed estimates are obtained by retransforming the componentwise rank-based estimate due to Davis and McKean (1993) and a componentwise generalized rank estimate. Asymptotic properties of the estimates are established under some regularity conditions. It is shown that both estimates have a multivariate normal limiting distribution. The influence function of the retransformed generalized rank estimate has a bounded influence in both factor and response spaces. It is shown thro ugh a simulation study that the transformed-retransformed R and GR estimates are highly efficient compared to the componentwis e R, GR and least absolute deviations estimates. Also, it is s hown that the new estimates perform better than the least squares estimate when the errors have a heavy tailed distribution. An example illustrating the estimation procedures is presented.

The K-Step Spatial Sign Covariance Matrix

The Sign Covariance Matrix is an orthogonal equivariant estimator of multivariate scale. It is often used as an easy-to-compute and highly robust estimator. In this paper we propose a k-step version of the Sign Covariance Matrix, which improves its efficiency while keeping the maximal breakdown point. If k tends to infinity, Tyler's M-estimator is obtained. It turns out that even for very low values of k, one gets almost the same efficiency as Tyler's M-estimator.

Tyler's M-Estimator, Random Matrix Theory, and Generalized Elliptical Distributions with Applications to Finance

In recent publications standard methods of random matrix theory have been applied to principal components analysis of high-dimensional financial data. We discuss the fundamental results and potential shortcomings of random matrix theory in the light of the stylized facts of empirical finance. It is shown that the Marcenko-Pastur law generally fails when analyzing the empirical distribution function of the eigenvalues given by the sample covariance matrix of generalized elliptically distributed data. As an alternative we derive a random matrix referred to as the 'spectral estimator' which is distribution-free within the class of generalized elliptical distributions. We show that the spectral estimator corresponds to Tyler's M-estimator. Substituting the sample covariance matrix by the spectral estimator resolves the problems which are due to the stylized facts and the Marcenko-Pastur law remains valid. This holds even if the data are not generalized elliptically distributed but mutually independent.