New Asymptotic Results in Principal Component Analysis

Abstract

Let X be a mean zero Gaussian random vector in a separable Hilbert space ${\mathbb H}$ with covariance operator ${\Sigma }:={\mathbb E}(X\otimes X).$ Let ${\Sigma }={\sum }_{r\geq 1}\mu _{r} P_{r}$ be the spectral decomposition of Σ with distinct eigenvalues $\mu _{1}>\mu _{2}> \dots $ and the corresponding spectral projectors $P_{1}, P_{2}, \dots .$ Given a sample $X_{1},\dots , X_{n}$ of size n of i.i.d. copies of X, the sample covariance operator is defined as $\hat {\Sigma }_{n} := n^{-1}{\sum }_{j=1}^{n} X_{j}\otimes X_{j}.$ The main goal of principal component analysis is to estimate spectral projectors $P_{1}, P_{2}, \dots $ by their empirical counterparts $\hat P_{1}, \hat P_{2}, \dots $ properly defined in terms of spectral decomposition of the sample covariance operator $\hat {\Sigma }_{n}.$ The aim of this paper is to study asymptotic distributions of important statistics related to this problem, in particular, of statistic $\|\hat P_{r}-P_{r}\|_{2}^{2},$ where $\|\cdot \|_{2}^{2}$ is the squared Hilbert–Schmidt norm. This is done in a “high-complexity” asymptotic framework in which the so called effective rank $\textbf {r}({\Sigma }):=\frac {\text {tr}({\Sigma })}{\|{\Sigma }\|_{\infty }}$ (tr(⋅) being the trace and $\|\cdot \|_{\infty }$ being the operator norm) of the true covariance Σ is becoming large simultaneously with the sample size n, but r(Σ) = o(n) as $n\to \infty .$ In this setting, we prove that, in the case of one-dimensional spectral projector P r , the properly centered and normalized statistic $\|\hat P_{r}-P_{r}\|_{2}^{2}$ with data-dependent centering and normalization converges in distribution to a Cauchy type limit. The proofs of this and other related results rely on perturbation analysis and Gaussian concentration.