A Universal Expectation Bound on Empirical Projections of Deformed Random Matrices

Abstract

Let $C$ be a real-valued $M\times M$ matrix with singular values $\lambda_1\ge...\ge\lambda_M$ and $E$ a random matrix of centered i.i.d. entries with finite fourth moment. In this paper we give a universal upper bound on the expectation of $||\hat\pi_rX||_{S_2}^2-||\pi_rX||^2_{S_2}$, where $X:=C+E$ and $\hat\pi_r$ (resp. $\pi_r$) is a rank-$r$ projection maximizing the Hilbert-Schmidt norm $||\tilde\pi_rX||_{S_2}$ (resp. $||\tilde\pi_rC||_{S_2}$) over the set $\S_{M,r}$ of all orthogonal rank-$r$ projections. This result is a generalization of a theorem for Gaussian matrices due to Rohde (2012). Our approach differs substantially from the techniques of the mentioned article. We analyze $||\hat\pi_rX||_{S_2}^2-||\pi_rX||^2_{S_2}$ from a rather deterministic point of view by an upper bound on $||\hat\pi_rX||_{S_2}^2-||\pi_rX||^2_{S_2}$, whose randomness is totally determined by the largest singular value of $E$.