Order statistics of vectors with dependent coordinates, and the Karhunen–Loève basis

Abstract

Let $X$ be an $n$-dimensional random centered Gaussian vector with independent but not identically distributed coordinates and let $T$ be an orthogonal transformation of $\mathbb{R}^{n}$. We show that the random vector $Y=T(X)$ satisfies \begin{equation*}\mathbb{E}\sum_{j=1}^{k}j\mbox{-}\min_{i\leq n}{X_{i}}^{2}\leq C\mathbb{E}\sum_{j=1}^{k}j\mbox{-}\min_{i\leq n}{Y_{i}}^{2}\end{equation*} for all $k\leq n$, where “$j\mbox{-}\min$” denotes the $j$th smallest component of the corresponding vector and $C>0$ is a universal constant. This resolves (up to a multiplicative constant) an old question of S. Mallat and O. Zeitouni regarding optimality of the Karhunen–Loeve basis for the nonlinear signal approximation. As a by-product, we obtain some relations for order statistics of random vectors (not only Gaussian) which are of independent interest.