On the Characterization of <formula formulatype="inline"><tex Notation="TeX">$\ell _{p}$</tex></formula>-Compressible Ergodic Sequences

Abstract

This work offers a necessary and sufficient condition for a stationary and ergodic process to be $\ell _{p}$ -compressible in the sense proposed by Amini, Unser and Marvasti [“Compressibility of deterministic and random infinity sequences,” IEEE Trans. Signal Process., vol. 59, no. 11, pp. 5193–5201, 2011, Def. 6]. The condition reduces to check that the $p$ -moment of the invariant distribution of the process is well defined, which contextualizes and extends the result presented by Gribonval, Cevher and Davies in [“Compressible distributions for high-dimensional statistics,” IEEE Trans. Inf. Theory, vol. 58, no. 8, pp. 5016–5034, 2012, Prop. 1]. Furthermore, for the scenario of non- $\ell _{p}$ -compressible ergodic sequences, we provide a closed-form expression for the best $k$ -term relative approximation error (in the $\ell _{p}$ -norm sense) when only a fraction (rate) of the most significant sequence coefficients are kept as the sequence-length tends to infinity. We analyze basic properties of this rate-approximation error curve, which is again a function of the invariant measure of the process. Revisiting the case of i.i.d. sequences, we completely identify the family of $\ell _{p}$ -compressible processes, which reduces to look at a polynomial order decay (heavy-tail) property of the distribution.