Sparse covariance estimation based on sparse-graph codes

Abstract

We consider the problem of recovering a sparse covariance matrix Σ∈ℝn×n from m quadratic measurements yi = aiTΣai+wi, 1 ≤ i ≤ m, where ai ∈ ln is a measurement vector and wi is additive noise. We assume that ℝ has K non-zero off-diagonal entries. We first consider the simplified noiseless problem where wi = 0 for all i. We introduce two low complexity algorithms, the first a “message-passing” algorithm and the second a “forward” algorithm, that are based on a sparse-graph coding framework. We show that under some simplifying assumptions, the message passing algorithm can recover an arbitrarily-large fraction of the K non-zero components with cK measurements, where c is a small constant that can be precisely characterized. As one instance, the message passing algorithm can recover, with high probability, a fraction 1 − 10−4 of the non-zero components, using only m = 6K quadratic measurements, which is a small constant factor from the fundamental limit, with an optimal O(K) decoding complexity. We further show that the forward algorithm can recover all the K non-zero entries with high probability with m = Θ(K) measurements and O(K log(K)) decoding complexity. However, the forward algorithm suffers from significantly larger constants in terms of the number of required measurements, and is indeed less practical despite providing stronger theoretical guarantees. We then consider the noisy setting, and show that both proposed algorithms can be robustified to noise with m = Θ(K log2(n)) measurements. Finally, we provide extensive simulation results that support our theoretical claims.