Restricted Isometry Property for General p-Norms

Zeyuan Allen-Zhu,Ilya Razenshteyn,Rati Gelashvili

doi:10.1109/tit.2016.2598296

Zeyuan Allen-Zhu, Ilya Razenshteyn + Show 1 more

Open Access

https://doi.org/10.1109/tit.2016.2598296

Copy DOI

Abstract

The restricted isometry property (RIP) is a fundamental property of a matrix, which enables sparse recovery. Informally, an $m \times n$ matrix satisfies RIP of order $k$ for the $\ell _{p}$ norm, if $\|Ax\|_{p} \approx \|x\|_{p}$ for every vector $x$ with at most $k$ non-zero coordinates. For every $1 \leq p , we obtain almost tight bounds on the minimum number of rows $m$ necessary for the RIP property to hold. Prior to this paper, only the cases $p = 1$ , $1 + 1 / \log n$ , and 2 were studied. Interestingly, our results show that the case $p = 2$ is a singularity point: the optimal number of rows $m$ is $\widetilde {\Theta }(k^{p})$ for all $p\in [1,\infty )\setminus \{2\}$ , as opposed to $\widetilde {\Theta }(k)$ for $p=2$ . We also obtain almost tight bounds for the column sparsity of RIP matrices and discuss the implications of our results for the stable sparse recovery problem as defined by Candes et al.

Highlights

The main object of our interest is a matrix with Restricted Isometry Property for the p norm (RIP-p)
In this work we investigate the following question: given p ∈ [1, ∞), n ∈ N, k ∈ [n], and D > 1, What is the smallest m ∈ N so that there exists a (k, D)-RIP-p matrix A ∈ Rm×n?
On the dimension side, for every p ∈ (1, ∞) \ {2}, distortion D > 1, and (k, D)-RIP-p matrix A ∈ Rm×n, we show that m = Ω(kp), where Ω(·) hides factors that depend on p and D

Summary

Introduction

The main object of our interest is a matrix with Restricted Isometry Property for the p norm (RIP-p). An m × n matrix A ∈ Rm×n is said to have (k, D)-RIP-p property for sparsity k ∈ [n] =def {1, . N}, distortion D > 1, and the p norm for p ∈ [1, ∞), if for every vector x ∈ Rn with at most k non-zero coordinates one has x p ≤ Ax p ≤ D · x p. In this work we investigate the following question: given p ∈ [1, ∞), n ∈ N, k ∈ [n], and D > 1, What is the smallest m ∈ N so that there exists a (k, D)-RIP-p matrix A ∈ Rm×n?. The following question arises naturally from the complexity of computing Ax: What is the smallest column sparsity d for such a (k, D)-RIP-p matrix A ∈ Rm×n?. Leibniz International Proceedings in Informatics Schloss Dagstuhl – Leibniz-Zentrum für Informatik, Dagstuhl Publishing, Germany

Motivation

Our results

Upper bounds

Lower bounds

The Lower-Tail Step

The Upper-Tail Step