Restricted Isometry Property Constant Research Articles

Geometric Analysis of Noisy Low-Rank Matrix Recovery in the Exact Parametrized and the Overparametrized Regimes

The matrix sensing problem is an important low-rank optimization problem that has found a wide range of applications, such as matrix completion, phase synchornization/retrieval, robust principal component analysis (PCA), and power system state estimation. In this work, we focus on the general matrix sensing problem with linear measurements that are corrupted by random noise. We investigate the scenario where the search rank r is equal to the true rank [Formula: see text] of the unknown ground truth (the exact parametrized case), as well as the scenario where r is greater than [Formula: see text] (the overparametrized case). We quantify the role of the restricted isometry property (RIP) in shaping the landscape of the nonconvex factorized formulation and assisting with the success of local search algorithms. First, we develop a global guarantee on the maximum distance between an arbitrary local minimizer of the nonconvex problem and the ground truth under the assumption that the RIP constant is smaller than [Formula: see text]. We then present a local guarantee for problems with an arbitrary RIP constant, which states that any local minimizer is either considerably close to the ground truth or far away from it. More importantly, we prove that this noisy, overparametrized problem exhibits the strict saddle property, which leads to the global convergence of perturbed gradient descent algorithm in polynomial time. The results of this work provide a comprehensive understanding of the geometric landscape of the matrix sensing problem in the noisy and overparametrized regime. Funding: This work was supported by grants from the National Science Foundation, Office of Naval Research, Air Force Office of Scientific Research, and Army Research Office.

Sharp Restricted Isometry Property Bounds for Low-Rank Matrix Recovery Problems with Corrupted Measurements

In this paper, we study a general low-rank matrix recovery problem with linear measurements corrupted by some noise. The objective is to understand under what conditions on the restricted isometry property (RIP) of the problem local search methods can find the ground truth with a small error. By analyzing the landscape of the non-convex problem, we first propose a global guarantee on the maximum distance between an arbitrary local minimizer and the ground truth under the assumption that the RIP constant is smaller than 1/2. We show that this distance shrinks to zero as the intensity of the noise reduces. Our new guarantee is sharp in terms of the RIP constant and is much stronger than the existing results. We then present a local guarantee for problems with an arbitrary RIP constant, which states that any local minimizer is either considerably close to the ground truth or far away from it. Next, we prove the strict saddle property, which guarantees the global convergence of the perturbed gradient descent method in polynomial time. The developed results demonstrate how the noise intensity and the RIP constant of the problem affect the landscape of the problem.

The effect of perturbation and noise folding on the recovery performance of low-rank matrix via the nuclear norm minimization

• The completely perturbed low rank matrix recovery model takes in consideration the perturbations in the measurement map and the measurement matrix. • The perturbation on the measurement map result in a multiplicative noise. • The completely perturbed model is equivalent to the partially perturbed model. • The impact of the multiplicative noise on the Signal-to-Noise Ratio (SNR) is different than the additive noise. Previous works in low-rank matrix recovery literature focused on the study of the partially perturbed low-rank matrix recovery model y = A ( X ) + ω , where y ∈ R M is the observed vector, A : R m × n → R M is the measurement operator, X ∈ R m × n is the matrix to be recovered, and ω ∈ R M is the additive noise. However in practice A may only represent a perturbed version of the measurement operator since it models a physical phenomena and there is no guarantee that the desired conditions such as the restricted isometry property (RIP) hold. Similar to the measurement matrix X ∈ R m × n , in a variety of application scenarios, it may also be corrupted by noise. In this paper we introduce the completely perturbed low-rank matrix model by incorporating to the partially perturbed model a non zero perturbation E : R m × n → R M into the measurement operator A which results in a multiplicative noise, and a noise Z ∈ R m × n into the measurement matrix X . First by extending the results of Zhou et al. (2016) we explore the RIP constants of our new proposed model. Then based on the RIP we establish a sufficient condition for robust and stable recovery of low-rank matrices and give an upper bound estimation of the recovery error in the completely perturbed context. The analysis has shown that our model after whitening is equivalent to the partially perturbed model the only distinction is noise variance increase which has a sever impact on the Signal-to-Noise Ratio (SNR).

Lower Bound for RIP Constants and Concentration of Sum of Top Order Statistics

Restricted Isometry Property (RIP) is of fundamental importance in the theory of compressed sensing and forms the base of many exact and robust recovery guarantees in this field. A quantitative description of RIP involves bounding the so-called RIP constants of measurement matrices. In this respect, it is noteworthy that most results in the literature concerning RIP are upper bounds of RIP constants, which can be interpreted as a theoretical guarantee of successful sparse recovery. On the contrary, the land of lower bounds for RIP constants remains uncultivated. Lower bounds of RIP constants, if exist, can be interpreted as the fundamental limit aspect of successful sparse recovery. In this paper, the lower bound of RIP constants Gaussian random matrices are derived, along with a guide for generalization to sub-Gaussian random matrices. This provides a new proof of the fundamental limit that the minimal number of measurements needed to enforce the RIP of order $s$ is $\Omega(s\log({\rm e}N/s))$, which is more straight-forward than the classical Gelfand width argument. Furthermore, in the proof, we propose a useful technical tool featuring the concentration phenomenon for top-$k$ sum of a sequence of i.i.d. random variables, which is closely related to mainstream problems in statistics and is of independent interest.

Improving the bound on the restricted isometry property constant in multiple orthogonal least squares

By allowing multiple L indices to be chosen per iteration, multiple orthogonal least squares (MOLS) have been proposed that is an extension of the classical greedy algorithm OLS. Wang and Li 2017 demonstrated that MOLS can successfully reconstruct K-sparse signals from compressed measurements y = A x in at most K iterations if the sensing matrix A has unit l 2 -norm columns satisfying the restricted isometry property (RIP) of order LK with δ L K < 1 ( K / L ) + 2 . In this study, by increasing the RIP order just by one (i.e. L K + 1 from LK), the authors refine the bound further to δ L K + 1 < ( 1 / ( K / L ) + 2 ) . In the noisy case, they also propose a stopping criterion of MOLS algorithm, and with this criterion MOLS algorithm can recover the support of sparse signal successfully.

A Proof of Conjecture on Restricted Isometry Property Constants $\delta _{tk}\ \left(0&lt;t&lt;\frac {4}{3}\right)$

In this paper, we give a complete answer to the conjecture on restricted isometry property (RIP) constants $\delta _{tk} (0 , which was proposed by T. Cai and A. Zhang. We have shown that when $0 , the condition $\delta _{tk} is sufficient to guarantee the exact recovery for all $k$ -sparse signals in the noiseless case via the constrained $\ell _{1}$ -norm minimization. These bounds are sharp in the sense that for any $\epsilon >0,\,\,\delta _{tk} cannot guarantee the exact recovery of some $k$ -sparse signals. Furthermore, it will be shown that similar characterizations also hold for low-rank matrix recovery. Thus, combined with T. Cai and A. Zhang’s work, a complete characterization for sharp RIP constants $\delta _{tk}$ for all $t > 0$ is obtained to guarantee the exact recovery of all $k$ -sparse signals and matrices with rank at most $k$ by $\ell _{1}$ -norm minimization and nuclear norm minimization, respectively. Noisy cases and approximately sparse cases are also considered. To solve the conjecture, we construct a few identities so that RIP of order $tk$ , which is the target of our main results, can be perfectly applied to them.

On the post selection inference constant under restricted isometry properties

Uniformly valid confidence intervals post model selection in regression can be constructed based on Post-Selection Inference (PoSI) constants. PoSI constants are minimal for orthogonal design matrices, and can be upper bounded in function of the sparsity of the set of models under consideration, for generic design matrices. In order to improve on these generic sparse upper bounds, we consider design matrices satisfying a Restricted Isometry Property (RIP) condition. We provide a new upper bound on the PoSI constant in this setting. This upper bound is an explicit function of the RIP constant of the design matrix, thereby giving an interpolation between the orthogonal setting and the generic sparse setting. We show that this upper bound is asymptotically optimal in many settings by constructing a matching lower bound.

Stable recovery of low-dimensional cones in Hilbert spaces: One RIP to rule them all

Many inverse problems in signal processing deal with the robust estimation of unknown data from underdetermined linear observations. Low-dimensional models, when combined with appropriate regularizers, have been shown to be efficient at performing this task. Sparse models with the 1-norm or low rank models with the nuclear norm are examples of such successful combinations. Stable recovery guarantees in these settings have been established using a common tool adapted to each case: the notion of restricted isometry property (RIP). In this paper, we establish generic RIP-based guarantees for the stable recovery of cones (positively homogeneous model sets) with arbitrary regularizers. These guarantees are illustrated on selected examples. For block structured sparsity in the infinite-dimensional setting, we use the guarantees for a family of regularizers which efficiency in terms of RIP constant can be controlled, leading to stronger and sharper guarantees than the state of the art.

Compressed Sensing: How Sharp Is the Restricted Isometry Property?

Compressed sensing (CS) seeks to recover an unknown vector with $N$ entries by making far fewer than $N$ measurements; it posits that the number of CS measurements should be comparable to the information content of the vector, not simply $N$. CS combines directly the important task of compression with the measurement task. Since its introduction in 2004 there have been hundreds of papers on CS, a large fraction of which develop algorithms to recover a signal from its compressed measurements. Because of the paradoxical nature of CS—exact reconstruction from seemingly undersampled measurements—it is crucial for acceptance of an algorithm that rigorous analyses verify the degree of undersampling the algorithm permits. The restricted isometry property (RIP) has become the dominant tool used for the analysis in such cases. We present here an asymmetric form of RIP that gives tighter bounds than the usual symmetric one. We give the best known bounds on the RIP constants for matrices from the Gaussian ensemble. Our derivations illustrate the way in which the combinatorial nature of CS is controlled. Our quantitative bounds on the RIP allow precise statements as to how aggressively a signal can be undersampled, the essential question for practitioners. We also document the extent to which RIP gives precise information about the true performance limits of CS, by comparison with approaches from high-dimensional geometry.

Phase transitions for greedy sparse approximation algorithms

A major enterprise in compressed sensing and sparse approximation is the design and analysis of computationally tractable algorithms for recovering sparse, exact or approximate, solutions of underdetermined linear systems of equations. Many such algorithms have now been proven to have optimal-order uniform recovery guarantees using the ubiquitous Restricted Isometry Property (RIP) (Candès and Tao (2005) [11]). However, without specifying a matrix, or class of matrices, it is unclear when the RIP-based sufficient conditions on the algorithm are satisfied. Bounds on RIP constants can be inserted into the algorithms RIP-based conditions, translating the conditions into requirements on the signal's sparsity level, length, and number of measurements. We illustrate this approach for Gaussian matrices on three of the state-of-the-art greedy algorithms: CoSaMP (Needell and Tropp (2009) [29]), Subspace Pursuit (SP) (Dai and Milenkovic (2009) [13]) and Iterative Hard Thresholding (IHT) (Blumensath and Davies (2009) [8]). Designed to allow a direct comparison of existing theory, our framework implies that, according to the best available analysis on these three algorithms, IHT requires the fewest number of compressed sensing measurements, has the best proven stability bounds, and has the lowest per iteration computational cost.