Conditions For Exact Recovery Research Articles

Low-rank matrix recovery problem minimizing a new ratio of two norms approximating the rank function then using an ADMM-type solver with applications

Since the nuclear norm may lead to suboptimal solutions to rank minimization problems, many nonconvex surrogates have been proposed to provide better rank function approximations. In this paper, we use the ratio of the nuclear norm and the Frobenius norm, denoted as N/F, as a new nonconvex surrogate of the rank function. The N/F can provide a close approximation for the matrix rank. In particular, the N/F is the same as the rank for rank 1 matrices. Furthermore, compared to other popular nonconvex approximations, the N/F model has no extra parameters to tune. Theoretically, we establish the exact recovery condition and the stable recovery condition for the N/F minimization problem with or without noise, respectively. Computationally, we use the alternating direction method of multipliers with a specific variable-splitting scheme to solve the N/F minimization problem. What is more, we incorporate a box constraint in the N/F model, and propose the ADMM-box to solve it. The convergence analysis of ADMM and ADMM-box are also given. Extensive experiments on both synthetic data and color images verify the effectiveness of our proposed methods.

Phase transitions for support recovery under local differential privacy

We address the problem of variable selection in a high-dimensional but sparse mean model, under the additional constraint that only privatized data are available for inference. The original data are vectors with independent entries having a symmetric, strongly log-concave distribution on \mathbb{R} . For this purpose, we adopt a recent generalization of classical minimax theory to the framework of local \alpha -differential privacy. We provide lower and upper bounds on the rate of convergence for the expected Hamming loss over classes of at most s -sparse vectors whose non-zero coordinates are separated from 0 by a constant a>0 . As corollaries, we derive necessary and sufficient conditions (up to log factors) for exact recovery and for almost full recovery. When we restrict our attention to non-interactive mechanisms that act independently on each coordinate our lower bound shows that, contrary to the non-private setting, both exact and almost full recovery are impossible whatever the value of a in the high-dimensional regime such that n \alpha^2/ d^2\lesssim 1 . However, in the regime n\alpha^2/d^2\gg \log(d) we can exhibit a critical value a^* (up to a logarithmic factor) such that exact and almost full recovery are possible for all a\gg a^* and impossible for a\leq a^* . We show that these results can be improved when allowing for all non-interactive (that act globally on all coordinates) locally \alpha -differentially private mechanisms in the sense that phase transitions occur at lower levels.

Minimax optimal clustering of bipartite graphs with a generalized power method

Abstract Clustering bipartite graphs is a fundamental task in network analysis. In the high-dimensional regime where the number of rows $n_{1}$ and the number of columns $n_{2}$ of the associated adjacency matrix are of different order, the existing methods derived from the ones used for symmetric graphs can come with sub-optimal guarantees. Due to increasing number of applications for bipartite graphs in the high-dimensional regime, it is of fundamental importance to design optimal algorithms for this setting. The recent work of Ndaoud et al. (2022, IEEE Trans. Inf. Theory, 68, 1960–1975) improves the existing upper-bound for the misclustering rate in the special case where the columns (resp. rows) can be partitioned into $L = 2$ (resp. $K = 2$) communities. Unfortunately, their algorithm cannot be extended to the more general setting where $K \neq L \geq 2$. We overcome this limitation by introducing a new algorithm based on the power method. We derive conditions for exact recovery in the general setting where $K \neq L \geq 2$, and show that it recovers the result in Ndaoud et al. (2022, IEEE Trans. Inf. Theory, 68, 1960–1975). We also derive a minimax lower bound on the misclustering error when $K=L$ under a symmetric version of our model, which matches the corresponding upper bound up to a factor depending on $K$.

Joint Block Support Recovery for Sub-Nyquist Sampling Cooperative Spectrum Sensing

This letter aims to improve the efficiency and reliability of joint block sparse support recovery by exploiting block features in the MMV problem. For the block MMV problem, we first propose a Block SOMP algorithm extended from SOMP, which can reduce the number of iterations of the original method. Then, we perform an in-depth theoretical analysis of the proposed algorithm using Exact Recovery Condition (ERC). Both theoretical analysis and simulation experiments confirm that the proposed algorithm can provide reliable recovery for higher sparsity. However, the premise is that the design of the cooperative compressed sensing matrix requires special attention.

A Signal Dependent Analysis of Orthogonal Least Squares Based on the Restricted Isometry Property

The Orthogonal Least Squares (OLS) is a widespread greedy algorithm for sparse signal approximation or reconstruction. We here address the problem of sufficient condition for exact signal support recovery via OLS. Based on the generalized Wielandt inequality, new bounds on the OLS selection rule are provided that enable a simple approach of the Restricted Isometry Property based analysis of OLS. Since these new bounds naturally take the signal magnitude distribution into account, they allow a derivation of a new relaxed exact support recovery condition in the noise-free case. Moreover, we study their influence on the design of such new conditions in bounded noise and Gaussian noise cases.

Community Detection With Known, Unknown, or Partially Known Auxiliary Latent Variables

Empirical observations suggest that in practice, community membership does not completely explain the dependency between the edges of an observation graph. The residual dependence of the graph edges are modeled in this paper, to first order, by auxiliary node latent variables that affect the statistics of the graph edges but <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">carry no information about the communities of interest.</i> We then study community detection in graphs obeying the stochastic block model and censored block model with auxiliary latent variables. We analyze the conditions for exact recovery when these auxiliary latent variables are unknown, representing unknown nuisance parameters or model mismatch. We also analyze exact recovery when these secondary latent variables have been either fully or partially revealed. Finally, we propose a semidefinite programming algorithm for recovering the desired labels when the secondary labels are either known or unknown. We show that exact recovery is possible by semidefinite programming down to the respective maximum likelihood exact recovery threshold.

K-median: exact recovery in the extended stochastic ball model

We study exact recovery conditions for the linear programming relaxation of the k-median problem in the stochastic ball model (SBM). In Awasthi et al. (Relax, no need to round: integrality of clustering formulations. arXiv:1408.4045, 2015; in: Proceedings of the 2015 conference on innovations in theoretical computer science, pp 191–200, 2015), the authors give a tight result for the k-median LP in the SBM, saying that exact recovery can be achieved as long as the balls are pairwise disjoint. We give a counterexample to their result, thereby showing that the k-median LP is not tight in low dimension. Instead, we give a near optimal result showing that the k-median LP in the SBM is tight in high dimension. We also show that, if the probability measure satisfies some concentration assumptions, then the k-median LP in the SBM is tight in every dimension. Furthermore, we propose a new model of data called extended stochastic ball model (ESBM), which significantly generalizes the well-known SBM. We then show that exact recovery can still be achieved in the ESBM.

Stable Recovery and Separation of Structured Signals by Convex Geometry

We consider the ill-posed problem of identifying the signal that is structured in some general dictionary (i.e., possibly redundant or over-complete matrix) from corrupted measurements, where the corruption is structured in another general dictionary. We formulate the problem by applying appropriate convex constraints to the signal and corruption according to their structures and provide conditions for exact recovery from structured corruption and stable recovery from structured corruption with added stochastic bounded noise. In addition, this paper provides estimates of the number of the measurements needed for recovery. These estimates are based on computing the Gaussian complexity of a tangent cone and the Gaussian distance to a subdifferential. Applications covered by the proposed programs include the recovery of signals that is disturbed by impulse noise, missing entries, sparse outliers, random bounded noise, and signal separation. Numerical simulation results are presented to verify and complement the theoretical results.

Recovery Conditions of Sparse Signals Using Orthogonal Least Squares-Type Algorithms

Orthogonal least squares (OLS)-type algorithms are efficient in reconstructing sparse signals, which include the well-known OLS, multiple OLS (MOLS) and block OLS (BOLS). In this paper, we first investigate the noiseless exact recovery conditions of these algorithms. Specifically, based on mutual incoherence property (MIP), we provide theoretical analysis of OLS and MOLS to ensure that the correct nonzero support can be selected during the iterative procedure. Nevertheless, theoretical analysis for BOLS utilizes the block-MIP to deal with the block sparsity. Furthermore, the noiseless MIP-based analyses are extended to the noisy scenario. Our results indicate that for K-sparse signals, when MIP or SNR satisfies certain conditions, OLS and MOLS obtain reliable reconstruction in at most K iterations, while BOLS succeeds in at most (K/d) iterations where d is the block length. It is shown that our derived theoretical results improve the existing ones, which are verified by simulation tests.

Recovering a Single Community With Side Information

We study the effect of the quality and quantity of side information on the recovery of a hidden community of size K = o(n) in a graph of size n. Side information for each node in the graph is modeled by a random vector in which either the vector dimension or the LLR of each component with respect to node labels is independent of n. These two models represent the variation in quality and quantity of side information. Under maximum likelihood detection, we calculate tight necessary and sufficient conditions for exact recovery of the labels. We demonstrate how side information needs to evolve with n in terms of either its quantity, or quality, to improve the exact recovery threshold. A similar set of results are obtained for weak recovery. Under belief propagation, tight necessary and sufficient conditions for weak recovery are calculated when the LLRs are constant, and sufficient conditions when the LLRs vary with n. Moreover, we design and analyze a local voting procedure using side information that can achieve exact recovery when applied after belief propagation.

Tight recovery guarantees for orthogonal matching pursuit under Gaussian noise

Abstract Orthogonal matching pursuit (OMP) is a popular algorithm to estimate an unknown sparse vector from multiple linear measurements of it. Assuming exact sparsity and that the measurements are corrupted by additive Gaussian noise, the success of OMP is often formulated as exactly recovering the support of the sparse vector. Several authors derived a sufficient condition for exact support recovery by OMP with high probability depending on the signal-to-noise ratio, defined as the magnitude of the smallest non-zero coefficient of the vector divided by the noise level. We make two contributions. First, we derive a slightly sharper sufficient condition for two variants of OMP, in which either the sparsity level or the noise level is known. Next, we show that this sharper sufficient condition is tight, in the following sense: for a wide range of problem parameters, there exist a dictionary of linear measurements and a sparse vector with a signal-to-noise ratio slightly below that of the sufficient condition, for which with high probability OMP fails to recover its support. Finally, we present simulations that illustrate that our condition is tight for a much broader range of dictionaries.

On the exact recovery conditions of 3D human motion from 2D landmark motion with sparse articulated motion

In this paper, we address the problem of exact recovery condition in retrieving 3D human motion from 2D landmark motion. We use a skeletal kinematic model to represent the 3D motion as a vector of angular articulation motion. We address this problem based on the observation that at high tracking rate, regardless of the global rigid motion, only few angular articulations have non-zero motion. We propose a first ideal formulation with ℓ0-norm to minimize the cardinal of non-zero angular articulation motion given an equality constraint on the time-differentiation of the reprojection error. The second relaxed formulation relies on an ℓ1-norm to minimize the sum of absolute values of the angular articulation motion. This formulation has the advantage of being able to provide 3D motion even in the under-determined case when twice the number of 2D landmarks is smaller than the number of angular articulations. We define a specific property which is the Projective Kinematic Space Property (PKSP) that takes into account the reprojection constraint and the kinematic model. We prove that, for the relaxed formulation, we are able to recover the exact 3D human motion from 2D landmarks if and only if the PKSP property is verified. We further demonstrate that solving the relaxed formulation provides the same ground-truth solution as the ideal formulation if and only if the PKSP condition is filled. Results with simulated sparse skeletal angular motion show the ability of the proposed method to recover exact location of angular motion. We provide results on publicly available real data (Human3.6M, Panoptic, MPI-I3DHPand 3DPW) and show that our method is able to achieve state-of-the-art 3D reconstructions.

When does OMP achieve exact recovery with continuous dictionaries?

This paper presents new theoretical results on sparse recovery guarantees for a greedy algorithm, Orthogonal Matching Pursuit (OMP), in the context of continuous parametric dictionaries. Here, the continuous setting means that the dictionary is made up of an infinite uncountable number of atoms. In this work, we rely on the Hilbert structure of the observation space to express our recovery results as a property of the kernel defined by the inner product between two atoms. Using a continuous extension of Tropp's Exact Recovery Condition, we identify key assumptions allowing to analyze OMP in the continuous setting. Under these assumptions, OMP unambiguously identifies in exactly $k$ steps the atom parameters from any observed linear combination of $k$ atoms. These parameters play the role of the so-called support of a sparse representation in traditional sparse recovery. In our paper, any kernel and set of parameters that satisfy these conditions are said to be admissible. In the one-dimensional setting, we exhibit a family of kernels relying on completely monotone functions for which admissibility holds for any set of atom parameters. For higher dimensional parameter spaces, the analysis turns out to be more subtle. An additional assumption, so-called axis admissibility, is imposed to ensure a form of delayed recovery (in at most $k^D$ steps, where $D$ is the dimension of the parameter space). Furthermore, guarantees for recovery in exactly $k$ steps are derived under an additional algebraic condition involving a finite subset of atoms (built as an extension of the set of atoms to be recovered). We show that the latter technical conditions simplify in the case of Laplacian kernels, allowing us to derive simple conditions for $k$-step exact recovery, and to carry out a coherence-based analysis in terms of a minimum separation assumption between the atoms to be recovered.

Preconditioned generalized orthogonal matching pursuit

Recently, compressed sensing (CS) has aroused much attention for that sparse signals can be retrieved from a small set of linear samples. Algorithms for CS reconstruction can be roughly classified into two categories: (1) optimization-based algorithms and (2) greedy search ones. In this paper, we propose an algorithm called the preconditioned generalized orthogonal matching pursuit (Pre-gOMP) to promote the recovery performance. We provide a sufficient condition for exact recovery via the Pre-gOMP algorithm, which says that if the mutual coherence of the preconditioned sampling matrix Φ satisfies mu ({Phi }) < frac {1}{SK -S + 1}, then the Pre-gOMP algorithm exactly recovers any K-sparse signals from the compressed samples, where S (>1) is the number of indices selected in each iteration of Pre-gOMP. We also apply the Pre-gOMP algorithm to the application of ghost imaging. Our experimental results demonstrate that the Pre-gOMP can largely improve the imaging quality of ghost imaging, while boosting the imaging speed.

Isolated calmness of solution mappings and exact recovery conditions for nuclear norm optimization problems

ABSTRACT We derive some conditions for a feasible point to be the isolated optimal solution for three classes of nuclear norm optimization problems by using the isolated calmness of two set-valued mappings: one is the level set mapping constructed by these problems themselves, and the other is the multiplier set mapping of their dual problems. Among others, the conditions from the dual angle are shown to be weaker than those from the primal angle, while the conditions from the primal angle are almost necessary and require the same number of measurements as the sufficient and necessary one does when the estimation mechanism in [Chandrasekaran et al. The convex geometry of linear inverse problems. Found Comput Math. 2012;12:805–849; Amelunxen et al. Living on the edge: phase transitions in convex programs with random data. Inf Inference. 2014;3:224–294] is used. In particular, we show that the deterministic exact recovery conditions in [Candès and Recht. Exact matrix completion via convex optimization. Found Comput Math. 2009;9:717–772; Chandrasekaran et al. Rank-sparsity inchoherence for matrix decomposition. SIAM J Optim. 2011;21:572–596] are stronger than the constraint nondegeneracy of the dual problems, whereas our conditions from the dual view are equivalent to the strict Robinson constraint qualification for them.

Dictionary Learning With BLOTLESS Update

Algorithms for learning a dictionary to sparsely represent a given dataset typically alternate between sparse coding and dictionary update stages. Methods for dictionary update aim to minimise expansion error by updating dictionary vectors and expansion coefficients given patterns of non-zero coefficients obtained in the sparse coding stage. We propose a block total least squares (BLOTLESS) algorithm for dictionary update. BLOTLESS updates a block of dictionary elements and the corresponding sparse coefficients simultaneously. In the error free case, three necessary conditions for exact recovery are identified. Lower bounds on the number of training data are established so that the necessary conditions hold with high probability. Numerical simulations show that the bounds approximate well the number of training data needed for exact dictionary recovery. Numerical experiments further demonstrate several benefits of dictionary learning with BLOTLESS update compared with state-of-the-art algorithms especially when the amount of training data is small.

A Deterministic Theory for Exact Non-Convex Phase Retrieval

In this paper, we analyze the non-convex framework of Wirtinger Flow (WF) for phase retrieval and identify a novel sufficient condition for universal exact recovery through the lens of low rank matrix recovery theory. Via a perspective in the lifted domain, we show that the convergence of the WF iterates to a true solution is attained geometrically under a single condition on the lifted forward model. As a result, a deterministic relationship between the accuracy of spectral initialization and the validity of {the regularity condition} is derived. In particular, we determine that a certain concentration property on the spectral matrix must hold uniformly with a sufficiently tight constant. This culminates into a sufficient condition that is equivalent to a restricted isometry-type property over rank-1, positive semi-definite matrices, and amounts to a less stringent requirement on the lifted forward model than those of prominent low-rank-matrix-recovery methods in the literature. We characterize the performance limits of our framework in terms of the tightness of the concentration property via novel bounds on the convergence rate and on the signal-to-noise ratio such that the theoretical guarantees are valid using the spectral initialization at the proper sample complexity.

Exact Multistatic Interferometric Imaging via Generalized Wirtinger Flow

In this paper, we present a novel approach that can exactly recover extended targets in wave-based multistatic interferometric imaging, based on Generalized Wirtinger Flow (GWF) theory [1]. Interferometric imaging is a generalization of phase retrieval, which arises from cross-correlation of measurements from pairs of receivers in multistatic configuration. Unlike standard Wirtinger Flow, GWF theory guarantees exact recovery for arbitrary lifted forward models that satisfy the restricted isometry property over rank-1, positive semi-definite (PSD) matrices with a sufficiently small restricted isometry constant (RIC). To this end, we design a deterministic, lifted forward model for interferometric multistatic radar satisfying the exact recovery conditions of the GWF theory. Our results quantify a lower limit on the pixel spacing and the minimal sample complexity for exact multistatic radar imaging via GWF. We provide a numerical study of our RIC and pixel spacing bounds, which shows that GWF can achieve exact recovery with super-resolution. While our primary interest lies in radar imaging, our method is also applicable to other multistatic wave-based imaging problems such as those arising in acoustics and geophysics.

High-Dimensional Uncertainty Quantification of Electronic and Photonic IC With Non-Gaussian Correlated Process Variations

Uncertainty quantification based on generalized polynomial chaos has been used in many applications. It has also achieved great success in variation-aware design automation. However, almost all existing techniques assume that the parameters are mutually independent or Gaussian correlated, which is rarely true in real applications. For instance, in chip manufacturing, many process variations are actually correlated. Recently, some techniques have been developed to handle non-Gaussian correlated random parameters, but they are time-consuming for high-dimensional problems. We present a new framework to solve uncertainty quantification problems with many non-Gaussian correlated uncertainties. First, we propose a set of smooth basis functions to well capture the impact of non-Gaussian correlated process variations. We develop a tensor approach to compute these basis functions in a high-dimension setting. Second, we investigate the theoretical aspect and practical implementation of a sparse solver to compute the coefficients of all basis functions. We provide some theoretical analysis for the exact recovery condition and error bound of this sparse solver in the context of uncertainty quantification. We present three adaptive sampling approaches to improve the performance of the sparse solver. Finally, we validate our methods by synthetic and practical electronic/photonic ICs with 19 to 57 non-Gaussian correlated variation parameters. Our approach outperforms Monte Carlo by thousands of times in terms of efficiency. It can also accurately predict the output density functions with multiple peaks caused by non-Gaussian correlations, which are hard to capture by existing methods.

A Unified Framework for Compression and Compressed Sensing of Light Fields and Light Field Videos

In this article we present a novel dictionary learning framework designed for compression and sampling of light fields and light field videos. Unlike previous methods, where a single dictionary with one-dimensional atoms is learned, we propose to train a Multidimensional Dictionary Ensemble (MDE). It is shown that learning an ensemble in the native dimensionality of the data promotes sparsity, hence increasing the compression ratio and sampling efficiency. To make maximum use of correlations within the light field data sets, we also introduce a novel nonlocal pre-clustering approach that constructs an Aggregate MDE (AMDE). The pre-clustering not only improves the image quality but also reduces the training time by an order of magnitude in most cases. The decoding algorithm supports efficient local reconstruction of the compressed data, which enables efficient real-time playback of high-resolution light field videos. Moreover, we discuss the application of AMDE for compressed sensing. A theoretical analysis is presented that indicates the required conditions for exact recovery of point-sampled light fields that are sparse under AMDE. The analysis provides guidelines for designing efficient compressive light field cameras. We use various synthetic and natural light field and light field video data sets to demonstrate the utility of our approach in comparison with the state-of-the-art learning-based dictionaries, as well as established analytical dictionaries.