Restricted Isometry Research Articles

The restricted isometry property of block diagonal matrices for group-sparse signal recovery

Group-sparsity is a common low-complexity signal model with widespread application across various domains of science and engineering. The recovery of such signal ensembles from compressive measurements has been extensively studied in the literature under the assumption that measurement operators are modeled as densely populated random matrices. In this paper, we turn our attention to an acquisition model intended to ease the energy consumption of sensing devices by splitting the measurements up into distinct signal blocks. More precisely, we present uniform guarantees for group-sparse signal recovery in the scenario where a number of sensors obtain independent partial signal observations modeled by block diagonal measurement matrices. We establish a group-sparse variant of the classical restricted isometry property (RIP) for block diagonal sensing matrices acting on group-sparse vectors, and provide conditions under which subgaussian block diagonal random matrices satisfy this group-RIP with high probability. Two different scenarios are considered in particular. In the first scenario, we assume that each sensor is equipped with an independently drawn measurement matrix. We later lift this requirement by considering measurement matrices with constant block diagonal entries. In other words, every sensor is equipped with a copy of the same prototype matrix. The problem of establishing the group-RIP is cast into a form in which one needs to establish the concentration behavior of the suprema of chaos processes which involves estimating Talagrand's γ 2 functional. As a side effect of the proof, we present an extension to Maurey's empirical method to provide new bounds on the covering number of sets consisting of finite convex combinations of possibly infinite sets.

Large-Scale Affine Matrix Rank Minimization With a Novel Nonconvex Regularizer.

Low-rank minimization aims to recover a matrix of minimum rank subject to linear system constraint. It can be found in various data analysis and machine learning areas, such as recommender systems, video denoising, and signal processing. Nuclear norm minimization is a dominating approach to handle it. However, such a method ignores the difference among singular values of target matrix. To address this issue, nonconvex low-rank regularizers have been widely used. Unfortunately, existing methods suffer from different drawbacks, such as inefficiency and inaccuracy. To alleviate such problems, this article proposes a flexible model with a novel nonconvex regularizer. Such a model not only promotes low rankness but also can be solved much faster and more accurate. With it, the original low-rank problem can be equivalently transformed into the resulting optimization problem under the rank restricted isometry property (rank-RIP) condition. Subsequently, Nesterov's rule and inexact proximal strategies are adopted to achieve a novel algorithm highly efficient in solving this problem at a convergence rate of O(1/K) , with K being the iterate count. Besides, the asymptotic convergence rate is also analyzed rigorously by adopting the Kurdyka- ojasiewicz (KL) inequality. Furthermore, we apply the proposed optimization model to typical low-rank problems, including matrix completion, robust principal component analysis (RPCA), and tensor completion. Exhaustively empirical studies regarding data analysis tasks, i.e., synthetic data analysis, image recovery, personalized recommendation, and background subtraction, indicate that the proposed model outperforms state-of-the-art models in both accuracy and efficiency.

RIP analysis for the weighted ℓr-ℓ1 minimization method

The weighted ℓr−ℓ1 minimization method with 0<r≤1 largely generalizes the classical ℓr minimization method and achieves very good performance in compressive sensing. However, its restricted isometry property (RIP) and high-order RIP analysis results remain unknown. In this paper, we fill in this gap by adopting newly developed analysis tools. Moreover, through a novel decomposition of the objective function into a difference of two convex functions, we propose to solve the weighted ℓr−ℓ1 minimization problem via the difference of convex functions algorithms (DCA) directly. Numerical experiments show that our DCA based weighted ℓr−ℓ1 minimization method gives satisfactory results in sparse recovery no matter whether the measurement matrix is coherent or not. For highly coherent measurements, our proposed method even outperforms the state-of-art ℓ1−ℓ2 minimization method.

A novel method for identification of synchronous resonance frequency of blades based on random arrangement of tip-timing probes

Blade tip-timing (BTT) signal suffers from the undersampling feature, which causes the high-frequency information to be aliased to the low-frequency band, especially the aliasing problem of the synchronous resonance frequency more serious. In order to realize the frequency identification of undersampled signals in synchronous resonance, a novel multiscale cyclic-reverse optimized arrangement (MC-ROA) method based on the random arrangement of tip-timing probes is proposed in this paper. Firstly, a sparse model for frequency recovery is built based on the link between the known undersampled signals and the unknown original signals in the frequency domain, and the subspace pursuit (SP) algorithm of compressive sensing (CS) is used for frequency recovery. Then, under the random arrangement of probes, the number of ideal probes is adjusted at multiple scales and the datum probes are selected cyclically to obtain multiple sets of solutions. Finally, the target frequency weights are used to overcome the limitations of the restricted isometry property (RIP) of the parameterized compression matrix to determine the optimal recovery frequency. The BTT experiments are given to verify the effectiveness of the proposed method.

Performance analysis for unconstrained analysis based approaches* *The work was supported by NSFC (Nos.11901037, 11871109, 61932010, 61932011, 11771162, 12001067), CAEP Foundation (Grant No. CX20200027).

Efficient recovery of sparse signals from underdetermined linear systems has received extensive attentions in recent decade. This paper considers the stable recovery of a signal x , which is not k-sparse itself but is k-sparse in terms of a Parseval frame D , from the observations y = Ax + e via three unconstrained minimization approaches. We show that if the sensing matrix A satisfies the restricted isometry property adapted to D with for any t > 1, then these three approaches can stably recover x . As a consequence, when t = 2 and 3, our result significantly improves existing best sufficient conditions. Furthermore, we theoretically characterize the recovery errors of these methods.

Performance guarantees of regularized [formula omitted]-minimization for robust sparse recovery

Based on the powerful restricted isometry property (RIP) and the coherence tools, this paper develops two types of robust recovery results for a (non-convex) regularized ℓ1−2-minimization model itself, which include some (sufficient) recovery conditions and their resultant recovery error estimates. All these theoretical results show that under some conditions this model is also able to ensure the robust recovery of any signal that is not necessary to be sparse. On the one hand, our RIP-based results not only provide some insights into the regularized ℓ1−2-minimization model, but also well complement the previous ones established for the constrained ℓ1−2-minimization model. On the other hand, our coherence-based recovery condition is demonstrated to be much better than the state-of-the-art one except for a very small number of cases, which to some degree can be neglected in practice. Furthermore, we also discuss the approximate error resulting from the scenario when the regularized ℓ1−2-minimization model is used to replace the classical constrained ℓ1−2-minimization model.

Low rank matrix recovery with impulsive noise

In this paper, we consider low rank matrix recovery with impulsive noise. We first study the difference of nuclear norm and Frobenius norm model and present a stable recovery result based on the matrix restricted isometry property. Then we find the truncated difference of nuclear norm and Frobenius norm model can also stably recover low rank matrices with impulsive noise.

Maximum turn‐off control for discrete‐time linear systems

AbstractWe aim to address the problem of finding a control input maximizing the time instants when all control channels take a zero‐value (are turned off) while stabilizing the system to zero over a given horizon length. This problem is called the maximum turn‐off control problem. To solve it, we reduce the problem into a block‐sparse optimization problem with respect to the control input sequence, where the norm of the control input sequence is the objective function that must be minimized. Because the problem is not convex, we introduce a relaxed problem based on the norm, which is a convex function, and characterize the equivalence relation between the original and relaxed problems using the so‐called block restricted isometry property (block‐RIP). Based on the equivalence, the solution can be obtained by solving the convex relaxed problem. However, the block‐RIP is not easy to interpret and verify. Thus, we propose the notion of sparse controllability Gramians, which is an extension of the controllability Gramians, and show that the block‐RIP can be interpreted by the eigenvalues of the sparse controllability Gramian. This study presents an easy‐to‐check condition of the block‐RIP. Moreover, the above control framework is extended to a model predictive control scheme. These results are demonstrated using numerical examples.

A Novel Sparsity Adaptive Algorithm for Underwater Acoustic Signal Reconstruction.

In view of the fact that most of the traditional algorithms for reconstructing underwater acoustic signals from low-dimensional compressed data are based on known sparsity, a sparsity adaptive and variable step-size matching pursuit (SAVSMP) algorithm is proposed. Firstly, the algorithm uses Restricted Isometry Property (RIP) criterion to estimate the initial value of sparsity, and then employs curve fitting method to adjust the initial value of sparsity to avoid underestimation or overestimation, before finally realizing the close approach of the sparsity level with the adaptive step size. The algorithm selects the atoms by matching test, and uses the Least Squares Method to filter out the unsuitable atoms, so as to realize the precise reconstruction of underwater acoustic signal received by the sonar system. The experimental comparison reveals that the proposed algorithm overcomes the drawbacks of existing algorithms, in terms of high computation time and low reconstruction quality.

Sharp Restricted Isometry Property Bounds for Low-Rank Matrix Recovery Problems with Corrupted Measurements

In this paper, we study a general low-rank matrix recovery problem with linear measurements corrupted by some noise. The objective is to understand under what conditions on the restricted isometry property (RIP) of the problem local search methods can find the ground truth with a small error. By analyzing the landscape of the non-convex problem, we first propose a global guarantee on the maximum distance between an arbitrary local minimizer and the ground truth under the assumption that the RIP constant is smaller than 1/2. We show that this distance shrinks to zero as the intensity of the noise reduces. Our new guarantee is sharp in terms of the RIP constant and is much stronger than the existing results. We then present a local guarantee for problems with an arbitrary RIP constant, which states that any local minimizer is either considerably close to the ground truth or far away from it. Next, we prove the strict saddle property, which guarantees the global convergence of the perturbed gradient descent method in polynomial time. The developed results demonstrate how the noise intensity and the RIP constant of the problem affect the landscape of the problem.

Local and Global Linear Convergence of General Low-Rank Matrix Recovery Problems

We study the convergence rate of gradient-based local search methods for solving low-rank matrix recovery problems with general objectives in both symmetric and asymmetric cases, under the assumption of the restricted isometry property. First, we develop a new technique to verify the Polyak-Lojasiewicz inequality in a neighborhood of the global minimizers, which leads to a local linear convergence region for the gradient descent method. Second, based on the local convergence result and a sharp strict saddle property proven in this paper, we present two new conditions that guarantee the global linear convergence of the perturbed gradient descent method. The developed local and global convergence results provide much stronger theoretical guarantees than the existing results. As a by-product, this work significantly improves the existing bounds on the RIP constant required to guarantee the non-existence of spurious solutions.

Convergence of the forward-backward algorithm: beyond the worst-case with the help of geometry

We provide a comprehensive study of the convergence of the forward-backward algorithm under suitable geometric conditions, such as conditioning or Łojasiewicz properties. These geometrical notions are usually local by nature, and may fail to describe the fine geometry of objective functions relevant in inverse problems and signal processing, that have a nice behaviour on manifolds, or sets open with respect to a weak topology. Motivated by this observation, we revisit those geometric notions over arbitrary sets. In turn, this allows us to present several new results as well as collect in a unified view a variety of results scattered in the literature. Our contributions include the analysis of infinite dimensional convex minimization problems, showing the first Łojasiewicz inequality for a quadratic function associated to a compact operator, and the derivation of new linear rates for problems arising from inverse problems with low-complexity priors. Our approach allows to establish unexpected connections between geometry and a priori conditions in inverse problems, such as source conditions, or restricted isometry properties.

Sparse signal reconstruction via recurrent neural networks with hyperbolic tangent function

In this paper, several recurrent neural networks (RNNs) for solving the L1-minimization problem are proposed. First, a one-layer RNN based on the hyperbolic tangent function and the projection matrix is designed. In addition, the stability and global convergence of the previously presented RNN are proved by the Lyapunov method. Then, the sliding mode control technique is introduced into the former RNN to design finite-time RNN (FTRNN). Under the condition that the projection matrix satisfies the Restricted Isometry Property (RIP), a suitable Lyapunov function is constructed to prove that the FTRNN is stable in the Lyapunov sense and has the finite-time convergence property. Finally, we make a comparison of the proposed RNN and FTRNN with the existing RNNs. To achieve this, we implement experiments for sparse signal reconstruction and image reconstruction. The results further demonstrate the effectiveness and superior performance of the proposed RNN and FTRNN.

Performance analysis of the compressed distributed least squares algorithm

In this paper, we consider the distributed estimation problem of unknown high-dimensional sparse signals for a random dynamic system. We propose a compressed distributed algorithm by using the compressive sensing theory and the distributed least squares (LS) algorithm. Under a compressed cooperative persistent excitation condition, the upper bound of the estimation error is established which is positively related to the restricted isometry constant. Our results are obtained without relying on some stringent conditions such as independency or stationarity of the regression vectors. Finally, we provide a simulation example to show that the compressed distributed least squares algorithm has better performance than the regularized distributed LS algorithm with l 1 penalty for the estimation of high-dimensional sparse signals. • High-dimensional sparse signal. • Compressed distributed least squares. • System identification. • Random dynamic system

Compressive independent component analysis: theory and algorithms

Abstract Compressive learning forms the exciting intersection between compressed sensing and statistical learning where one exploits sparsity of the learning model to reduce the memory and/or computational complexity of the algorithms used to solve the learning task. In this paper, we look at the independent component analysis (ICA) model through the compressive learning lens. In particular, we show that solutions to the cumulant-based ICA model have a particular structure that induces a low-dimensional model set that resides in the cumulant tensor space. By showing that a restricted isometry property holds for random cumulants e.g. Gaussian ensembles, we prove the existence of a compressive ICA scheme. Thereafter, we propose two algorithms of the form of an iterative projection gradient and an alternating steepest descent algorithm for compressive ICA, where the order of compression asserted from the restricted isometry property is realized through empirical results. We provide analysis of the CICA algorithms including the effects of finite samples. The effects of compression are characterized by a trade-off between the sketch size and the statistical efficiency of the ICA estimates. By considering synthetic and real datasets, we show the substantial memory gains achieved over well-known ICA algorithms by using one of the proposed CICA algorithms.

Hierarchical isometry properties of hierarchical measurements

A new class of measurement operators, coined hierarchical measurement operators, and prove results guaranteeing the efficient, stable and robust recovery of hierarchically structured signals from such measurements. We derive bounds on their hierarchical restricted isometry properties based on the restricted isometry constants of their constituent matrices, generalizing and extending prior work on Kronecker-product measurements. As an exemplary application, we apply the theory to two communication scenarios. The fast and scalable HiHTP algorithm is shown to be suitable for solving these types of problems and its performance is evaluated numerically in terms of sparse signal recovery and block detection capability.

Robust recovery of a kind of weighted l1-minimization without noise level

This paper studies a kind of weighted [Formula: see text]-minimization that is motivated by function interpolation. Combining with the weighted robust null space property, we first propose a new sufficient condition for robust recovery via the weighted [Formula: see text]-minimization when the measurements are corrupted by arbitrary noise without requiring the proper estimation of noise level. Second, we investigate the instance optimality of the weighted [Formula: see text]-minimization decoder according to the weighted quotient property and the weighted restricted isometry property (RIP). In addition, to give a better error estimation in a general problem to recover noisy compressible signals, we improve the [Formula: see text]-RIP constant [Formula: see text] from [Formula: see text] to [Formula: see text] by using the weighted sparsity and Stechkin-type estimate. Our results show that the weighted [Formula: see text]-minimization remains not only stable but also robust to reconstruct signals with noisy observations.

Learning a compressive sensing matrix with structural constraints via maximum mean discrepancy optimization

We introduce a learning-based algorithm to obtain a measurement matrix for compressive sensing related recovery problems. The focus lies on matrices with a constant modulus constraint which, e.g., represent a network of analog phase shifters in hybrid precoding/combining architectures. We interpret a matrix with restricted isometry property as a mapping from a high- to a low-dimensional hypersphere. We argue that points on the low-dimensional hypersphere should be uniformly distributed to combat measurement noise. This notion is formalized as an optimization problem which uses a maximum mean discrepancy metric as objective function. Recent success of such metrics in neural network related topics motivate a solution of the optimization problem based on machine learning. Numerical experiments show a better performance than random matrices that are typical for compressive sensing. Further, we adapt a method from the literature to the constant modulus constraint. This method can also compete with random matrices and harmonizes well with the proposed algorithm if it is used as an initialization. Lastly, we describe how other structural matrix constraints, e.g., a Toeplitz constraint, can be taken into account as well.

An Anti-Jamming Method against Interrupted Sampling Repeater Jamming Based on Compressed Sensing.

Interrupted sampling repeater jamming (ISRJ) is an attracted coherent jamming method to inverse synthetic aperture radar (ISAR) in the past decades. By means of different jamming parameters settings, realistic dense false targets can be formed around the true target. This paper proposed an adaptive anti-jamming method against ISRJ by adjusting the number of measurements based on compressed sensing (CS). The jamming signal is energy concentrated and segmented sparse in the frequency domain. The measurements number of the reconstructed target signal and the jamming signal is different. According to the restricted isometry property (RIP) condition of CS theory, signal reconstructing performance depends on the number of measurements that varies with the sparsity of the vector. Thus, the jamming signal is suppressed, and the true target signal is retained by altering the measurements number of echo signals. Besides, the two-dimensional (2D) anti-jamming method is derived in detail. The anti-jamming effect is analyzed with different signal-to-noise ratios (SNR), sampling rates, and jam-to-signal ratios (JSR). Simulations prove the effectiveness of the proposed anti-jamming method.

Analysis of sparse recovery for Legendre expansions using envelope bound

We provide novel sufficient conditions for the uniform recovery of sparse Legendre expansions using ℓ1 minimization, where the sampling points are drawn according to orthogonalization (uniform) measure. So far, conditions of the form m ≳ Θ 2 s × log factors have been relied on to determine the minimum number of samples m that guarantees successful reconstruction of s-sparse vectors when the measurement matrix is associated to an orthonormal system. However, in case of sparse Legendre expansions, the uniform bound Θ of Legendre systems is so high that these conditions are unable to provide meaningful guarantees. In this paper, we present an analysis which employs the envelop bound of all Legendre polynomials instead, and prove a new recovery guarantee for s-sparse Legendre expansions, m ≳ s 2 × log factors , which is independent of Θ. Arguably, this is the first recovery condition established for orthonormal systems without assuming the uniform boundedness of the sampling matrix. The key ingredient of our analysis is an extension of chaining arguments, recently developed in Bourgain and Chkifa et al., to handle the envelope bound. Furthermore, our recovery condition is proved via restricted eigenvalue property, a less demanding replacement of restricted isometry property which is perfectly suited to the considered scenario. Along the way, we derive simple criteria to detect good sample sets. Our numerical tests show that sets of uniformly sampled points that meet these criteria will perform better recovery on average.