Unknown Sparse Vector Research Articles

Multiple Choice Hard Thresholding Pursuit (MCHTP) for simultaneous sparse recovery and sparsity order estimation

In this work, we propose a new greedy algorithm, referred to as the Multiple Choice Hard Thresholding Pursuit (MCHTP) which recovers a sparse vector from compressed linear measurements without explicit knowledge of its sparsity. MCHTP achieves this by suitably combining the steps of hard thresholding pursuit (HTP) with a novel sparsity selection criterion to iteratively select one among two possible sparsity values and subsequently recovering the unknown sparse vector along with its unknown sparsity order. We provide provable theoretical guarantees which ensure that MCHTP can estimate the sparsity order as well as the unknown sparse vector exactly from linear measurements. We also propose an extension of MCHTP, referred to as MCHTP−d, which can provide finer control on the estimated sparsity by tuning the parameter d. The simulation results corroborate the theoretical findings demonstrating the superior performance of MCHTP and MCHTP−d compared to state-of-the-art techniques in terms of sparse recovery and sparsity order estimation.

Codes for exact support recovery of sparse vectors from inaccurate linear measurements and their decoding

We construct codes that allow to exactly recover the support of an unknown sparse vector with almost equal absolute values of all nonzero coordinates given results of linear measurements in the presence of noise with ℓp-norm bounded from above. We propose a decoding algorithm with asymptotically minimum complexity.

Joint channel and impulse noise estimation based on compressed sensing and Kalman filter for OFDM system

Impulse noise (IN) widely exists in many communication systems, which seriously affects the performance of OFDM communication systems. A joint channel and IN estimation method based on all subcarriers is designed. This method uses a sparse Bayesian learning (SBL) algorithm incorporating forward–backward Kalman filter (FB-Kalman) to tackle the problem of joint channel and IN estimation and data detection for OFDM systems. Firstly, the channel impulse response and IN are regarded as unknown sparse vectors, and a SBL framework using all subcarriers is proposed to estimate the unknown vector. The SBL theory is used based on the prior distribution of variables, and then the forward–backward joint system is established, which applies the data detection simultaneously. We also propose the FB-Kalman implementation algorithm by using the expectation maximization updates. Explicit expressions of mean and covariance matrix of the posterior distribution are derived in the E-step. Simulation results show that the proposed algorithm improves the normalized mean square error and bit error rate performance of OFDM system in the presence of IN communication environment.

Adaptive Diffusion Pairwise Fused Lasso LMS Algorithm Over Networks.

The topic of identification for sparse vector in a distributed way has triggered great interest in the area of adaptive filtering. Grouping components in the sparse vector has been validated to be an efficient way for enhancing identification performance for sparse parameter. The technique of pairwise fused lasso, which can promote similarity between each possible pair of nonnegligible components in the sparse vector, does not require that the nonnegligible components have to be distributed in one or multiple clusters. In other words, the nonnegligible components may be randomly scattered in the unknown sparse vector. In this article, based on the technique of pairwise fused lasso, we propose the novel pairwise fused lasso diffusion least mean-square (PFL-DLMS) algorithm, to identify sparse vector. The objective function we construct consists of three terms, i.e., the mean-square error (MSE) term, the regularizing term promoting the sparsity of all components, and the regularizing term promoting the sparsity of difference between each pair of components in the unknown sparse vector. After investigating mean stability condition of mean-square behavior in theoretical analysis, we propose the strategy of variable regularizing coefficients to overcome the difficulty that the optimal regularizing coefficients are usually unknown. Finally, numerical experiments are conducted to verify the effectiveness of the PFL-DLMS algorithm in identifying and tracking sparse parameter vector.

Distributed sparse identification for stochastic dynamic systems under cooperative non-persistent excitation condition

This paper considers the distributed sparse identification problem over wireless sensor networks such that all sensors cooperatively estimate the unknown sparse parameter vector of stochastic dynamic systems by using the local information from neighbors. A distributed sparse least squares algorithm is proposed by minimizing a local information criterion formulated as a linear combination of accumulative local estimation error and L1-regularization term. The upper bound of the estimation error of the proposed algorithm is presented. Furthermore, by designing a suitable adaptive weighting coefficient based on the local observation data, the set convergence of zero elements with a finite number of observations is obtained under a cooperative non-persistent excitation condition. It is shown that the proposed distributed algorithm can work well in a cooperative way even though none of the individual sensors can fulfill the estimation task. Our theoretical results are obtained without relying on the independency assumptions of regression signals that have been commonly used in the existing literature. Thus, our results are expected to be applied to stochastic feedback systems. Finally, the numerical simulations are provided to demonstrate the effectiveness of our theoretical results.

Tight recovery guarantees for orthogonal matching pursuit under Gaussian noise

Abstract Orthogonal matching pursuit (OMP) is a popular algorithm to estimate an unknown sparse vector from multiple linear measurements of it. Assuming exact sparsity and that the measurements are corrupted by additive Gaussian noise, the success of OMP is often formulated as exactly recovering the support of the sparse vector. Several authors derived a sufficient condition for exact support recovery by OMP with high probability depending on the signal-to-noise ratio, defined as the magnitude of the smallest non-zero coefficient of the vector divided by the noise level. We make two contributions. First, we derive a slightly sharper sufficient condition for two variants of OMP, in which either the sparsity level or the noise level is known. Next, we show that this sharper sufficient condition is tight, in the following sense: for a wide range of problem parameters, there exist a dictionary of linear measurements and a sparse vector with a signal-to-noise ratio slightly below that of the sufficient condition, for which with high probability OMP fails to recover its support. Finally, we present simulations that illustrate that our condition is tight for a much broader range of dictionaries.

Kalman Filter Based Recursive Estimation of Slowly Fading Sparse Channel in Impulsive Noise Environment for OFDM Systems

In this paper, we propose a recursive sparse channel estimation algorithm in the presence of impulse noise. Firstly the channel impulse response and impulsive noise are jointly viewed as an unknown sparse vector. Then a novel recursive Kalman filtering based compressed sensing algorithm for joint channel and impulsive noise estimation is proposed by using the first order autoregressive model for tracking slowly time varying wireless channel. This algorithm can be extended also to quasi-static, block-fading scenario conveniently. Simulation results illustrate the efficiency of the proposed techniques in terms of the mean square error and bit error rate performance.

Imaging with highly incomplete and corrupted data

We consider the problem of imaging sparse scenes from a few noisy data using an -minimization approach. This problem can be cast as a linear system of the form , where is an measurement matrix. We assume that the dimension of the unknown sparse vector is much larger than the dimension of the data vector , i.e. . We provide a theoretical framework that allows us to examine under what conditions the -minimization problem admits a solution that is close to the exact one in the presence of noise. Our analysis shows that -minimization is not robust for imaging with noisy data when high resolution is required. To improve the performance of -minimization we propose to solve instead the augmented linear system , where the matrix is a noise collector. It is constructed so as its column vectors provide a frame on which the noise of the data, a vector of dimension N, can be well approximated. Theoretically, the dimension of the noise collector should be which would make its use not practical. However, our numerical results illustrate that robust results in the presence of noise can be obtained with a large enough number of columns .

A Sharp Sufficient Condition for Sparsity Pattern Recovery

Sufficient number of linear and noisy measurements for exact and approximate sparsity pattern/support set recovery in the high dimensional setting is derived. Although this problem as been addressed in the recent literature, there is still considerable gaps between those results and the exact limits of the perfect support set recovery. To reduce this gap, in this paper, the sufficient condition is enhanced. A specific form of a Joint Typicality decoder is used for the support recovery task. Two performance metrics are considered for the recovery validation; one, which considers exact support recovery, and the other which seeks partial support recovery. First, an upper bound is obtained on the error probability of the sparsity pattern recovery. Next, using the mentioned upper bound, sufficient number of measurements for reliable support recovery is derived. It is shown that the sufficient condition for reliable support recovery depends on three key parameters of the problem; the noise variance, the minimum nonzero entry of the unknown sparse vector and the sparsity level. Simulations are performed for different sparsity rate, different noise variances, and different distortion levels. The results show that for all the mentioned cases the proposed methodology increases convergence rate of upper bound of the error probability of support recovery significantly which leads to a lower error probability bound compared with previously proposed bounds.

Guaranteed Localization of More Sources Than Sensors With Finite Snapshots in Multiple Measurement Vector Models Using Difference Co-Arrays

The Multiple Measurement Vector (MMV) problem is central to sparse signal processing, where the goal is to recover the common support of a set of unknown sparse vectors of size $N$ , from $L$ compressed measurement vectors, each of size $M . Recent advances in correlation-aware and Bayesian techniques for MMV models show promising evidence that under appropriate assumptions, it is possible to recover supports of size ( $s$ ) larger than the dimension ( $M$ ) of each measurement vector. However, these results are primarily asymptotic in $L$ , and cannot provide support recovery guarantees for finite $L$ . This paper overcomes such drawback by focusing on a broader family of correlation-aware optimization problems (which include convex problems), and establishing rigorous non-asymptotic probabilistic guarantees in the regime $s>M$ when the measurements are collected using appropriately designed sparse sensor arrays (such as nested and coprime). Assuming the source locations obey a certain “minimum separation” condition, we develop uniform upper bounds on the estimation error that is obeyed by any algorithm belonging to this family, and utilize this bound to ensure probabilistic support recovery in the regime $s>M$ . Our results crucially rely upon (i) the unique geometry of the difference co-array of the sparse arrays, and (ii) certain non-negativity constraints on the optimization variable. As a result of independent interest, we also show that this upper bound is tight with respect to the dimension $N$ . Extensive numerical simulations (including phase transition plots) are presented to validate the theoretical claims.

Vector Recovery for a Linear System Corrupted by Unknown Sparse Error Vectors With Applications to Secure State Estimation

A method to recover the unknown sparse error $e$ in a corrupted linear system $b=Ax+e$ is proposed. The original problem is first transformed into a convex optimization problem with equality constraints using the QR decomposition of $A$ . The transformed problem is then solved using 1-norm minimization. The proposed method is applied to the secure state estimation of a cyber physical system (CPS). Two different methods for solving such problems are also discussed.

Weighted diffusion continuous mixed p-norm algorithm for distributed estimation in non-uniform noise environment

This paper presents weighted diffusion least mean p-power (LMP) algorithm for distributed estimation of an unknown sparse vector in a sensor network. We consider a network, in which the variances of the noise for sensors are different, i.e., non-uniform noise condition. We replace the sum of mean square errors with a weighted sum of LMP for global and local cost functions of a sensor network. The weights are adaptive and are updated by a simple steepest-descent recursion to minimize the global and local cost functions of the adaptive algorithm. Further, we propose the adaptive weighted diffusion continuous mixed p-norm (CMPN) algorithm, which further improves the performance of the proposed weighted diffusion LMP algorithm. In the proposed weighted diffusion CMPN algorithm, p-power is considered adaptive and continuous in the range 1 ≤ p ≤ 2. Unlike the CMPN algorithm in the literature with uniform weights, the weighted diffusion CMPN algorithm uses adaptive weights, which are updated by a simple steepest descent recursion. We also extend the proposed algorithm to one-bit distributed estimation scenario. Performance analysis and simulation results show the efficacy of the proposed weighted diffusion LMP and CMPN algorithms. Further, the proposed one-bit weighted diffusion algorithms exhibit robustness against the different noise distributions.

Learning Mixtures of Sparse Linear Regressions Using Sparse Graph Codes

In this paper, we consider the mixture of sparse linear regressions model. Let ${\beta}^{(1)},\ldots,{\beta}^{(L)}\in\mathbb{C}^n$ be $ L $ unknown sparse parameter vectors with a total of $ K $ non-zero coefficients. Noisy linear measurements are obtained in the form $y_i={x}_i^H {\beta}^{(\ell_i)} + w_i$, each of which is generated randomly from one of the sparse vectors with the label $ \ell_i $ unknown. The goal is to estimate the parameter vectors efficiently with low sample and computational costs. This problem presents significant challenges as one needs to simultaneously solve the demixing problem of recovering the labels $ \ell_i $ as well as the estimation problem of recovering the sparse vectors $ {\beta}^{(\ell)} $. Our solution to the problem leverages the connection between modern coding theory and statistical inference. We introduce a new algorithm, Mixed-Coloring, which samples the mixture strategically using query vectors $ {x}_i $ constructed based on ideas from sparse graph codes. Our novel code design allows for both efficient demixing and parameter estimation. In the noiseless setting, for a constant number of sparse parameter vectors, our algorithm achieves the order-optimal sample and time complexities of $\Theta(K)$. In the presence of Gaussian noise, for the problem with two parameter vectors (i.e., $L=2$), we show that the Robust Mixed-Coloring algorithm achieves near-optimal $\Theta(K polylog(n))$ sample and time complexities. When $K=O(n^{\alpha})$ for some constant $\alpha\in(0,1)$ (i.e., $K$ is sublinear in $n$), we can achieve sample and time complexities both sublinear in the ambient dimension. In one of our experiments, to recover a mixture of two regressions with dimension $n=500$ and sparsity $K=50$, our algorithm is more than $300$ times faster than EM algorithm, with about one third of its sample cost.

SA-M-SBL: An Algorithm for CSI-Based Device-Free Localization With Faulty Prior Information

As an emerging technique, device-free localization (DFL) is promising to localize the target without attaching any transceivers. Recently, the benefits of channel state information (CSI) on DFL have been revealed in this paper. Motivated by this, in this paper, we propose to exploit the channel diversity of CSI measurements for multi-target DFL under the compressive sensing (CS) framework. The CSI-based multi-target DFL problem is formulated as a joint sparse recovery problem which reconstructs the unknown sparse vectors of multiple channels. Moreover, in practice, some faulty prior information (e.g., coarse positions) is usually available. To take advantage of this information for joint sparse recovery, novel support knowledge-aided multiple sparse Bayesian learning (SA-M-SBL) algorithm is introduced, which incorporates the prior information into a three-layer hierarchical prior model. With this model, the joint sparsity of the sparse vectors can be induced, and their values can be estimated via the variational Bayesian inference (VBI). The numerical simulation results demonstrate the outstanding performance of the proposed method compared with the state-of-the-art CS-based multi-target DFL methods.

Iterative Sparsification-Projection: Fast and Robust Sparse Signal Approximation

In this paper, we address recovery of sparse signals from compressed measurements, and sparse signal approximation, which have received considerable attention over the last decade. First, we revisit smoothed L0 (SL0), a well-known sparse recovery algorithm, and give some insights into it that have not been noticed previously. Specifically, we re-derive the SL0 algorithm based on proximal methods, and using recent results in solving nonconvex problems by proximal algorithms, we provide a convergence guarantee for it. In addition, inspired by this derivation, we propose a general family of algorithms, which we call iterative sparsification-projection (ISP), having SL0 as a special member. Our algorithmic framework starts with an initial guess for the unknown sparse vector, and then iteratively sparsifies it (using a fixed threshold) followed by projecting the result onto the admissible solution set. The threshold is then decreased, and the same process is repeated. The algorithm terminates when the threshold becomes sufficiently small, or another stopping criterion is satisfied. We also propose a robust projection to handle the situations with observation noise or model uncertainties. Our extensive simulations confirm the promising performance of the ISP algorithms compared with some well-known algorithms.

Impulsive noise suppression in the case of frequency estimation by exploring signal sparsity

The frequency estimation problem is addressed in this work in the presence of impulsive noise. Two typical scenarios are considered; that is, the received data are assumed to be uniformly sampled, i.e., without data missing for the first case and data are randomly missed for the second case. The main objective of this work is to explore the signal sparsity in the frequency domain to perform frequency estimation under the impulsive noise. Therefore, to that end, a DFT-like matrix is created in which the frequency sparsity is provided. The missing measurements are modeled by a sparse representation as well, where missing samples are set to be zeros. Based on this model, the missing pattern represented by a vector is indeed sparse since it only contains zeros and ones. The impulsive noise is remodeled as a superposition of a unknown sparse vector and a Gaussian vector because of the impulsive nature of noise. By utilizing the sparse property of the vector, the impulsive noise can be treated as a unknown parameter and hence it can be canceled efficiently. By exploring the sparsity obtained, therefore, a joint estimation method is devised under optimization framework. It renders one to simultaneously estimate the frequency, noise, and the missing pattern. Numerical studies and an application to speech denoising indicate that the joint estimation method always offers precise and consistent performance when compared to the non-joint estimation approach.

Fast reconstruction algorithm for perturbed compressive sensing based on total least-squares and proximal splitting

We consider the problem of finding a sparse solution for an underdetermined linear system of equations when the known parameters on both sides of the system are subject to perturbation. This problem is particularly relevant to reconstruction in fully-perturbed compressive-sensing setups where both the projected measurements of an unknown sparse vector and the knowledge of the associated projection matrix are perturbed due to noise, error, mismatch, etc. We propose a new iterative algorithm for tackling this problem. The proposed algorithm utilizes the proximal-gradient method to find a sparse total least-squares solution by minimizing an l1-regularized Rayleigh-quotient cost function. We determine the step-size of the algorithm at each iteration using an adaptive rule accompanied by backtracking line search to improve the algorithm’s convergence speed and preserve its stability. The proposed algorithm is considerably faster than a popular previously-proposed algorithm, which employs the alternating-direction method and coordinate-descent iterations, as it requires significantly fewer computations to deliver the same accuracy. We demonstrate the effectiveness of the proposed algorithm via simulation results.

Bayesian linear regression with sparse priors

We study full Bayesian procedures for high-dimensional linear regression under sparsity constraints. The prior is a mixture of point masses at zero and continuous distributions. Under compatibility conditions on the design matrix, the posterior distribution is shown to contract at the optimal rate for recovery of the unknown sparse vector, and to give optimal prediction of the response vector. It is also shown to select the correct sparse model, or at least the coefficients that are significantly different from zero. The asymptotic shape of the posterior distribution is characterized and employed to the construction and study of credible sets for uncertainty quantification.

Information-theoretic analysis of support recovery from sparsely corrupted measurements

This paper studies the recovery of the support of sparse signal that is corrupted by both dense noise and gross error. The gross error is an unknown sparse vector whose nonzero entries maybe unbounded. This setup covers a wide range of applications, such as face recognition, inpainting and sensor networks. We derive the information-theoretic lower bounds on the sampling rate required to obtain a desired error rate, which depend on the properties of both the signal and the gross error. The investigations are given in the high-dimensional setting. Some illustrations are provided to further reveal the relationship of these bounds.

Homotopy Based Algorithms for $\ell _{\scriptscriptstyle 0}$-Regularized Least-Squares

Sparse signal restoration is usually formulated as the minimization of a quadratic cost function $\|y-Ax\|_2^2$, where A is a dictionary and x is an unknown sparse vector. It is well-known that imposing an $\ell_0$ constraint leads to an NP-hard minimization problem. The convex relaxation approach has received considerable attention, where the $\ell_0$-norm is replaced by the $\ell_1$-norm. Among the many efficient $\ell_1$ solvers, the homotopy algorithm minimizes $\|y-Ax\|_2^2+\lambda\|x\|_1$ with respect to x for a continuum of $\lambda$'s. It is inspired by the piecewise regularity of the $\ell_1$-regularization path, also referred to as the homotopy path. In this paper, we address the minimization problem $\|y-Ax\|_2^2+\lambda\|x\|_0$ for a continuum of $\lambda$'s and propose two heuristic search algorithms for $\ell_0$-homotopy. Continuation Single Best Replacement is a forward-backward greedy strategy extending the Single Best Replacement algorithm, previously proposed for $\ell_0$-minimization at a given $\lambda$. The adaptive search of the $\lambda$-values is inspired by $\ell_1$-homotopy. $\ell_0$ Regularization Path Descent is a more complex algorithm exploiting the structural properties of the $\ell_0$-regularization path, which is piecewise constant with respect to $\lambda$. Both algorithms are empirically evaluated for difficult inverse problems involving ill-conditioned dictionaries. Finally, we show that they can be easily coupled with usual methods of model order selection.