High-dimensional Sparse Signals Research Articles

Performance analysis of the compressed distributed least squares algorithm

In this paper, we consider the distributed estimation problem of unknown high-dimensional sparse signals for a random dynamic system. We propose a compressed distributed algorithm by using the compressive sensing theory and the distributed least squares (LS) algorithm. Under a compressed cooperative persistent excitation condition, the upper bound of the estimation error is established which is positively related to the restricted isometry constant. Our results are obtained without relying on some stringent conditions such as independency or stationarity of the regression vectors. Finally, we provide a simulation example to show that the compressed distributed least squares algorithm has better performance than the regularized distributed LS algorithm with l 1 penalty for the estimation of high-dimensional sparse signals. • High-dimensional sparse signal. • Compressed distributed least squares. • System identification. • Random dynamic system

Compressed Least-Squares Regression on Sparse Spaces

Recent advances in the area of compressed sensing suggest that it is possible to reconstruct high-dimensional sparse signals from a small number of random projections. Domains in which the sparsity assumption is applicable also offer many interesting large-scale machine learning prediction tasks. It is therefore important to study the effect of random projections as a dimensionality reduction method under such sparsity assumptions. In this paper we develop the bias-variance analysis of a least-squares regression estimator in compressed spaces when random projections are applied on sparse input signals. Leveraging the sparsity assumption, we are able to work with arbitrary non i.i.d. sampling strategies and derive a worst-case bound on the entire space. Empirical results on synthetic and real-world datasets shows how the choice of the projection size affects the performance of regression on compressed spaces, and highlights a range of problems where the method is useful.

High-dimensional sparse FFT based on sampling along multiple rank-1 lattices

The reconstruction of high-dimensional sparse signals is a challenging task in a wide range of applications. In order to deal with high-dimensional problems, efficient sparse fast Fourier transform algorithms are essential tools. The second and third authors have recently proposed a dimension-incremental approach, which only scales almost linear in the number of required sampling values and almost quadratic in the arithmetic complexity with respect to the spatial dimension d . Using reconstructing rank-1 lattices as sampling scheme, the method showed reliable reconstruction results in numerical tests but suffers from relatively large numbers of samples and arithmetic operations. Combining the preferable properties of reconstructing rank-1 lattices with small sample and arithmetic complexities, the first author developed the concept of multiple rank-1 lattices. In this paper, both concepts — dimension-incremental reconstruction and multiple rank-1 lattices — are coupled, which yields a distinctly improved high-dimensional sparse fast Fourier transform. Moreover, the resulting algorithm is analyzed in detail with respect to success probability, number of required samples, and arithmetic complexity. In comparison to single rank-1 lattices, the utilization of multiple rank-1 lattices results in a reduction in the complexities by an almost linear factor with respect to the sparsity. Various numerical tests confirm the theoretical results, the high performance, and the reliability of the proposed method.

Distribution agnostic Bayesian matching pursuit based on the exponential embedded family

Abstract Compressed sensing (CS) is an emerging field that allows to recover high-dimensional sparse signal from a few compressed measurements. Classical CS algorithms using the Bayesian framework generally impose a sparseness-promoting prior on the signal, which may not characterize the real signals with diversity. Moreover, estimating the parameters involved in the prior is challenging especially for non-i.i.d. signals. In this paper, we propose an efficient prior-agnostic Bayesian matching pursuit for sparse signal recovery, which avoids the risk of imposing mismatched prior on the signals and enjoys lower complexity than Bayesian approaches due to the elimination of prior parameter estimation. Specifically, we utilize the noise model and the reduced exponential embedded family (EEF) to obtain an approximate likelihood of interest, and then find the most dominant supports with maximum approximate likelihoods in a greedy manner to give an approximate minimum mean squared error (MMSE) estimate for the sparse signal. Experiment results demonstrate that our method achieves lower normalized mean squared error (NMSE) while higher efficiency compared to the state-of-the-art methods.

Analysis of compressed distributed adaptive filters

Abstract In order to estimate an unknown high-dimensional sparse signal in the network, we present a class of compressed distributed adaptive filtering algorithms based on consensus strategies and compressive sensing (CS) method. This class of algorithms is designed to first use the compressed regressor data to obtain an estimate for the compressed unknown signal, then apply some signal reconstruction algorithms to obtain a high-dimensional estimate for the original unknown signal. Here we consider the compressed consensus normalized least mean squares (NLMS) algorithm, and show that even if the traditional non-compressed distributed algorithm cannot fulfill the estimation or tracking task due to the sparsity of the regressors, the compressed algorithm introduced in this paper can be used to estimate the unknown high-dimensional sparse signal under a compressed information condition, which is much weaker than the cooperative information condition used in the existing literature, without such stringent conditions as independence and stationarity for the system signals.

Robust Bayesian compressed sensing with outliers

Abstract We consider the problem of robust compressed sensing where the objective is to recover a high-dimensional sparse signal from compressed measurements partially corrupted by outliers. A new sparse Bayesian learning method is developed for this purpose. The basic idea of the proposed method is to identify the outliers and exclude them from sparse signal recovery. To automatically identify the outliers, we employ a set of binary indicator variables to indicate which observations are outliers. These indicator variables are assigned a beta-Bernoulli hierarchical prior such that their values are confined to be binary. In addition, a Gaussian-inverse Gamma prior is imposed on the sparse signal to promote sparsity. Based on this hierarchical prior model, we develop a variational Bayesian method to estimate the indicator variables as well as the sparse signal. Simulation results show that the proposed method achieves a substantial performance improvement over existing robust compressed sensing techniques.

Reconstruction of Sparse Wavelet Signals From Partial Fourier Measurements

In this paper, we show that high-dimensional sparse wavelet signals of finite levels can be constructed from their partial Fourier measurements on a deterministic sampling set with cardinality about a multiple of signal sparsity.

Convergence of ℓ2/3 Regularization for Sparse Signal Recovery

In this paper, we consider the problem of finding the sparsest solution to underdetermined linear systems. Unlike the literatures which use the ℓ1 regularization to approximate the original problem, we consider the ℓ2/3 regularization which leads to a better approximation but a nonconvex, nonsmooth, and non-Lipschitz optimization problem. Through developing a fixed point representation theory associated with the two thirds thresholding operator for ℓ2/3 regularization solutions, we propose a fixed point iterative thresholding algorithm based on two thirds norm for solving the k-sparsity problems. Relying on the restricted isometry property, we provide subsequentional convergence guarantee for this fixed point iterative thresholding algorithm on recovering a sparse signal. By discussing the preferred regularization parameters and studying the phase diagram, we get an adequate and efficient algorithm for the high-dimensional sparse signal recovery. Finally, comparing with the existing algorithms, such as the standard ℓ1 minimization, the iterative reweighted ℓ2 minimization, the iterative reweighted ℓ1 minimization, and iterative Half thresholding algorithm, we display the results of the experiment which indicate that the two thirds norm fixed point iterative thresholding algorithm applied to sparse signal recovery and large scale imageries from noisy measurements can be accepted as an effective solver for ℓ2/3 regularization.

Recovery of High-Dimensional Sparse Signals via -Minimization

We consider the recovery of high-dimensional sparse signals via -minimization under mutual incoherence condition, which is shown to be sufficient for sparse signals recovery in the noiseless and noise cases. We study both -minimization under the constraint and the Dantzig selector. Using the two -minimization methods and a technical inequality, some results are obtained. They improve the results of the error bounds in the literature and are extended to the general case of reconstructing an arbitrary signal.

Orthogonal Matching Pursuit for Sparse Signal Recovery With Noise

We consider the orthogonal matching pursuit (OMP) algorithm for the recovery of a high-dimensional sparse signal based on a small number of noisy linear measurements. OMP is an iterative greedy algorithm that selects at each step the column, which is most correlated with the current residuals. In this paper, we present a fully data driven OMP algorithm with explicit stopping rules. It is shown that under conditions on the mutual incoherence and the minimum magnitude of the nonzero components of the signal, the support of the signal can be recovered exactly by the OMP algorithm with high probability. In addition, we also consider the problem of identifying significant components in the case where some of the nonzero components are possibly small. It is shown that in this case the OMP algorithm will still select all the significant components before possibly selecting incorrect ones. Moreover, with modified stopping rules, the OMP algorithm can ensure that no zero components are selected.

Toeplitz Compressed Sensing Matrices With Applications to Sparse Channel Estimation

Compressed sensing (CS) has recently emerged as a powerful signal acquisition paradigm. In essence, CS enables the recovery of high-dimensional sparse signals from relatively few linear observations in the form of projections onto a collection of test vectors. Existing results show that if the entries of the test vectors are independent realizations of certain zero-mean random variables, then with high probability the unknown signals can be recovered by solving a tractable convex optimization. This work extends CS theory to settings where the entries of the test vectors exhibit structured statistical dependencies. It follows that CS can be effectively utilized in linear, time-invariant system identification problems provided the impulse response of the system is (approximately or exactly) sparse. An immediate application is in wireless multipath channel estimation. It is shown here that time-domain probing of a multipath channel with a random binary sequence, along with utilization of CS reconstruction techniques, can provide significant improvements in estimation accuracy compared to traditional least-squares based linear channel estimation strategies. Abstract extensions of the main results are also discussed, where the theory of equitable graph coloring is employed to establish the utility of CS in settings where the test vectors exhibit more general statistical dependencies.

Fast OMP algorithm for 2D angle estimation in MIMO radar

A high-dimensional sparse signal usually should be realigned as a long 1D signal to be recovered by orthogonal matching pursuit (OMP), an efficient algorithm for compressed sensing. Clearly, however, the realigned long signal will result in a large amount of computation in OMP. If each atom in the dictionary can be expressed as the Kronecker product of two vectors, it can possible to decompose this dictionary into two sub-dictionaries. By exploiting this property, a fast OMP algorithm for 2D sparse signals of this kind is presented, and applied to 2D angle estimation in MIMO radar. Simulation results verify its good reconstruction quality approximate to that of OMP and greatly improved computational efficiency.

Shifting Inequality and Recovery of Sparse Signals

In this paper, we present a concise and coherent analysis of the constrained ?1 minimization method for stable recovering of high-dimensional sparse signals both in the noiseless case and noisy case. The analysis is surprisingly simple and elementary, while leads to strong results. In particular, it is shown that the sparse recovery problem can be solved via ?1 minimization under weaker conditions than what is known in the literature. A key technical tool is an elementary inequality, called Shifting Inequality, which, for a given nonnegative decreasing sequence, bounds the ?2 norm of a subsequence in terms of the ?1 norm of another subsequence by shifting the elements to the upper end.

On Recovery of Sparse Signals Via $\ell _{1}$ Minimization

This paper considers constrained lscr1 minimization methods in a unified framework for the recovery of high-dimensional sparse signals in three settings: noiseless, bounded error, and Gaussian noise. Both lscr1 minimization with an lscrinfin constraint (Dantzig selector) and lscr1 minimization under an llscr2 constraint are considered. The results of this paper improve the existing results in the literature by weakening the conditions and tightening the error bounds. The improvement on the conditions shows that signals with larger support can be recovered accurately. In particular, our results illustrate the relationship between lscr1 minimization with an llscr2 constraint and lscr1 minimization with an lscrinfin constraint. This paper also establishes connections between restricted isometry property and the mutual incoherence property. Some results of Candes, Romberg, and Tao (2006), Candes and Tao (2007), and Donoho, Elad, and Temlyakov (2006) are extended.