Greedy Pursuit Research Articles

A simulated annealing algorithm for sparse recovery by l0 minimization

This paper addresses the sparse recovery problem by l0 minimization, which is of central importance in the compressed sensing theory. We model the problem as a combinatorial optimization problem and present a novel algorithm termed SASR based on simulated annealing (SA) and some greedy pursuit (GP) algorithms. In SASR, the initial solution is designed using the simple thresholding algorithm, and the generating mechanism is designed using the strategies existed in the subspace pursuit algorithm and the compressed sampling matching pursuit algorithm. On both the random Gaussian data and the face recognition task, the numerical simulation results illustrate the efficiency of SASR. Compared with the existing sparse recovery algorithms, SASR is more efficient in finding global optimums and performs relatively fast in some good cases. That is, SASR inherits the advantage of SA in finding global optimums and the advantage of GP in fast speed to some extent.

Optimization of Greedy Pursuit Algorithms Based on Probability Analysis

Through analyzing prior conditional probability of signal reconstruction in compressed sensing, the paper puts forward an improved method toward serials greedy pursuit algorithms. This method can achieve more accurate selection when select columns in perception matrix that is the most correlated with the residual. Meanwhile, this paper reviews most of greedy pursuit algorithms, and simulations validate the efficacy of the proposed method.

Improved sparse representation using adaptive spatial support for effective target detection in hyperspectral imagery

With increasing applications of hyperspectral imagery (HSI) in agriculture, mineralogy, military, and other fields, one of the fundamental tasks is accurate detection of the target of interest. In this article, improved sparse representation approaches using adaptive spatial support are proposed for effective target detection in HSI. For conventional sparse representation, an HSI pixel is represented as a sparse vector whose non-zero entries correspond to the weights of the selected training atoms from a structured dictionary. For improved sparse representation, spatial correlation and spectral similarity of adjacent neighbouring pixels are exploited as spatial support in this context. The size and shape of the spatial support is automatically determined using both adaptive window and adaptive neighbourhood strategies. Accordingly, a solution based on greedy pursuit algorithms is also given to solve the extended optimization problem in recovering the desired sparse representation. Comprehensive experiments on three different data sets using both visual inspection and quantitative evaluation are carried out. The results from these data sets have indicated that the proposed approaches help to generate improved results in terms of efficacy and efficiency.

A Subspace Pursuit-based Iterative Greedy Hierarchical solution to the neuromagnetic inverse problem

Magnetoencephalography (MEG) is an important non-invasive method for studying activity within the human brain. Source localization methods can be used to estimate spatiotemporal activity from MEG measurements with high temporal resolution, but the spatial resolution of these estimates is poor due to the ill-posed nature of the MEG inverse problem. Recent developments in source localization methodology have emphasized temporal as well as spatial constraints to improve source localization accuracy, but these methods can be computationally intense. Solutions emphasizing spatial sparsity hold tremendous promise, since the underlying neurophysiological processes generating MEG signals are often sparse in nature, whether in the form of focal sources, or distributed sources representing large-scale functional networks. Recent developments in the theory of compressed sensing (CS) provide a rigorous framework to estimate signals with sparse structure. In particular, a class of CS algorithms referred to as greedy pursuit algorithms can provide both high recovery accuracy and low computational complexity. Greedy pursuit algorithms are difficult to apply directly to the MEG inverse problem because of the high-dimensional structure of the MEG source space and the high spatial correlation in MEG measurements. In this paper, we develop a novel greedy pursuit algorithm for sparse MEG source localization that overcomes these fundamental problems. This algorithm, which we refer to as the Subspace Pursuit-based Iterative Greedy Hierarchical (SPIGH) inverse solution, exhibits very low computational complexity while achieving very high localization accuracy. We evaluate the performance of the proposed algorithm using comprehensive simulations, as well as the analysis of human MEG data during spontaneous brain activity and somatosensory stimuli. These studies reveal substantial performance gains provided by the SPIGH algorithm in terms of computational complexity, localization accuracy, and robustness.

Oblique Pursuits for Compressed Sensing

Compressed sensing is a new data acquisition paradigm enabling universal, simple, and reduced-cost acquisition, by exploiting a sparse signal model. Most notably, recovery of the signal by computationally efficient algorithms is guaranteed for certain randomized acquisition systems. However, there is a discrepancy between the theoretical guarantees and practical applications. In applications, including Fourier imaging in various modalities, the measurements are acquired by inner products with vectors selected randomly (sampled) from a frame. Currently available guarantees are derived using the so-called restricted isometry property (RIP), which has only been shown to hold under ideal assumptions. For example, the sampling from the frame needs to be independent and identically distributed with the uniform distribution, and the frame must be tight. In practice though, one or more of the ideal assumptions are typically violated and none of the RIP-based guarantees applies. Motivated by this discrepancy, we propose two related changes in the existing framework: 1) a generalized RIP called the restricted biorthogonality property (RBOP); and 2) correspondingly modified versions of existing greedy pursuit algorithms, which we call oblique pursuits. Oblique pursuits are guaranteed using the RBOP without requiring ideal assumptions; hence, the guarantees apply to practical acquisition schemes. Numerical results show that oblique pursuits also perform competitively with, or sometimes better than their conventional counterparts.

Hierarchized Block Wise Image Approximation by Greedy Pursuit Strategies

An approach for effective implementation of greedy selection methodologies, to approximate an image partitioned into blocks, is proposed. The method is specially designed for approximating partitions on a transformed image. It evolves by selecting, at each iteration step, i) the elements for approximating each of the blocks partitioning the image and ii) the hierarchized sequence in which the blocks are approximated to reach the required global condition on sparsity.

Stable Signal Recovery via Randomly Enhanced Adaptive Subspace Pursuit Method

This letter addresses the stable signal recovery problem in the presence of noisy perturbations, and presents a variant of Subspace Pursuit (SP) method, called Randomly Enhanced Adaptive SP (REASP) method. REASP first adds correlative and irrelative atoms with an adaptive technique and a random strategy respectively, next detects the overall reliability, and then deletes unreliable atoms per iteration. Through these modifications, REASP is expected to achieve superior performance. The complexity analysis shows that the computational complexity of REASP is a bit higher but acceptable. The experimental results illustrate the superior performance of REASP, compared to the frequently used greedy pursuit (GP) and l1 basis pursuit (BP) methods.

Sparse Signal Recovery Based on Simulated Annealing

Sparse signal recovery is a hot topic in the fields of optimization theory and signal processing. Two main algorithmic approaches, i.e. greedy pursuit algorithms and convex relaxation algorithms have been extensively used to solve this problem. However, these algorithms cannot guarantee to find the global optimum solution, and then they perform poorly when the sparsity level is relatively large. Based on the simulated annealing algorithm and greedy pursuit algorithms, we propose a novel algorithm on solving the sparse recovery problem. Numerical simulations show that the proposed algorithm has very good recovery performance.

Compressed sensing signal recovery via forward–backward pursuit

Recovery of sparse signals from compressed measurements constitutes an l0 norm minimization problem, which is unpractical to solve. A number of sparse recovery approaches have appeared in the literature, including l1 minimization techniques, greedy pursuit algorithms, Bayesian methods and nonconvex optimization techniques among others. This manuscript introduces a novel two stage greedy approach, called the Forward-Backward Pursuit (FBP). FBP is an iterative approach where each iteration consists of consecutive forward and backward stages. The forward step first expands the support estimate by the forward step size, while the following backward step shrinks it by the backward step size. The forward step size is larger than the backward step size, hence the initially empty support estimate is expanded at the end of each iteration. Forward and backward steps are iterated until the residual power of the observation vector falls below a threshold. This structure of FBP does not necessitate the sparsity level to be known a priori in contrast to the Subspace Pursuit or Compressive Sampling Matching Pursuit algorithms. FBP recovery performance is demonstrated via simulations including recovery of random sparse signals with different nonzero coefficient distributions in noisy and noise-free scenarios in addition to the recovery of a sparse image.

Effect of multipath channel models to the recovery algorithms on compressed sensing in UWB channel estimation

Multipath arrivals in an Ultra-WideBand (UWB) channel have a long time intervals between clusters and rays where the signal takes on zero or negligible values. It is precisely the signal sparsity of the impulse response of the UWB channel that is exploited in this work aiming at UWB channel estimation based on Compressed Sensing (CS). However, these multipath arrivals mainly depend on the channel environments that generate different sparse levels (low-sparse or high-sparse) of the UWB channels. According to this basis, we have analyzed the two most basic recovery algorithms, one based on linear programming Basis Pursuit (BP), another using greedy method Orthogonal Matching Pursuit (OMP), and chosen the best recovery algorithm which are suitable to the sparse level for each type of channel environment. Besides, the results of this work is an open topic for further research aimed at creating a optimal algorithm specially for application of CS based UWB systems.

Compressive radar with off-grid targets: a perturbation approach

Compressed sensing (CS) schemes are proposed for monostatic as well as synthetic aperture radar (SAR) imaging with chirped signals and ultra-narrowband (UNB) continuous waveforms. In particular, a simple, perturbation method is developed to reduce the gridding error for off-grid targets. A coherence bound is obtained for the resulting measurement matrix. A greedy pursuit algorithm, support-constrained orthogonal matching pursuit, is proposed to take advantage of the support constraint in the perturbation formulation and proved to have the capacity of determining the off-grid targets to the grid accuracy under favorable conditions. Alternatively, the locally optimized thresholding is proposed to enhance the performance of the CS method, basis pursuit. For the advantages of higher signal-to-noise ratio and signal-to-interference ratio, it is proposed that Spotlight SAR imaging will be implemented with CS techniques and multi-frequency UNB waveforms. Numerical simulations show promising results of the proposed approach and algorithms.

Oracle-Order Recovery Performance of Greedy Pursuits With Replacement Against General Perturbations

Applying the theory of compressive sensing in practice always takes different kinds of perturbations into consideration. In this paper, the recovery performance of greedy pursuits with replacement for sparse recovery is analyzed when both the measurement vector and the sensing matrix are contaminated with additive perturbations. Specifically, greedy pursuits with replacement include three algorithms, compressive sampling matching pursuit (CoSaMP), subspace pursuit (SP), and iterative hard thresholding (IHT), where the support estimation is evaluated and updated in each iteration. Based on restricted isometry property, a unified form of the error bounds of these recovery algorithms is derived under general perturbations for compressible signals. The results reveal that the recovery performance is stable against both perturbations. In addition, these bounds are compared with that of oracle recovery-least squares solution with the locations of some largest entries in magnitude known a priori. The comparison shows that the error bounds of these algorithms only differ in coefficients from the lower bound of oracle recovery for some certain signal and perturbations, as reveals that oracle-order recovery performance of greedy pursuits with replacement is guaranteed. Numerical simulations are performed to verify the conclusions.

TV-min and greedy pursuit for constrained joint sparsity and application to inverse scattering

This paper proposes a general framework for compressed sensing of constrained joint sparsity (CJS) which includes total variation minimization (TV-min) as an example. TV- and 2-norm error bounds, independent of the ambient dimension, are derived for the CJS version of Basis Pursuit and Orthogonal Matching Pursuit. As an application the results extend Cand`es, Romberg and Tao's proof of exact recovery of piecewise constant objects with noiseless incomplete Fourier data to the case of noisy data.

Hyperspectral Image Classification via Kernel Sparse Representation

In this paper, a novel nonlinear technique for hyperspectral image (HSI) classification is proposed. Our approach relies on sparsely representing a test sample in terms of all of the training samples in a feature space induced by a kernel function. For each test pixel in the feature space, a sparse representation vector is obtained by decomposing the test pixel over a training dictionary, also in the same feature space, by using a kernel-based greedy pursuit algorithm. The recovered sparse representation vector is then used directly to determine the class label of the test pixel. Projecting the samples into a high-dimensional feature space and kernelizing the sparse representation improve the data separability between different classes, providing a higher classification accuracy compared to the more conventional linear sparsity-based classification algorithms. Moreover, the spatial coherency across neighboring pixels is also incorporated through a kernelized joint sparsity model, where all of the pixels within a small neighborhood are jointly represented in the feature space by selecting a few common training samples. Kernel greedy optimization algorithms are suggested in this paper to solve the kernel versions of the single-pixel and multi-pixel joint sparsity-based recovery problems. Experimental results on several HSIs show that the proposed technique outperforms the linear sparsity-based classification technique, as well as the classical support vector machines and sparse kernel logistic regression classifiers.

Perturbation Analysis of Orthogonal Matching Pursuit

Orthogonal Matching Pursuit (OMP) is a canonical greedy pursuit algorithm for sparse approximation. Previous studies of OMP have mainly considered the exact recovery of a sparse signal $\bm x$ through $\bm \Phi$ and $\bm y=\bm \Phi \bm x$, where $\bm \Phi$ is a matrix with more columns than rows. In this paper, based on Restricted Isometry Property (RIP), the performance of OMP is analyzed under general perturbations, which means both $\bm y$ and $\bm \Phi$ are perturbed. Though exact recovery of an almost sparse signal $\bm x$ is no longer feasible, the main contribution reveals that the exact recovery of the locations of $k$ largest magnitude entries of $\bm x$ can be guaranteed under reasonable conditions. The error between $\bm x$ and solution of OMP is also estimated. It is also demonstrated that the sufficient condition is rather tight by constructing an example. When $\bm x$ is strong-decaying, it is proved that the sufficient conditions can be relaxed, and the locations can even be recovered in the order of the entries' magnitude.

Sparse SAR imaging based on L 1/2 regularization

In this paper, a novel method for synthetic aperture radar (SAR) imaging is proposed. The approach is based on L 1/2 regularization to reconstruct the scattering field, which optimizes a quadratic error term of the SAR observation process subject to the interested scene sparsity. Compared to the conventional SAR imaging technique, the new method implements SAR imaging effectively at much lower sampling rate than the Nyquist rate, and produces high-quality images with reduced sidelobes and increased resolution. Also, over the prevalent greedy pursuit and L 1 regularization based SAR imaging methods, there are remarkable performance improvements of the new method. On one hand, the new method significantly reduces the number of measurements needed for reconstruction, as supported by a phase transition diagram study. On the other hand, the new method is more robust to the observation noise. These fundamental properties of the new method are supported and demonstrated both by simulations and real SAR data experiments.

Model of a sparse encoding neuron

Neurons as Time Encoding Machines (TEMs) have been proposed to capture the information present in sensory stimuli and to encode it into spike trains [1]. These neurons, however, produce spikes at firing rates above Nyquist, which is usually much higher than the amount of information actually present in stimuli. We propose a low-rate neuron which exploits the sparsity or compressibility present in natural signals to produce spikes at a firing rate proportional to the amount of information present in the signal rather than its duration. We consider the IAF (Integrate-and-Fire) neuron circuit as presented in [1], provide appropriate modifications to convert it into a low-rate encoder and develop an algorithm for reconstructing the input stimulus using Compressive Sampling (CS) techniques. The class of input signals is assumed to be a mixture of periodic waveforms, consistent with the brain mechanism of generating and entraining oscillations at multiple frequencies (S in number) simultaneously. The LowRate IAF neuron circuit uses fixed thresholds (δ) as opposed to random thresholds used in [1]. The randomness in inter-spike-interval exhibited in spike trains is produced by an additional component that switches off the IAF circuit (mimicking the “absolute refractory” period) for a random amount of time (with mean μ) after each spike (see figure ​figure1A).1A). We compare the performance of our LowRate neuron firing at spike-rate K (which is determined by the parameters δ and μ) with IAF neurons in [1] operating at and above Nyquist rate N (>K). Because we inject additive white Gaussian noise into the input signal, we use the traditional measure of signal-to-noise ratio (SNR) as our performance metric. The recovery method developed is a greedy pursuit algorithm similar to the one described in [2]. Figure ​Figure1B1B plots the mean output SNR vs. input SNR for a signal with S = 10 frequencies and sparse-encoding ratio K/N = 0.3052. The LowRate IAF neuron (even when operating at about one third the Nyquist rate in this example) outperforms the IAF neurons operating at and above Nyquist rates. Figure ​Figure1C1C (for signals with S = 60) demonstrates that an increase in sparse-encoding ratio K/N improves the performance of LowRate IAF neuron. We are currently extending this methodology to signals sparse in other domains as well. Figure 1 CS IAF neuron A) circuit; output SNR vs. input SNR for B) signals with S = 10 C) signals with S = 60.

Joint Doppler and time delay estimation by compressed sampling

Joint Doppler shift and time delay estimation is an important topic in radar, sonar and communication applications. Least square (LS) is a classical and effective method for solving the problem. However, the performance degrades severely in the scenario of low ratio of signal-noise (SNR), due to the instability of matrix inversing. In this paper, a high-resolution method is proposed basing on the compressive sampling theory. The 2-dimension channel response can be sparsely recovered, and high-resolution Doppler shift-time delay estimation can be described with an underdetermined equation solving problem. Three categories of algorithms, including diagonal loading least squares, l1 Regularization, and Greedy Pursuit, are adopt to solve the problem and show outstanding resolving capabilities. The three categories of algorithms are analyzed and compared in different conditions. For the dictionary appears not to have unit norm columns, Greedy pursuit is not good as l1 Regularization in general, and simulation ...

A* orthogonal matching pursuit: Best-first search for compressed sensing signal recovery

Compressed sensing is a developing field aiming at reconstruction of sparse signals acquired in reduced dimensions, which make the recovery process under-determined. The required solution is the one with minimum $\ell_0$ norm due to sparsity, however it is not practical to solve the $\ell_0$ minimization problem. Commonly used techniques include $\ell_1$ minimization, such as Basis Pursuit (BP) and greedy pursuit algorithms such as Orthogonal Matching Pursuit (OMP) and Subspace Pursuit (SP). This manuscript proposes a novel semi-greedy recovery approach, namely A* Orthogonal Matching Pursuit (A*OMP). A*OMP performs A* search to look for the sparsest solution on a tree whose paths grow similar to the Orthogonal Matching Pursuit (OMP) algorithm. Paths on the tree are evaluated according to a cost function, which should compensate for different path lengths. For this purpose, three different auxiliary structures are defined, including novel dynamic ones. A*OMP also incorporates pruning techniques which enable practical applications of the algorithm. Moreover, the adjustable search parameters provide means for a complexity-accuracy trade-off. We demonstrate the reconstruction ability of the proposed scheme on both synthetically generated data and images using Gaussian and Bernoulli observation matrices, where A*OMP yields less reconstruction error and higher exact recovery frequency than BP, OMP and SP. Results also indicate that novel dynamic cost functions provide improved results as compared to a conventional choice.

Probabilistic greedy pursuit for streaming compressed spectrum sensing

This paper presents a probabilistic greedy pursuit (PGP) algorithm for compressed wide-band spectrum sensing under cognitive radio (CR) scenario. PGP relies on streaming compressed sensing (CS) framework, which differs from traditional CS processing way that only focuses on fixed-length signal's compressive sampling and reconstruction. It utilizes analog-to-information converter (AIC) to perform sub-Nyquist rate signal acquisition at the radio front-end (RF) of CR, the measurement process of which is carefully designed for streaming framework. Since the sparsity of wide-band spectrum is unavailable in practical situation, PGP introduces the probabilistic scheme by dynamically updating support confidence coefficient and utilizes greedy pursuit to perform streaming spectrum estimation, which gains sensing performance promotion progressively. The proposed algorithm enables robust spectrum estimation without the priori sparsity knowledge, and keeps low computational complexity simultaneously, which is more suitable for practical on-line applications. Various simulations and comparisons validate the effectiveness of our approach.