Sparse Support Recovery Research Articles

Streaming Algorithms with Few State Changes

In this paper, we study streaming algorithms that minimize the number of changes made to their internal state (i.e., memory contents). While the design of streaming algorithms typically focuses on minimizing space and update time, these metrics fail to capture the asymmetric costs, inherent in modern hardware and database systems, of reading versus writing to memory. In fact, most streaming algorithms write to their memory on every update, which is undesirable when writing is significantly more expensive than reading. This raises the question of whether streaming algorithms with small space and number of memory writes are possible. We first demonstrate that, for the fundamental F p moment estimation problem with p ≥ 1, any streaming algorithm that achieves a constant factor approximation must make Ω(n 1-1/p ) internal state changes, regardless of how much space it uses. Perhaps surprisingly, we show that this lower bound can be matched by an algorithm which also has near-optimal space complexity. Specifically, we give a (1+ε)-approximation algorithm for F p moment estimation that use a near-optimal ~O ε (n 1-1/p ) number of state changes, while simultaneously achieving near-optimal space, i.e., for p∈[1,2), our algorithm uses poly(log n,1/ε) bits of space for, while for p>2, the algorithm uses ~O ε (n 1-1/p ) space. We similarly design streaming algorithms that are simultaneously near-optimal in both space complexity and the number of state changes for the heavy-hitters problem, sparse support recovery, and entropy estimation. Our results demonstrate that an optimal number of state changes can be achieved without sacrificing space complexity.

Deep Learning-Based Activity Detection for Grant-Free Random Access

The cellular Internet of Things wireless network is a promising solution to provide massive connectivity for machine-type devices. However, designing grant-free random access (GF-RA) protocols to manage such connections is challenging, since they must operate in interference-aware scenarios with sporadic device activation patterns and a shortage of mutually orthogonal resources. Supervised machine learning models have provided efficient solutions for activity detection, noncoherent data detection, and nonorthogonal preamble design in scenarios with massive connectivity. In this article, we develop two deep learning (DL) sparse support recovery algorithms to detect active devices in massive machine-type communication random access. The DL algorithms, developed to deploy GF-RA protocols, are based on the deep multilayer perceptron and the convolutional neural network models. Unlike previous works, we investigate the impact of the type of sequences for preamble design on the activity detection accuracy. Our results reveal that preambles based on the Zadoff–Chu sequences, which present good correlation properties, achieve better activity detection accuracy with the proposed algorithms than random sequences. Besides, we demonstrate that our DL algorithms achieve activity detection accuracy comparable to state-of-the-art techniques with extremely low computational complexity.

Joint Block Support Recovery for Sub-Nyquist Sampling Cooperative Spectrum Sensing

This letter aims to improve the efficiency and reliability of joint block sparse support recovery by exploiting block features in the MMV problem. For the block MMV problem, we first propose a Block SOMP algorithm extended from SOMP, which can reduce the number of iterations of the original method. Then, we perform an in-depth theoretical analysis of the proposed algorithm using Exact Recovery Condition (ERC). Both theoretical analysis and simulation experiments confirm that the proposed algorithm can provide reliable recovery for higher sparsity. However, the premise is that the design of the cooperative compressed sensing matrix requires special attention.

Correlation-Aware Joint Support Recovery With Separation Prior

Sparse support recovery from multiple measurement vectors (MMV) is studied in this paper. For years, a noticeable mismatch exists between the theories and algorithms in the research of super-resolution and direction-of-arrival (DOA) estimation that variants of separation conditions are assumed to ensure stable recovery while the corresponding algorithms rarely exploit such structural constraints. Due to the discrete nature of the separation condition, we propose to incorporate such prior information in a Mixed Integer Programming (MIP) problem with an <inline-formula><tex-math notation="LaTeX">$\ell _{0}$</tex-math></inline-formula>-based constraint. We develop a specialized branch and bound (B&amp;B) algorithm that can efficiently exploit the separation prior with guaranteed complexity reduction. Moreover, we show that computational complexity can be further reduced by leveraging the sparse array idea along with a particular perspective formulation of the MIP. The superior performance of the proposed algorithm is demonstrated via numerical experiments.

On Landscape of Nonconvex Regularized Least Squares for Sparse Support Recovery

Exact sparse support recovery from noisy deterministic Fourier measurements is studied in this paper. The ideal <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{0}$</tex-math></inline-formula> -regularized least squares (LS) program is known to have strict local minimizers for multiple sparse supports. By applying a specific non-convex surrogate of the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{0}$</tex-math></inline-formula> -regularizer, we show that, under certain conditions, the set of local minimizers with particularly separated supports is a singleton. Thus, any converging algorithms that can enforce the corresponding separation constraint will lead to the true support as the unique solution. In this regard, we construct a novel optimal mapping operator that can fulfill the separation condition, and several empirically effective algorithms are developed. The simulation results demonstrate the theoretical claims made in this paper.

Fast and Robust Sparsity Learning Over Networks: A Decentralized Surrogate Median Regression Approach

Decentralized sparsity learning has attracted a significant amount of attention recently due to its rapidly growing applications. To obtain the robust and sparse estimators, a natural idea is to adopt the non-smooth median loss combined with a $\ell_1$ sparsity regularizer. However, most of the existing methods suffer from slow convergence performance caused by the {\em double} non-smooth objective. To accelerate the computation, in this paper, we proposed a decentralized surrogate median regression (deSMR) method for efficiently solving the decentralized sparsity learning problem. We show that our proposed algorithm enjoys a linear convergence rate with a simple implementation. We also investigate the statistical guarantee, and it shows that our proposed estimator achieves a near-oracle convergence rate without any restriction on the number of network nodes. Moreover, we establish the theoretical results for sparse support recovery. Thorough numerical experiments and real data study are provided to demonstrate the effectiveness of our method.

Support Recovery for Sparse Multidimensional Phase Retrieval

We consider the \textit{phase retrieval} problem of recovering a sparse signal $\mathbf{x}$ in $\mathbb{R}^d$ from intensity-only measurements in dimension $d \geq 2$. Phase retrieval can be equivalently formulated as the problem of recovering a signal from its autocorrelation, which is in turn directly related to the combinatorial problem of recovering a set from its pairwise differences. In one spatial dimension, this problem is well studied and known as the \textit{turnpike problem}. In this work, we present MISTR (Multidimensional Intersection Sparse supporT Recovery), an algorithm which exploits this formulation to recover the support of a multidimensional signal from magnitude-only measurements. MISTR takes advantage of the structure of multiple dimensions to provably achieve the same accuracy as the best one-dimensional algorithms in dramatically less time. We prove theoretically that MISTR correctly recovers the support of signals distributed as a Gaussian point process with high probability as long as sparsity is at most $\mathcal{O}\left(n^{d\theta}\right)$ for any $\theta < 1/2$, where $n^d$ represents pixel size in a fixed image window. In the case that magnitude measurements are corrupted by noise, we provide a thresholding scheme with theoretical guarantees for sparsity at most $\mathcal{O}\left(n^{d\theta}\right)$ for $\theta < 1/4$ that obviates the need for MISTR to explicitly handle noisy autocorrelation data. Detailed and reproducible numerical experiments demonstrate the effectiveness of our algorithm, showing that in practice MISTR enjoys time complexity which is nearly linear in the size of the input.

On the rate of concentration of maxima in Gaussian arrays

Recently in Gao and Stoev (2020) it was established that the concentration of maxima phenomenon is the key to solving the exact sparse support recovery problem in high dimensions. This phenomenon, known also as relative stability, has been little studied in the context of dependence. Here, we obtain bounds on the rate of concentration of maxima in Gaussian triangular arrays. These results are used to establish sufficient conditions for the uniform relative stability of functions of Gaussian arrays, leading to new models that exhibit phase transitions in the exact support recovery problem. Finally, the optimal rate of concentration for Gaussian arrays is studied under general assumptions implied by the classic condition of Berman (1964).

Jointly Sparse Signal Recovery and Support Recovery via Deep Learning With Applications in MIMO-Based Grant-Free Random Access

In this article, we investigate jointly sparse signal recovery and jointly sparse support recovery in Multiple Measurement Vector (MMV) models for complex signals, which arise in many applications in communications and signal processing. Recent key applications include channel estimation and device activity detection in MIMO-based grant-free random access which is proposed to support massive machine-type communications (mMTC) for Internet of Things (IoT). Utilizing techniques in compressive sensing, optimization and deep learning, we propose two model-driven approaches, based on the standard auto-encoder structure for real numbers. One is to jointly design the common measurement matrix and jointly sparse signal recovery method, and the other aims to jointly design the common measurement matrix and jointly sparse support recovery method. The proposed model-driven approaches can effectively utilize features of sparsity patterns in designing common measurement matrices and adjusting model-driven decoders, and can greatly benefit from the underlying state-of-the-art recovery methods with theoretical guarantee. Hence, the obtained common measurement matrices and recovery methods can significantly outperform the underlying advanced recovery methods. We conduct extensive numerical results on channel estimation and device activity detection in MIMO-based grant-free random access. The numerical results show that the proposed approaches provide pilot sequences and channel estimation or device activity detection methods which can achieve higher estimation or detection accuracy with shorter computation time than existing ones. Furthermore, the numerical results explain how such gains are achieved via the proposed approaches.

Learning on Hypergraphs With Sparsity.

Hypergraph is a general way of representing high-order relations on a set of objects. It is a generalization of graph, in which only pairwise relations can be represented. It finds applications in various domains where relationships of more than two objects are observed. On a hypergraph, as a generalization of graph, one wishes to learn a smooth function with respect to its topology. A fundamental issue is to find suitable smoothness measures of functions on the nodes of a graph/hypergraph. We show a general framework that generalizes previously proposed smoothness measures and also generates new ones. To address the problem of irrelevant or noisy data, we wish to incorporate sparse learning framework into learning on hypergraphs. We propose sparsely smooth formulations that learn smooth functions and induce sparsity on hypergraphs at both hyperedge and node levels. We show their properties and sparse support recovery results. We conduct experiments to show that our sparsely smooth models are beneficial to learning irrelevant and noisy data, and usually give similar or improved performances compared to dense models.

Joint Design of Measurement Matrix and Sparse Support Recovery Method via Deep Auto-Encoder

Sparse support recovery arises in many applications in communications and signal processing. Existing methods tackle sparse support recovery problems for a given measurement matrix, and cannot flexibly exploit the properties of sparsity patterns for improving performance. In this letter, we propose a data-driven approach to jointly design the measurement matrix and support recovery method for complex sparse signals, using auto-encoder in deep learning. The proposed architecture includes two components, an auto-encoder and a hard thresholding module. The proposed auto-encoder successfully handles complex signals using standard auto-encoder for real numbers. The proposed approach can effectively exploit properties of sparsity patterns, and is especially useful when these underlying properties do not have analytic models. In addition, the proposed approach can achieve sparse support recovery with low computational complexity. Experiments are conducted on an application ex-ample, device activity detection in grant-free massive access for massive machine type communications (mMTC). Numerical results show that the proposed approach achieves significantly better performance with much less computation time than classic methods, in the presence of extra structures in sparsity patterns.

Learning Simple Thresholded Features With Sparse Support Recovery

The thresholded feature has recently emerged as an extremely efficient, yet rough empirical approximation, of the time-consuming sparse coding inference process. Such an approximation has not yet been rigorously examined, and standard dictionaries often lead to non-optimal performance when used for computing thresholded features. In this paper, we first present two theoretical recovery guarantees for the thresholded feature to exactly recover the nonzero support of the sparse code. Motivated by them, we then formulate the Dictionary Learning for Thresholded Features (DLTF) model, which learns an optimized dictionary for applying the thresholded feature. In particular, for the $(k, 2)$ norm involved, a novel proximal operator with log-linear time complexity $O(m\log m)$ is derived. We evaluate the performance of DLTF on a vast range of synthetic and real-data tasks, where DLTF demonstrates remarkable efficiency, effectiveness and robustness in all experiments. In addition, we briefly discuss the potential link between DLTF and deep learning building blocks.

Time varying sparse support recovery

In the applications like spatial modulation in multi-antenna systems, multi-user detection in sensor networks and massive random access in machine to machine communications, the set of nodes which transmit data to the receiver varies with time. Also, the number of ‘active’ nodes is very less as compared to the total number of nodes, thereby inducing time-varying sparsity in the system. At the receiver, a major challenge is to track the set of active nodes at each instance and also estimate the data transmitted by each node. In this work, we use compressed sensing framework to employ a two-level approach for tracking the dynamic supports. In this approach, we consider multiple measurement vectors to facilitate the joint sparse support recovery in the first level. In the second level, we refine the supports using the modified Least squares and QR decomposition techniques. For empirical study, we consider the effect of increasing the number of measurement vectors for slow and fast rate of sparsity variations and also study the consequence of unknown signal variance on the recovery performance. Additionally, we observe significant improvement in the performance of the proposed methods when compared with the Structured Matching Pursuit method and the rank evolution approach.

Joint Sparsity Pattern Recovery With 1-b Compressive Sensing in Distributed Sensor Networks

In this paper, we study the problem of joint sparse support recovery with 1-b quantized compressive measurements in a distributed sensor network. Multiple nodes in the network are assumed to observe sparse signals having the same but unknown sparse support. Each node quantizes its measurement vector element-wise to 1 b. First, we consider that all the quantized measurements are available at a central fusion center. We derive performance bounds for sparsity pattern recovery using 1-bit quantized measurements from multiple sensors when the maximum likelihood decoder is employed. We further develop two computationally tractable algorithms for joint sparse support recovery in the centralized setting. One algorithm minimizes a cost function defined as the sum of the likelihood function and the $l_{1,\infty }$ quasi-norm, while the other algorithm extends the binary iterative hard thresholding algorithm to the multiple measurement vector case. Second, we consider a decentralized setting where each node transmits 1-b measurements to its one-hop neighbors. The basic idea behind the algorithms developed in the decentralized setting is to embed collaboration among nodes and fusion strategies. We show that even with noisy 1-b compressed measurements, joint support recovery can be carried out accurately in both centralized and decentralized settings. We further show that the performance of the proposed 1-bit compressive sensing-based algorithms is very close to that of their real-valued counterparts except when the signal-to-noise ratio is very small.

Photoacoustic imaging beyond the acoustic diffraction-limit with dynamic speckle illumination and sparse joint support recovery.

In deep tissue photoacoustic imaging the spatial resolution is inherently limited by the acoustic wavelength. Recently, it was demonstrated that it is possible to surpass the acoustic diffraction limit by analyzing fluctuations in a set of photoacoustic images obtained under unknown speckle illumination patterns. Here, we purpose an approach to boost reconstruction fidelity and resolution, while reducing the number of acquired images by utilizing a compressed sensing computational reconstruction framework. The approach takes into account prior knowledge of the system response and sparsity of the target structure. We provide proof of principle experiments of the approach and demonstrate that improved performance is obtained when both speckle fluctuations and object priors are used. We numerically study the expected performance as a function of the measurement's signal to noise ratio and sample spatial-sparsity. The presented reconstruction framework can be applied to analyze existing photoacoustic experimental data sets containing dynamic fluctuations.

Sparse support recovery using correlation information in the presence of additive noise

The correlation based framework has recently been proposed for sparse support recovery in noiseless case. To solve this framework, the constrained least absolute shrinkage and selection operator (LASSO) was employed. The regularization parameter in the constrained LASSO was found to be a key to the recovery. This paper will discuss the sparse support recoverability via the framework and adjustment of the regularization parameter in noisy case. The main contribution is to provide noise-related conditions to guarantee the sparse support recovery. It is pointed out that the candidates of the regularization parameter taken from the noise-related region can achieve the optimization and the effect of the noise cannot be ignored. When the number of the samples is finite, the sparse support recoverability is further discussed by estimating the recovery probability for the fixed regularization parameter in the region. The asymptotic consistency is obtained in probabilistic sense when the number of the samples tends to infinity. Simulations are given to demonstrate the validity of our results.

Asymptotic of Sparse Support Recovery for Positive Measures

We study sparse spikes deconvolution over the space of Radon measures when the input measure is a finite sum of positive Dirac masses using the BLASSO convex program. We focus on the recovery properties of the support and the amplitudes of the initial measure in the presence of noise when the minimum separation distance t of the input measure (the minimum distance between two spikes) tends to zero. We show that when ||ω||2/λ, ||ω||2/t2N-1 and λ/t2N-1 are small enough (where λ is the regularization parameter, ω the noise and N the number of spikes), which corresponds roughly to a sufficient signal-to-noise ratio and a noise level and a regularization parameter small enough with respect to the minimum separation distance, there exists a unique solution to the BLASSO program with exactly the same number of spikes as the original measure. We provide an upper bound on the error with respect to the initial measure. As a by-product, we show that the amplitudes and positions of the spikes of the solution both converge towards those of the input measure when λ and ω drop to zero faster than t2N-1.

Subspace Recovery From Structured Union of Subspaces

Lower dimensional signal representation schemes frequently assume that the signal of interest lies in a single vector space. In the context of the recently developed theory of compressive sensing, it is often assumed that the signal of interest is sparse in an orthonormal basis. However, in many practical applications, this requirement may be too restrictive. A generalization of the standard sparsity assumption is that the signal lies in a union of subspaces. Recovery of such signals from a small number of samples has been studied recently in several works. Here, we consider the problem of only subspace recovery in which our goal is to identify the subspace (from the union) in which the signal lies using a small number of samples, in the presence of noise. More specifically, we derive performance bounds and conditions under which reliable subspace recovery is guaranteed using maximum likelihood (ML) estimation. We begin by treating general unions and then obtain the results for the special case in which the subspaces have structure leading to block sparsity. In our analysis, we treat both general sampling operators and random sampling matrices. With general unions, we show that under certain conditions, the number of measurements required for reliable subspace recovery in the presence of noise via ML is less than that implied using the restricted isometry property, which guarantees complete signal recovery. In the special case of block sparse signals, we quantify the gain achievable over standard sparsity in subspace recovery. Our results also strengthen existing results on sparse support recovery in the presence of noise under the standard sparsity model.

Pushing the Limits of Sparse Support Recovery Using Correlation Information

A new framework for the problem of sparse support recovery is proposed, which exploits statistical information about the unknown sparse signal in the form of its correlation. A key contribution of this paper is to show that if existing algorithms can recover sparse support of size s, then using such correlation information, the guaranteed size of recoverable support can be increased to O(s <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ), although the sparse signal itself may not be recoverable. This is proved to be possible by (a) formulating the sparse support recovery problem in terms of the covariance matrix of the measurements, and (b) designing a suitable measurement/sampling matrix which inherently exploits the correlation priors. The so-called Khatri-Rao product of the measurement matrix is shown to play an important role in deciding the level of recoverable sparsity. A systematic analysis of the proposed framework is also presented for the cases when the covariance matrix is only approximately known, by estimating it from finite number of measurements, obtained from the Multiple Measurement Vector (MMV) model. In this case, the use of LASSO on the estimated covariance matrix is proposed for recovering the support. However, the recovery may not be exact and hence a probabilistic guarantee is developed both for sources with arbitrary distribution as well as for Gaussian sources. In the latter case, it is shown that such recovery can happen with overwhelming probability as the number of available measurement vectors increases.

High density 3D localization microscopy using sparse support recovery.

Single-molecule localization microscopy methods offer high spatial resolution, but they are not always suitable for live cell imaging due to limited temporal resolution. One strategy is to increase the density of photoactivated molecules present in each image, however suitable analysis algorithms for such data are still lacking. We present 3denseSTORM, a new algorithm for localization microscopy which is able to recover 2D or 3D super-resolution images from a sequence of diffraction limited images with high densities of photoactivated molecules. The algorithm is based on sparse support recovery and uses a Poisson noise model, which becomes critical in low-light conditions. For 3D data reconstruction we use the astigmatism and biplane imaging methods. We derive the theoretical resolution limits of the method and show examples of image reconstructions in simulations and in real 2D and 3D biological samples. The method is suitable for fast image acquisition in densely labeled samples and helps facilitate live cell studies with single molecule localization microscopy.