Expectation maximization hard thresholding methods for sparse signal reconstruction

Abstract

Reconstructing a high dimensional sparse signal from low dimensional linear measurements has been an important problem in various research disciplines including statistics, machining learning, data mining and signal processing. In this dissertation, we develop a probabilistic framework for sparse signal reconstruction and propose several novel algorithms for computing the maximum likelihood (ML) estimates under this framework. We first consider an underdetermined linear model where the regression-coefficient vector is the sum of an unknown deterministic sparse signal component and a zero-mean white Gaussian component with an unknown variance. Our reconstruction schemes are based on an expectation-conditional maximization either (ECME) iteration that aims at maximizing the likelihood function with respect to the unknown parameters for a given signal sparsity level. We propose a double overrelaxation (DORE) thresholding scheme for accelerating the ECME iteration and prove that, under certain conditions, the ECME and DORE iterations converge to local maxima of the likelihood function and achieve near-optimal or perfect recovery of approximately sparse or sparse signals, respectively. If the signal sparsity level is unknown, we introduce an unconstrained sparsity selection (USS) criterion and a tuning-free automatic double overrelaxation (ADORE) thresholding method that employs USS to estimate the sparsity level. In applications such as tomographic imaging, the signal of interest is nonnegative. We modify our probabilistic model and the ECME algorithm to incorporate the additional signal nonnegativity constraint. The maximization step of the modified ECME iteration is approximated by a difference map iteration. We compare the proposed and existing sparse reconstruction methods using simulated and real-data tomographic imaging experiments. Finally, we develop a generalized expectation-maximization (GEM) algorithm for sparse