An Accelerated Preconditioned Primal-Dual Gradient Algorithm for Nonconvex Composite Optimization Problems with Applications

The past decade has witnessed the emergence of compressed sensing as a way of acquiring sparsely representable signals in a compressed form. These developments have greatly motivated research in sparse signal recovery, which lies at the heart of compressed sensing, and which has recently found its use in altogether new applications. In the first part of this thesis we study the theoretical aspects of jointsparse recovery by means of sum-of-norms minimization, and the ReMBo-`1 algorithm, which combines boosting techniques with `1-minimization. For the sum-of-norms approach we derive necessary and sufficient conditions for recovery, by extending existing results to the joint-sparse setting. We focus in particular on minimization of the sum of `1, and `2 norms, and give concrete examples where recovery succeeds with one formulation but not with the other. We base our analysis of ReMBo-`1 on its geometrical interpretation, which leads to a study of orthant intersections with randomly oriented subspaces. This work establishes a clear picture of the mechanics behind the method, and explains the different aspects of its performance. The second part and main contribution of this thesis is the development of a framework for solving a wide class of convex optimization problems for sparse recovery. We provide a detailed account of the application of the framework on several problems, but also consider its limitations. The framework has been implemented in the spgl1 algorithm, which is already well established as an effective solver. Numerical results show that our algorithm is state-of-the-art, and compares favorably even with solvers for the easier—but less natural— Lagrangian formulations. The last part of this thesis discusses two supporting software packages: sparco, which provides a suite of test problems for sparse recovery, and spot, a Matlab toolbox for the creation and manipulation of linear operators. spot greatly facilitates rapid prototyping in sparse recovery and compressed sensing, where linear operators form the elementary building blocks. Following the practice of reproducible research, all code used for the experiments and generation of figures is available online at http://www.cs.ubc.ca/labs/scl/thesis/09vandenBerg/

We propose a new sparsity-promoting objective function to be used in sparse signal recovery. Specifically, the objective is an entropy function 𝑙1 defined on the sparse signal x. Compared to the conventional 𝑙1, it is a nonconvex function and the optimization problem can be solved based on the fast iterative shrinkage thresholding algorithm (FISTA). Experiments on 1-dimensional sparse signal recovery and 2-dimensional real image recovery show that minimizing 𝑙p favors sparse solutions, and that it could recover sparse signals better than the convex 𝑙1 norm minimization and the nonconvex l p -norm minimization.

It is now well understood that (1) it is possible to reconstruct sparse signals exactly from what appear to be highly incomplete sets of linear measurements and (2) that this can be done by constrained ℓ 1 minimization.In this paper, we study a novel method for sparse signal recovery that in many situations outperforms ℓ 1 minimization in the sense that substantially fewer measurements are needed for exact recovery.The algorithm consists of solving a sequence of weighted ℓ 1 -minimization problems where the weights used for the next iteration are computed from the value of the current solution.We present a series of experiments demonstrating the remarkable performance and broad applicability of this algorithm in the areas of sparse signal recovery, statistical estimation, error correction and image processing.Interestingly, superior gains are also achieved when our method is applied to recover signals with assumed near-sparsity in overcomplete representations-not by reweighting the ℓ 1 norm of the coefficient sequence as is common, but by reweighting the ℓ 1 norm of the transformed object.An immediate consequence is the possibility of highly efficient data acquisition protocols by improving on a technique known as compressed sensing.

It is now well understood that (1) it is possible to reconstruct sparse signals exactly from what appear to be highly incomplete sets of linear measurements and (2) that this can be done by constrained l1 minimization. In this paper, we study a novel method for sparse signal recovery that in many situations outperforms l1 minimization in the sense that substantially fewer measurements are needed for exact recovery. The algorithm consists of solving a sequence of weighted l1-minimization problems where the weights used for the next iteration are computed from the value of the current solution. We present a series of experiments demonstrating the remarkable performance and broad applicability of this algorithm in the areas of sparse signal recovery, statistical estimation, error correction and image processing. Interestingly, superior gains are also achieved when our method is applied to recover signals with assumed near-sparsity in overcomplete representations—not by reweighting the l1 norm of the coefficient sequence as is common, but by reweighting the l1 norm of the transformed object. An immediate consequence is the possibility of highly efficient data acquisition protocols by improving on a technique known as Compressive Sensing.

Finding sparse solutions to a system of equations and/or inequalities is an important topic in many application areas such as signal processing, statistical regression and nonparametric modeling. Various continuous relaxation models have been proposed and widely studied to deal with the discrete nature of the underlying problem. In this paper, we propose a quadratically constrained [Formula: see text] (0 < q < 1) minimization model for finding sparse solutions to a quadratic system. We prove that solving the proposed model is strongly NP-hard. To tackle the computation difficulty, a first order necessary condition for local minimizers is derived. Various properties of the proposed model are studied for designing an active-set-based descent algorithm to find candidate solutions satisfying the proposed condition. In addition to providing a theoretical convergence proof, we conduct extensive computational experiments using synthetic and real-life data to validate the effectiveness of the proposed algorithm and to show the superior capability in finding sparse solutions of the proposed model compared with other known models in the literature. We also extend our results to a quadratically constrained [Formula: see text] (0 < q < 1) minimization model with multiple convex quadratic constraints for further potential applications.Summary of Contribution: In this paper, we propose and study a quadratically constrained [Formula: see text] minimization (0 < q < 1) model for finding sparse solutions to a quadratic system which has wide applications in sparse signal recovery, image processing and machine learning. The proposed quadratically constrained [Formula: see text] minimization model extends the linearly constrained [Formula: see text] and unconstrained [Formula: see text]-[Formula: see text] models. We study various properties of the proposed model in aim of designing an efficient algorithm. Especially, we propose an unrelaxed KKT condition for local/global minimizers. Followed by the properties studied, an active-set based descent algorithm is then proposed with its convergence proof being given. Extensive numerical experiments with synthetic and real-life Sparco datasets are conducted to show that the proposed algorithm works very effectively and efficiently. Its sparse recovery capability is superior to that of other known models in the literature.

Robust signal recovery algorithm for structured perturbation compressive sensing

In this paper, we consider a broad class of nonconvex and nonsmooth composition optimization problems that can be used to model many applications in signal processing and image processing, such as sparse signal recovery and image restoration. However, due to the nonconvex nonsmooth properties of the objective function, solving this class of problems using classical methods like alternating minimization will face challenges in theoretical analysis and numerical calculation. For this, we propose a proximal alternating partially linearized minimization (PAPLM) algorithm by linearizing the nonconvex term and combining it with the traditional proximal algorithm. This algorithm enjoys simple and well-defined updates. By leveraging the Kurdyka-Łojasiewicz property, we prove that any sequence generated by the PAPLM algorithm globally converges to a critical point of the objective function under weaker assumptions. Numerical experiments on perturbed compressed sensing problems suggest that the proposed algorithm can achieve superior performance.

This paper presents a new fast iterative shrinkage-thresholding algorithm, termed AFISTA. The essential idea is to improve the convergence rate of FISTA using a new continuation strategy leading to a less number of iterations compared to FISTA. The convergence theorem of the AFISTA is proposed. In order to further accelerate the AFISTA method, it is equipped with the Barzilai-Borwein (BB) method. Also, for applications with orthogonal sensing matrix A, we proposed a specialized version of the AFISTA method. AFISTA is tailored for solving the basis pursuit problem which can be applied successfully on a variety of problems arising in signal and image processing issues such as sparse signal recovery, signal and image denoising, image restoration, and compressive sensing. To show the efficiency of the method, we compare our results with generalizations of linearized Bregman and fixed - point continuation (FPC) methods in sparse signal recovery applications, with split Bregman method in compressive sensing for sparse MRI and with Gradient projection for sparse reconstruction (GPSR) method in image deconvolution. Numerical results demonstrate that AFISTA overcomes all of the compared methods in convergence rate and some of them in both convergence rate and quality of reconstructed results.

Visualization and quantitative evaluation of delamination defects in GFRPs via sparse millimeter-wave imaging and image processing

Sparse aperture radar imaging is generally achieved by methods of compressive sensing (CS), or, sparse signal recovery (SSR). However, most of the traditional SSR methods cannot produce focused image stably, which limits their applications. l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> regularization and alternating direction method of multipliers(ADMM) are generally applied to the SSR problem, but its performance is sensitive to the selection of model parameters. This paper proposes a complex-valued ADMM-Net(CV-ADMMN) method to improve the stability of ADMM, and utilize it to achieve sparse aperture ISAR imaging and autofocusing. Firstly, the iterative procedure of ADMM is unrolled to be a deep network structure. Then, the parameters of the model are learned from a training dataset by utilizing an l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> regularized loss function. Finally, an autofocusing module based on entropy-minimization is plugged into the trained model to compensate the phase error. Experimental results based on both simulated and measured data validate the superiority of the proposed method over ADMM.

Sparse signal recovery via Kalman-filter-based ℓ1 minimization

A novel direction of arrival (DOA) estimation method for four-dimensional (4-D) linear arrays with optimized time sequences using the sparse signal recovery is proposed in this paper. By establishing the sparse signal recovery model applied to DOA estimation of 4-D linear arrays, it is found that the time sequences have great influence on sparse signal recovery. Thus mutual coherence and covariance matrix of noise are introduced to evaluate its influence quantificationally. The differential evolutionary (DE) algorithm is then used for the optimization of time sequences such that the capability of sparse signal recovery is improved and the noise after time modulation is close to Gaussian white noise. It can be found that the unidirectional phase center motion (UPCM) scheme is an excellent candidate by the optimization of an 8-element 4-D linear array. By comparison with the previous methods, simulated results demonstrate that the proposed method has distinct advantages over other methods in probability of resolution and estimation accuracy, especially for the cases where the number of snapshots is small and the SNR is low. Furthermore, an S band 4-D linear array with eight elements is designed and measured to demonstrate the performance of DOA estimation by the proposed method.

Compressive sensing (CS) enables the acquisition and recovery of sparse signals and images at sampling rates significantly below the classical Nyquist rate. Despite significant progress in the theory and methods of CS, little headway has been made in compressive video acquisition and recovery. Video CS is complicated by the ephemeral nature of dynamic events, which makes direct extensions of standard CS imaging architectures and signal models difficult. In this paper, we develop a new framework for video CS for dynamic textured scenes that models the evolution of the scene as a linear dynamical system (LDS). This reduces the video recovery problem to first estimating the model parameters of the LDS from compressive measurements and then reconstructing the image frames. We exploit the low-dimensional dynamic parameters (the state sequence) and high-dimensional static parameters (the observation matrix) of the LDS to devise a novel compressive measurement strategy that measures only the time-varying parameters at each instant and accumulates measurements over time to estimate the time-invariant parameters. This enables us to lower the compressive measurement rate considerably. We validate our approach and demonstrate its effectiveness with a range of experiments involving video recovery and scene classification.

As a kind of high speed rotation object, precession target is faced with migration through resolution cell (MTRC) in long synthetic aperture while using translational inverse synthetic aperture radar (ISAR) imaging algorithm. Compressed sensing (CS), by which we can exact recovery sparse signal from very limited samples, suggests that sparse aperture imaging of precession target maybe achievable. A cyclic shift algorithm based on CS is proposed in this paper to exploit the sparse apertures data for high-resolution ISAR imaging. The sparse signal recovery and imaging of precession target is achieved coupled with FOCUSS (focal undetermined system solver) algorithm. A conventional ISAR imaging is a two-dimensional (2-D) range-Doppler projection of a target and does not provide three-dimensional (3-D) information which is more reliable. For missile shaped like a flat-bottom cone, multistatic ISAR geometry model is built, and a 3-D reconstruction method, which is featured with stable structure characteristics, is proposed based on multistatic ISAR images. Simulation and real data results verify the validity and superiority of the proposed method.

In this paper, we present novel yet simple homotopy proximal mapping algorithms for reconstructing a sparse signal from (noisy) linear measurements of the signal or for learning a sparse linear model from observed data, where the former task is well-known in the field of compressive sensing and the latter task is known as model selection in statistics and machine learning. The algorithms adopt a simple proximal mapping of the $$\ell _1$$ norm at each iteration and gradually reduces the regularization parameter for the $$\ell _1$$ norm. We prove a global linear convergence of the proposed homotopy proximal mapping (HPM) algorithms for recovering the sparse signal under three different settings (i) sparse signal recovery under noiseless measurements, (ii) sparse signal recovery under noisy measurements, and (iii) nearly-sparse signal recovery under sub-Gaussian noisy measurements. In particular, we show that when the measurement matrix satisfies restricted isometric properties (RIP), one of the proposed algorithms with an appropriate setting of a parameter based on the RIP constants converges linearly to the optimal solution up to the noise level. In addition, in setting (iii), a practical variant of the proposed algorithms does not rely on the RIP constants and our results for sparse signal recovery are better than the previous results in the sense that our recovery error bound is smaller. Furthermore, our analysis explicitly exhibits that more observations lead to not only more accurate recovery but also faster convergence. Finally our empirical studies provide further support for the proposed homotopy proximal mapping algorithm and verify the theoretical results.