First-order Primal-dual Algorithm Research Articles

Nonconvex weighted variational metal artifacts removal via convergent primal-dual algorithms

Direct reconstruction through filtered back projection engenders metal artifacts in polychromatic computed tomography images, attributed to highly attenuating implants, which further poses great challenges for subsequent image analysis. Inpainting the metal trace directly in the Radon domain for the extant variational method leads to strong edge diffusion and potential inherent artifacts. With normalization based on pre-segmentation, the inpainted outcome can be notably ameliorated. However, its reconstructive fidelity is heavily contingent on the precision of the pre-segmentation, and highly accurate segmentation of images with metal artifacts is non-trivial in actuality. In this paper, we propose a nonconvex weighted variational approach for metal artifact reduction. Specifically, in lieu of employing a binary function with zeros in the metal trace, an adaptive weight function is designed in the Radon domain, with zeros in the overlapping regions of multiple disjoint metals as well as areas of highly attenuated projections, and the inverse square root of the measured projection in other regions. A nonconvex L1−αL2 regularization term is incorporated to further enhance edge contrast, alongside a box-constraint in the image domain. Efficient first-order primal-dual algorithms, proven to be globally convergent and of low computational cost owing to the closed-form solution of all subproblems, are devised to resolve such a constrained nonconvex model. Both simulated and real experiments are conducted with comparisons to other variational algorithms, validating the superiority of the presented method. Especially in comparison to Reweighted JSR, our proposed algorithm can curtail the total computational cost to at most one-third, and for the case of inaccurate pre-segmentation, the recovery outcomes by the proposed algorithms are notably enhanced.

First-order primal–dual algorithm for sparse-view neutron computed tomography-based three-dimensional image reconstruction

First-order primal–dual algorithm for sparse-view neutron computed tomography-based three-dimensional image reconstruction

First-order primal–dual algorithm for image restoration corrupted by mixed Poisson–Gaussian noise

The total variation infimal convolution (TV-IC) model combining Kullback–Leibler and ℓ2-norm data fidelity term works well for the inverse problem of mixed Poisson–Gaussian noise. Most existing algorithms for solving the TV-IC model rely on the Newton method to solve a nonlinear optimization subproblem, which inevitably increases the computation burden. In this study, we apply the first-order primal–dual Chambolle–Pock algorithm to solve the TV-IC model. In particular, we present an effective algorithm to solve the subproblem of the joint proximal operator with the Kullback–Leibler divergence and ℓ2-norm, which is based on the bilinear constraint alternating direction multiplier method. Numerical experiment results demonstrate that the proposed algorithm outperforms the state-of-the-art methods for mixed Poisson–Gaussian denoising and deblurring problems.

A new Lagrangian-based first-order method for nonconvex constrained optimization

In this paper, we introduce a new form of Lagrangian function and propose a simple first-order primal-dual algorithm for solving nonconvex optimization with nonlinear equality constraints. We show that the algorithm generates bounded primal-dual iterates, and establish the convergence to KKT points under standard assumptions. The key features of the proposed method are: (i) it does not require boundedness assumptions on dual iterates generated by the algorithm as well as the set of multipliers; (ii) it is a single-loop algorithm that does not involve any penalty subproblems.

Space-Time Electromagnetic Inverse Scattering Tomography of Objects With Affine Deformation

In this article, an electromagnetic inverse scattering problem (EISP) method is implemented successfully to capture the dynamic phenomena of the scene with affine deformation. In the presented method, a new variational model has been suggested which is based on two error functions of the current and state equations. Moreover, two types of regularizers have been adopted in the proposed scheme. In the first type, two separated spatial total variation (TV) and smooth, temporal regularization terms are used. In the second kind, a TV regularization, being a function of both spatial and temporal variables, is used. It is shown that the second type gives better results compared to the first type. The minimization problem is solved by the first-order primal-dual optimization algorithm of Chambolle and Pock.

WARPd: A Linearly Convergent First-Order Primal-Dual Algorithm for Inverse Problems with Approximate Sharpness Conditions

Sharpness conditions directly control the recovery performance of restart schemes for first-order optimization methods without the need for restrictive assumptions such as strong convexity. However, they are challenging to apply in the presence of noise or approximate model classes (e.g., approximate sparsity). We provide a first-order method: weighted, accelerated, and restarted primal-dual (WARPd), based on primal-dual iterations and a novel restart-reweight scheme. Under a generic approximate sharpness condition, WARPd achieves stable linear convergence to the desired vector. Many problems of interest fit into this framework. For example, we analyze sparse recovery in compressed sensing, low-rank matrix recovery, matrix completion, TV regularization, minimization of $\|Bx\|_{l^1}$ under constraints ($l^1$-analysis problems for general $B$), and mixed regularization problems. We show how several quantities controlling recovery performance also provide explicit approximate sharpness constants. Numerical experiments show that WARPd compares favorably with specialized state-of-the-art methods and is ideally suited for solving large-scale problems. We also present a noise-blind variant based on a square-root LASSO decoder. Finally, we show how to unroll WARPd as neural networks. This approximation theory result provides lower bounds for stable and accurate neural networks for inverse problems and sheds light on architecture choices. Code and a gallery of examples are available online as a MATLAB package.

Linear Convergence of First- and Zeroth-Order Primal–Dual Algorithms for Distributed Nonconvex Optimization

This article considers the distributed nonconvex optimization problem of minimizing a global cost function formed by a sum of local cost functions by using local information exchange. We first consider a distributed first-order primal–dual algorithm. We show that it converges sublinearly to a stationary point if each local cost function is smooth and linearly to a global optimum under an additional condition that the global cost function satisfies the Polyak–Łojasiewicz condition. This condition is weaker than strong convexity, which is a standard condition for proving linear convergence of distributed optimization algorithms, and the global minimizer is not necessarily unique. Motivated by the situations where the gradients are unavailable, we then propose a distributed zeroth-order algorithm, derived from the considered first-order algorithm by using a deterministic gradient estimator, and show that it has the same convergence properties as the considered first-order algorithm under the same conditions. The theoretical results are illustrated by numerical simulations.

Unified linear convergence of first-order primal-dual algorithms for saddle point problems

Unified linear convergence of first-order primal-dual algorithms for saddle point problems

On the linear convergence of the general first order primal-dual algorithm

<p style='text-indent:20px;'>In this paper, we consider the general first order primal-dual algorithm, which covers several recent popular algorithms such as the one proposed in [Chambolle, A. and Pock T., A first-order primal-dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vis., 40 (2011) 120-145] as a special case. Under suitable conditions, we prove its global convergence and analyze its linear rate of convergence. As compared to the results in the literature, we derive the corresponding results for the general case and under weaker conditions. Furthermore, the global linear rate of the linearized primal-dual algorithm is established in the same analytical framework.</p>

Accelerated Stochastic Algorithms for Convex-Concave Saddle-Point Problems

We develop stochastic first-order primal-dual algorithms to solve a class of convex-concave saddle-point problems. When the saddle function is strongly convex in the primal variable, we develop the first stochastic restart scheme for this problem. When the gradient noises obey sub-Gaussian distributions, the oracle complexity of our restart scheme is strictly better than any of the existing methods, even in the deterministic case. Furthermore, for each problem parameter of interest, whenever the lower bound exists, the oracle complexity of our restart scheme is either optimal or nearly optimal (up to a log factor). The subroutine used in this scheme is itself a new stochastic algorithm developed for the problem where the saddle function is nonstrongly convex in the primal variable. This new algorithm, which is based on the primal-dual hybrid gradient framework, achieves the state-of-the-art oracle complexity and may be of independent interest.

Cartoon and texture decomposition for color image in opponent color space

The Meyer model has been successfully applied to decompose cartoon component and texture component for the gray scale image, where the total variation (TV) norm and the G-norm are respectively modeled to capture the cartoon component and the texture component in an energy minimization method. In this paper, we extend this model to the color image in the opponent color space, which is closer to human perception than the RGB space. It is important to extend the TV norm and the G-norm correspondingly because the color image is viewed as a vector-valued vector. We introduce the definition of the L1 norm and L∞ norm for the vector-valued vector and accordingly define the TV norm and the G-norm for the color image. In order to handle the numerical difficulty caused by the non-differentiability of the TV norm and G-norm, the dual formulations are used to represent these norm. Then the decomposition problem is reformulated into a minimax problem. A first-order primal-dual algorithm is readily applied to compute the saddle point of the minimax problem. Numerical results are shown the performance of the proposed model.

A first-order inexact primal-dual algorithm for a class of convex-concave saddle point problems

In this paper, we study a first-order inexact primal-dual algorithm (I-PDA) for solving a class of convex-concave saddle point problems. The I-PDA, which involves a relative error criterion and generalizes the classical PDA, has the advantage of solving one subproblem inexactly when it does not have a closed-form solution. We show that the whole sequence generated by I-PDA converges to a saddle point solution with $\mathcal {O}(1/N)$ ergodic convergence rate, where N is the iteration number. In addition, under a mild calmness condition, we establish the global Q-linear convergence rate of the distance between the iterates generated by I-PDA and the solution set, and the R-linear convergence speed of the nonergodic iterates. Furthermore, we demonstrate that many problems arising from practical applications satisfy this calmness condition. Finally, some numerical experiments are performed to show the superiority and linear convergence behaviors of I-PDA.

Approximate first-order primal-dual algorithms for saddle point problems

Approximate first-order primal-dual algorithms for saddle point problems

X-Ray Phase-Contrast Image Reconstruction Based on the Chambolle-Pock Algorithm

The X-ray in-line phase-contrast imaging technique requires that the light source provides a spatially coherent X-ray beam typically generated by a microfocus X-ray tube in most laboratories. However, its small focal spot limits the lightness. An arrayed X-ray point source can overcome this drawback and provide an adequately bright spatial coherent light. However, the arrayed source will introduce artefacts to the phase-contrast image due to the point source spatial distribution. To eliminate these artefacts, an optimization-based algorithm is used to reconstruct the image as an optimization problem. An iterative method was developed to numerically solve the optimization problem by using the first-order primal-dual algorithm. The simulations demonstrate its convergence and reconstruction accuracy.

FlexPD: A Flexible Framework of First-Order Primal-Dual Algorithms for Distributed Optimization

In this paper, we study the problem of minimizing a sum of convex objective functions, which are locally available to agents in a network. Distributed optimization algorithms make it possible for the agents to cooperatively solve the problem through local computations and communications with neighbors. Lagrangian-based distributed optimization algorithms have received significant attention in recent years, due to their exact convergence property. However, many of these algorithms have slow convergence or are expensive to execute. In this paper, we propose a flexible framework of first-order primal-dual algorithms (FlexPD), which allows for an arbitrary number of primal steps per iteration. This framework includes three algorithms, FlexPD-F, FlexPD-G, and FlexPD-C that can be customized for various applications with different computation and communication requirements. For strongly convex and Lipschitz gradient objective functions, by carefully controlling the stepsize choices, we establish linear convergence of our proposed framework to the optimal solution. Simulation results confirm the superior performance of our framework compared to the existing methods.

A Double Extrapolation Primal-Dual Algorithm for Saddle Point Problems

The first-order primal-dual algorithms have received much considerable attention in the literature due to their quite promising performance in solving large-scale image processing models. In this paper, we consider a general saddle point problem and propose a double extrapolation primal-dual algorithm, which employs the efficient extrapolation strategy for both primal and dual variables. It is remarkable that the proposed algorithm enjoys a unified framework including several existing efficient solvers as special cases. Another exciting property is that, under quite flexible requirements on the involved extrapolation parameters, our algorithm is globally convergent to a saddle point of the problem under consideration. Moreover, the worst case $${{\mathcal {O}}}(1/t)$$ convergence rate in both ergodic and nonergodic senses, and the linear convergence rate can be established for more general cases, where t counts the iteration. Some computational results on solving image deblurring, image inpainting and the nearest correlation matrix problems further show that the proposed algorithm is efficient, and performs better than some existing first-order solvers in terms of taking less iterations and computing time in some cases.

Optimization-based algorithm for solving the discrete x-ray transform with nonlinear partial volume effect

Purpose: Inverting the discrete x-ray transform (DXT) with the nonlinear partial volume (NLPV) effect, which we refer to as the NLPV DXT, remains of theoretical and practical interest. We propose an optimization-based algorithm for accurately and directly inverting the NLPV DXT. Methods: Formulating the inversion of the NLPV DXT as a nonconvex optimization program, we propose an iterative algorithm, referred to as the nonconvex primal-dual (NCPD) algorithm, to solve the problem. We obtain the NCPD algorithm by modifying a first-order primal-dual algorithm to address the nonconvex optimization. Subsequently, we perform quantitative studies to verify and characterize the NCPD algorithm. Results: In addition to proposing the NCPD algorithm, we perform numerical studies to verify that the NCPD algorithm can reach the devised numerically necessary convergence conditions and, under the study conditions considered, invert the NLPV DXT by yielding numerically accurate image reconstruction. Conclusion: We have developed and verified with numerical studies the NCPD algorithm for accurate inversion of the NLPV DXT. The study and results may yield insights into the effective compensation for the NLPV artifacts in CT imaging and into the algorithm development for nonconvex optimization programs in CT and other tomographic imaging technologies.

Structure–texture image decomposition using a new non‐local TV‐Hilbert model

Combining the advantages of the non-local total variation (TV) and the Gabor function, a new Gabor function based non-local TV-Hilbert model is presented to separate the structure and texture components of the image. Computationally, by introducing the dual form of the non-local TV, the authors reformulate the non-local TV-Hilbert minimisation problem into a convex–concave saddle-point problem. In the aspect of solving algorithm, by transforming the Chambolle–Pock's first-order primal–dual algorithm into a different equivalent form. The authors propose a proximal-based primal–dual algorithm to solve the convex–concave saddle-point problem. At last, experimental results demonstrate that the proposed new model outperforms several existing state-of-the-art variational models.

A modified Chambolle-Pock primal-dual algorithm for Poisson noise removal

In this paper, we study the Poisson noise removal problem with total variation regularization term. Using the dual formulation of total variation and Lagrange dual, we formulate the problem as a new constrained minimax problem. Then, a modified Chambolle-Pock first-order primal-dual algorithm is developed to compute the saddle point of the minimax problem. The main idea of this paper is using different step size for different primal (dual) variables updating. Moreover, the convergence of the proposed method is also established under mild conditions. Numerical comparisons between new approach and several state-of-the-art algorithms are shown to demonstrate the effectiveness of the new algorithm.

A Block Successive Upper-Bound Minimization Method of Multipliers for Linearly Constrained Convex Optimization

Consider the problem of minimizing the sum of a smooth convex function and a separable nonsmooth convex function subject to linear coupling constraints. Problems of this form arise in many contemporary applications, including signal processing, wireless networking, and smart grid provisioning. Motivated by the huge size of these applications, we propose a new class of first-order primal–dual algorithms called the block successive upper-bound minimization method of multipliers (BSUM-M) to solve this family of problems. The BSUM-M updates the primal variable blocks successively by minimizing locally tight upper bounds of the augmented Lagrangian of the original problem, followed by a gradient-type update for the dual variable in closed form. We show that under certain regularity conditions, and when the primal block variables are updated in either a deterministic or a random fashion, the BSUM-M converges to a point in the set of optimal solutions. Moreover, in the absence of linear constraints and under similar conditions as in the previous result, we show that the randomized BSUM-M (which reduces to the randomized block successive upper-bound minimization method) converges at an asymptotically linear rate without relying on strong convexity.