Primal-dual Splitting Method Research Articles

Proximal Alternating Direction Method of Multipliers with Convex Combination Proximal Centers

Proximal alternating direction method of multipliers (PADMM) is a classical primal-dual splitting method for solving separable convex optimization problems with linear equality constraints, which have numerous applications in, e.g., signal and image processing, machine learning, and statistics. In this paper, we propose a new variant of PADMM, called PADMC, whose proximal centers are constructed by convex combinations of the iterates. PADMC is able to take advantage of problem structures and preserves the desirable properties of the classical PADMM. We establish iterate convergence as well as [Formula: see text] ergodic and [Formula: see text] nonergodic sublinear convergence rate results measured by function residual and feasibility violation, where [Formula: see text] denotes the iteration number. Moreover, we propose two fast variants of PADMC, one achieves faster [Formula: see text] ergodic convergence rate when one of the component functions is strongly convex, and the other ensures faster [Formula: see text] nonergodic convergence rate measured by constraint violation. Finally, preliminary numerical results on the LASSO and the elastic-net regularization problems are presented to demonstrate the performance of the proposed methods.

Learning Sparse Graph with Minimax Concave Penalty under Gaussian Markov Random Fields

This paper presents a convex-analytic framework to learn sparse graphs from data. While our problem formulation is inspired by an extension of the graphical lasso using the so-called combinatorial graph Laplacian framework, a key difference is the use of a nonconvex alternative to the ℓ1 norm to attain graphs with better interpretability. Specifically, we use the weakly-convex minimax concave penalty (the difference between the ℓ1 norm and the Huber function) which is known to yield sparse solutions with lower estimation bias than ℓ1 for regression problems. In our framework, the graph Laplacian is replaced in the optimization by a linear transform of the vector corresponding to its upper triangular part. Via a reformulation relying on Moreau's decomposition, we show that overall convexity is guaranteed by introducing a quadratic function to our cost function. The problem can be solved efficiently by the primal-dual splitting method, of which the admissible conditions for provable convergence are presented. Numerical examples show that the proposed method significantly outperforms the existing graph learning methods with reasonable computation time.

A fast inertial primal–dual algorithm to composite optimization models with application to image restoration problems

In this paper, we consider a fast inertial primal–dual algorithm (FIPD) for finding minimization problems of the sum of a smooth function with Lipschitzian gradient, a non-smooth proximable function, and linear composite functions. We not only prove the convergence of the proposed algorithm, but also prove that the new algorithm can achieve the worst-case o(1/k2) optimal convergence rate in terms of objective function value. This work brings together and notably extends several classical splitting schemes, like the primal–dual splitting method (PDS) proposed by Condat, the algorithm which is presented by Chambolle and Pock, as well as the recent modified fast iterative shrinkage-thresholding algorithm. The efficiency of the proposed method is demonstrated on image denoising and image deblurring. Numerical results show that our iterative algorithm (FIPD) performs better than the original one (PDS).

Deblurring galaxy images with Tikhonov regularization on magnitude domain

Abstract We propose a regularization-based deblurring method that works efficiently for galaxy images. The spatial resolution of a ground-based telescope is generally limited by seeing conditions and is much worse than space-based telescopes. This circumstance has generated considerable research interest in the restoration of spatial resolution. Since image deblurring is a typical inverse problem and often ill-posed, solutions tend to be unstable. To obtain a stable solution, much research has adopted regularization-based methods for image deblurring, but the regularization term is not necessarily appropriate for galaxy images. Although galaxies have an exponential or Sérsic profile, the conventional regularization assumes the image profiles to behave linearly in space. The significant deviation between the assumption and real situations leads to blurring of the images and smoothing out the detailed structures. Clearly, regularization on logarithmic domain, i.e., magnitude domain, should provide a more appropriate assumption, which we explore in this study. We formulate a problem of deblurring galaxy images by an objective function with a Tikhonov regularization term on a magnitude domain. We introduce an iterative algorithm minimizing the objective function with a primal–dual splitting method. We investigate the feasibility of the proposed method using simulation and observation images. In the simulation, we blur galaxy images with a realistic point spread function and add both Gaussian and Poisson noise. For the evaluation with the observed images, we use galaxy images taken by the Subaru HSC-SSP. Both of these evaluations show that our method successfully recovers the spatial resolution of the deblurred images and significantly outperforms the conventional methods. The code is publicly available from the GitHub 〈https://github.com/kzmurata-astro/PSFdeconv_amag〉.

Dense-Frequency Signal-Detection Based on the Primal-Dual Splitting Method.

Aiming to solve the problem of dense-frequency signals in the power system caused by the growing proportion of new energy, this paper proposes a dense-frequency signal-detection method based on the primal–dual splitting method. After establishing the Taylor–Fourier model of the signal, the proposed method uses the sparse property of the coefficient matrix to obtain the convex optimization form of the model. Then, the optimal solution of the estimated phasor is obtained by iterating over the fixed-point equation, finally acquiring the optimal estimation result for the dense signal. When representing the Taylor–Fourier model as a convex optimization form, the introduction of measuring-error entropy makes the solution of the model more rigorous. It can be further verified through simulation experiments that the estimation accuracy of the primal–dual splitting method proposed in this paper for dense signals can meet the M-class PMU accuracy requirements.

Graph Spatio-Spectral Total Variation Model for Hyperspectral Image Denoising

The spatio-spectral total variation (SSTV) model has been widely used as an effective regularization of hyperspectral images (HSI) for various applications such as mixed noise removal. However, since SSTV computes local spatial differences uniformly, it is difficult to remove noise while preserving complex spatial structures with fine edges and textures, especially in situations of high noise intensity. To solve this problem, we propose a new TV-type regularization called Graph-SSTV (GSSTV), which generates a graph explicitly reflecting the spatial structure of the target HSI from noisy HSIs and incorporates a weighted spatial difference operator designed based on this graph. Furthermore, we formulate the mixed noise removal problem as a convex optimization problem involving GSSTV and develop an efficient algorithm based on the primal-dual splitting method to solve this problem. Finally, we demonstrate the effectiveness of GSSTV compared with existing HSI regularization models through experiments on mixed noise removal. The source code will be available at https://www.mdi.c.titech.ac.jp/publications/gsstv.

Robust Hyperspectral Image Fusion With Simultaneous Guide Image Denoising via Constrained Convex Optimization

The paper proposes a new high spatial resolution hyperspectral (HR-HS) image estimation method based on convex optimization. The method assumes a low spatial resolution HS (LR-HS) image and a guide image as observations, where both observations are contaminated by noise. Our method simultaneously estimates an HR-HS image and a noiseless guide image, so the method can utilize spatial information in a guide image even if it is contaminated by heavy noise. The proposed estimation problem adopts hybrid spatio-spectral total variation as regularization and evaluates the edge similarity between HR-HS and guide images to effectively use apriori knowledge on an HR-HS image and spatial detail information in a guide image. To efficiently solve the problem, we apply a primal-dual splitting method. Experiments demonstrate the performance of our method and the advantage over several existing methods.

Robust Recovery of Jointly-Sparse Signals Using Minimax Concave Loss Function

We propose a robust approach to recovering jointly sparse signals in the presence of outliers. The robust recovery task is cast as a convex optimization problem involving a minimax concave loss function (which is weakly convex) and a strongly convex regularizer (which ensures the overall convexity). The use of the nonconvex loss makes the problem difficult to solve directly by the convex optimization methods even with the well-established firm shrinkage. We circumvent this difficulty by reformulating the problem via the Moreau decomposition so that the objective function becomes a sum of convex functions that can be minimized by the primal-dual splitting method. The parameter designs/ranges for the present specific case are derived to ensure the convergence. We demonstrate the remarkable robustness of the proposed approach against outliers by extensive simulations to the application of multi-lead electrocardiogram as well as synthetic data.

Transformed-Domain Robust Multiple-Exposure Blending With Huber Loss

Pixel-domain weighting methods for multiple-exposure blending can efficiently remove noise and under-/over-exposed pixels simultaneously in high dynamic range (HDR) image generation. Various types of noise such as non-Gaussian noise, e.g., Poisson, impulse noise, and pixel saturation, are often superimposed to multiple-exposure images taken with a high ISO setting in a low-light condition. Because almost all existing methods assume Gaussian noise, these methods cannot sufficiently reduce these types of noise. To achieve high-quality HDR image generation in such difficult conditions, we propose a novel multiple-exposure blending method in which image blending is performed in a wavelet domain so as to enhance the denoising performance. In addition, the Huber loss function is utilized as a fidelity measure in blending to make the method robust against outliers. We also introduce an efficient algorithm based on a primal-dual splitting method for solving our optimization problem. The experimental results demonstrate the advantages of the proposed method over several conventional methods.

Sparse Spatio-Temporal Imaging of Radio Transients

AbstractThe next-generation radio telescopes such as LOFAR and SKA will give access to high time-resolution and high instantaneous sensitivity that can be exploited to study slow and fast transients over the whole radio window. The search for radio transients in large datasets also represents a new signal-processing challenge requiring efficient and robust signal reconstruction algorithms. Using sparse representations and the general ‘compressed sensing’ framework, we developed a 2D–1D algorithm based on the primal-dual splitting method. We have performed our sparse 2D–1D reconstruction on three-dimensional data sets containing either simulated or real radio transients, at various levels of SNR and integration times. This report presents a summary of the current level of performance of our method.

A Preconditioning Technique for First-Order Primal-Dual Splitting Method in Convex Optimization

We introduce a preconditioning technique for the first-order primal-dual splitting method. The primal-dual splitting method offers a very general framework for solving a large class of optimization problems arising in image processing. The key idea of the preconditioning technique is that the constant iterative parameters are updated self-adaptively in the iteration process. We also give a simple and easy way to choose the diagonal preconditioners while the convergence of the iterative algorithm is maintained. The efficiency of the proposed method is demonstrated on an image denoising problem. Numerical results show that the preconditioned iterative algorithm performs better than the original one.

A parallel primal-dual splitting method for image restoration

We develop a parallel primal-dual splitting method to solve large-scale image restoration problems, which involve the sum of several linear-operator-coupled nonsmooth but proximable terms. With the proposed method, the objective function is decomposed into pieces that can be processed individually. No inverse operator is involved in our method and the highly parallel structure makes it preferable in distributed computation. The convergence is proven and the convergence rate is analyzed. Besides, its equivalence to the relaxed parallel linearized alternating direction method of multipliers (PLADMM) is addressed. Applications to image restoration problems with compound l1-regularizer and comparisons with state-of-the-art methods are detailed to show the superiority of the proposed method.

Convergence rates with inexact non-expansive operators

In this paper, we present a convergence rate analysis for the inexact Krasnosel'skii-Mann iteration built from nonexpansive operators. Our results include two main parts: we first establish global pointwise and ergodic iteration-complexity bounds, and then, under a metric subregularity assumption, we establish local linear convergence for the distance of the iterates to the set of fixed points. The obtained iteration-complexity result can be applied to analyze the convergence rate of various monotone operator splitting methods in the literature, including the Forward-Backward, the Generalized Forward-Backward, Douglas-Rachford, alternating direction method of multipliers (ADMM) and Primal-Dual splitting methods. For these methods, we also develop easily verifiable termination criteria for finding an approximate solution, which can be seen as a generalization of the termination criterion for the classical gradient descent method. We finally develop a parallel analysis for the non-stationary Krasnosel'skii-Mann iteration. The usefulness of our results is illustrated by applying them to a large class of structured monotone inclusion and convex optimization problems. Experiments on some large scale inverse problems in signal and image processing problems are shown.

Primal-dual splitting method for high-order model with application to image restoration

The total variation (TV) based iterative regularization method has been utilized to recover images degraded by blur and impulse noise. It is well-known that the TV regularization model preserves the edges well in the restored images while suffers from staircase effect. In this paper, we consider a high-order total variation minimization model which removes undesired artifacts for restoring blurry and noisy images. Then a primal-dual splitting algorithm is developed to solve the high-order minimization problem. The convergence of the proposed method is guaranteed. Numerical results illustrate that the proposed method is competitive with the state-of-the-art methods in terms of the peak signal-to-noise (PSNR) and the structural similarity index measurement (SSIM).

Convex risk minimization via proximal splitting methods

In this paper we investigate the applicability of a recently introduced primal-dual splitting method in the context of solving portfolio optimization problems which assume the minimization of risk measures associated to different convex utility functions. We show that, due to the splitting characteristic of the used primal-dual method, the main effort in implementing it constitutes in the calculation of the proximal points of the utility functions, which assume explicit expressions in a number of cases. When quantifying risk via the meanwhile classical conditional value-at-risk, an alternative approach relying on the use of its dual representation is presented as well. The theoretical results are finally illustrated via some numerical experiments on real and synthetic data sets.

Convergence Analysis for a Primal-Dual Monotone + Skew Splitting Algorithm with Applications to Total Variation Minimization

In this paper we investigate the convergence behavior of a primal-dual splitting method for solving monotone inclusions involving mixtures of composite, Lipschitzian and parallel sum type operators proposed by Combettes and Pesquet (in Set-Valued Var. Anal. 20(2):307---330, 2012). Firstly, in the particular case of convex minimization problems, we derive convergence rates for the partial primal-dual gap function associated to a primal-dual pair of optimization problems by making use of conjugate duality techniques. Secondly, we propose for the general monotone inclusion problem two new schemes which accelerate the sequences of primal and/or dual iterates, provided strong monotonicity assumptions for some of the involved operators are fulfilled. Finally, we apply the theoretical achievements in the context of different types of image restoration problems solved via total variation regularization.

Primal-Dual Decomposition by Operator Splitting and Applications to Image Deblurring

We present primal-dual decomposition algorithms for convex optimization problems with cost functions $f(x)+g(Ax)$, where $f$ and $g$ have inexpensive proximal operators and $A$ can be decomposed as a sum of two structured matrices. The methods are based on the Douglas--Rachford splitting algorithm applied to various splittings of the primal-dual optimality conditions. We discuss applications to image deblurring problems with nonquadratic data fidelity terms, different types of convex regularization, and simple convex constraints. In these applications, the primal-dual splitting approach allows us to handle general boundary conditions for the blurring operator. Numerical results indicate that the primal-dual splitting methods compare favorably with the alternating direction method of multipliers, the Douglas--Rachford algorithm applied to a reformulated primal problem, and the Chambolle--Pock primal-dual algorithm.