Primal-dual Splitting Algorithm Research Articles

Improved quantification in CEST-MRI by joint spatial total generalized variation.

In this work, the use of joint Total Generalized Variation (TGV) regularization to improve Multipool-Lorentzian fitting of chemical exchange saturation transfer (CEST) Spectra in terms of stability and parameter signal-to-noise ratio (SNR) was investigated. The joint TGV term was integrated into the nonlinear parameter fitting problem. To increase convergence and weight the gradients, preconditioning using a voxel-wise singular value decomposition was applied to the problem, which was then solved using the iteratively regularized Gauss-Newton method combined with a Primal-Dual splitting algorithm. The TGV method was evaluated on simulated numerical phantoms, 3T phantom data and 7T in vivo data with respect to systematic errors and robustness. Three reference methods were also implemented: The standard nonlinear fitting, a method using a nonlocal-means filter for denoising and the pyramid scheme, which uses downsampled images to acquire accurate start values. The proposed regularized fitting method showed significantly improved robustness (compared to the reference methods). In testing, over a range of SNR values the TGV fit outperformed the other methods and showed accurate results even for large amounts of added noise. Parameter values found were closer or comparable to the ground truth. For in vivo datasets, the added regularization increased the parameter map SNR and prevented instabilities. The proposed fitting method using TGV regularization leads to improved results over a range of different data-sets and noise levels. Furthermore, it can be applied to all Z-spectrum data, with different amounts of pools, where the improved SNR and stability can increase diagnostic confidence.

Weak and Strong Convergence of Split Douglas-Rachford Algorithms for Monotone Inclusions

We are concerned in this paper with the convergence analysis of the primal-dual splitting (PDS) and the split Douglas-Rachford (SDR) algorithms for monotone inclusions by using an operator-oriented approach. We shall show that both PDS and SDR algorithms can be driven by a (firmly) nonexpansive mapping in a product Hilbert space. We are then able to apply the Krasnoselskii-Mann and Halpern fixed point algorithms to PDS and SDR to get weakly and strongly convergent algorithms for finding solutions of the primal and dual monotone inclusions. Moreover, an additional projection technique is used to derive strong convergence of a modified SDR algorithm.

Alternated inertial forward-backward-forward splitting algorithm

In this paper, we propose a forward-backward-forward splitting algorithm with alternated inertial step for finding a zero of the sum of two monotone operators in a real Hilbert space. Then, by making use of primal-dual techniques we derive the alternated inertial primal-dual splitting algorithm of forward-backward-forward type for solving structured monotone inclusion problems involving parallel sums and compositions of maximally monotone operators with linear continuous ones and convex minimization problems. Finally, we demonstrate numerically that our proposed algorithm performs better than the existing algorithms.

FedEHR: A Federated Learning Approach towards the Prediction of Heart Diseases in IoT-Based Electronic Health Records.

In contemporary healthcare, the prediction and identification of cardiac diseases is crucial. By leveraging the capabilities of Internet of Things (IoT)-enabled devices and Electronic Health Records (EHRs), the healthcare sector can largely benefit to improve patient outcomes by increasing the accuracy of disease prediction. However, protecting data privacy is essential to promote participation and adhere to rules. The suggested methodology combines EHRs with IoT-generated health data to predict heart disease. For its capacity to manage high-dimensional data and choose pertinent features, a soft-margin L1-regularised Support Vector Machine (sSVM) classifier is used. The large-scale sSVM problem is successfully solved using the cluster primal-dual splitting algorithm, which improves computational complexity and scalability. The integration of federated learning provides a cooperative predictive analytics methodology that upholds data privacy. The use of a federated learning framework in this study, with a focus on peer-to-peer applications, is crucial for enabling collaborative predictive modeling while protecting the confidentiality of each participant's private medical information.

A primal-dual splitting algorithm for composite monotone inclusions with minimal lifting

In this work, we study resolvent splitting algorithms for solving composite monotone inclusion problems. The objective of these general problems is finding a zero in the sum of maximally monotone operators composed with linear operators. Our main contribution is establishing the first primal-dual splitting algorithm for composite monotone inclusions with minimal lifting. Specifically, the proposed scheme reduces the dimension of the product space where the underlying fixed point operator is defined, in comparison to other algorithms, without requiring additional evaluations of the resolvent operators. We prove the convergence of this new algorithm and analyze its performance in a problem arising in image deblurring and denoising. This work also contributes to the theory of resolvent splitting algorithms by extending the minimal lifting theorem recently proved by Malitsky and Tam to schemes with resolvent parameters.

Phase retrieval based on a total-variation-regularized Poisson model for X-ray ptychographic imaging of low-contrast objects

Hard X-ray ptychography has become an indispensable tool for observing the microscopic structure of a thick specimen. It measures diffraction patterns by scanning an X-ray beam and visualizes the complex-valued refractive index of the specimen by a computational reconstruction called phase retrieval. The quality of imaging is dependent on the used phase-retrieval algorithm, especially when the intensity of the diffraction patterns in the high-spatial-frequency range is low and/or when the spatial overlap of the illumination area is small. In this paper, a phase-retrieval algorithm, AMPAM, based on the Poisson model and total variation (TV) is proposed. It applies alternating minimization using primal-dual splitting and gradient-descent algorithms to compute the result without matrix inversion. The imaging capability of the proposed algorithm from low-dose and/or sparsely scanned data was investigated by numerical simulations. The proposed algorithm was compared with ADPr, which is the state-of-the-art algorithm based on the TV-regularized Poisson model. The results indicated that AMPAM can provide good-quality images with a computational cost 7–11 times less than ADPr. In addition, ink toner and macroporous silica particles were imaged at SPring-8 BL24XU to confirm the applicability of the algorithm to actual measurements.

Partial Smoothness and Constant Rank

The idea of partial smoothness in optimization blends certain smooth and nonsmooth properties of feasible regions and objective functions. As a consequence, the standard first-order conditions guarantee that diverse iterative algorithms (and post-optimality analyses) identify active structure or constraints. However, by instead focusing directly on the first-order conditions, the formal concept of partial smoothness simplifies dramatically: in basic differential geometric language, it is just a constant-rank condition. In this view, partial smoothness extends to more general mappings, such as saddlepoint operators underlying primal-dual splitting algorithms.

A General Destriping Framework for Remote Sensing Images Using Flatness Constraint

Removing stripe noise, i.e., destriping, from remote sensing images is an essential task in terms of visual quality and subsequent processing. Most existing destriping methods are designed by combining a particular image regularization with a stripe noise characterization that cooperates with the regularization, which precludes us to examine and activate different regularizations to adapt to various target images. To resolve this, two requirements need to be considered: a general framework that can handle a variety of image regularizations in destriping, and a strong stripe noise characterization that can consistently capture the nature of stripe noise, regardless of the choice of image regularization. To this end, this article proposes a general destriping framework using a newly introduced stripe noise characterization, named <i>flatness constraint (FC)</i>, where we can handle various regularization functions in a unified manner. Specifically, we formulate the destriping problem as a nonsmooth convex optimization problem involving a general form of image regularization and the FC. The constraint mathematically models that the intensity of each stripe is constant along one direction, resulting in a strong characterization of stripe noise. For solving the optimization problem, we also develop an efficient algorithm based on a diagonally preconditioned primal-dual splitting algorithm (DP-PDS), which can automatically adjust the step sizes. The effectiveness of our framework is demonstrated through destriping experiments, where we comprehensively compare combinations of a variety of image regularizations and stripe noise characterizations using hyperspectral images (HSIs) and infrared (IR) videos.

Primal-Dual Splitting Algorithms for Solving Structured Monotone Inclusion with Applications

This work proposes two different primal-dual splitting algorithms for solving structured monotone inclusion containing a cocoercive operator and the parallel-sum of maximally monotone operators. In particular, the parallel-sum is symmetry. The proposed primal-dual splitting algorithms are derived from two approaches: One is the preconditioned forward–backward splitting algorithm, and the other is the forward–backward–half-forward splitting algorithm. Both algorithms have a simple calculation framework. In particular, the single-valued operators are processed via explicit steps, while the set-valued operators are computed by their resolvents. Numerical experiments on constrained image denoising problems are presented to show the performance of the proposed algorithms.

Functional penalised basis pursuit on spheres

In this paper, we propose a unified theoretical and practical spherical approximation framework for functional inverse problems on the hypersphere. More specifically, we consider recovering spherical fields directly in the continuous domain using functional penalised basis pursuit problems with gTV regularisation terms. Our framework is compatible with various measurement types as well as non-differentiable convex cost functionals. Via a novel representer theorem, we characterise their solution sets in terms of spherical splines with sparse innovations. We use this result to derive an approximate canonical spline-based discretisation scheme, with vanishing approximation error. To solve the resulting finite-dimensional optimisation problem, we propose an efficient and provably convergent primal-dual splitting algorithm. We illustrate the versatility of our framework on real-life examples from the field of environmental sciences.

Determined BSS Based on Time-Frequency Masking and Its Application to Harmonic Vector Analysis

This paper proposes harmonic vector analysis (HVA) based on a general algorithmic framework of audio blind source separation (BSS) that is also presented in this paper. BSS for a convolutive audio mixture is usually performed by multichannel linear filtering when the numbers of microphones and sources are equal (determined situation). This paper addresses such determined BSS based on batch processing. To estimate the demixing filters, effective modeling of the source signals is important. One successful example is independent vector analysis (IVA) that models the signals via co-occurrence among the frequency components in each source. To give more freedom to the source modeling, a general framework of determined BSS is presented in this paper. It is based on the plug-and-play scheme using a primal-dual splitting algorithm and enables us to model the source signals implicitly through a time-frequency mask. By using the proposed framework, determined BSS algorithms can be developed by designing masks that enhance the source signals. As an example of its application, we propose HVA by defining a time-frequency mask that enhances the harmonic structure of audio signals via sparsity of cepstrum. The experiments showed that HVA outperforms IVA and independent low-rank matrix analysis (ILRMA) for both speech and music signals. A MATLAB code is provided along with the paper for a reference.

Choice of the parameters in a primal-dual algorithm for Bregman iterated variational regularization

Focus of this work is solving a non-smooth constraint minimization problem by a primal-dual splitting algorithm involving proximity operators. The problem is penalized by the Bregman divergence associated with the non-smooth total variation (TV) functional. We analyze two aspects: Firstly, the convergence of the regularized solution of the minimization problem to the minimum norm solution. Second, the convergence of the iteratively regularized minimizer to the minimum norm solution by a primal-dual algorithm. For both aspects, we use the assumption of a variational source condition (VSC). This work emphasizes the impact of the choice of the parameters in stabilization of a primal-dual algorithm. Rates of convergence are obtained in terms of some concave, positive definite index function. The algorithm is applied to a simple two-dimensional image processing problem. Sufficient error analysis profiles are provided based on the size of the forward operator and the noise level in the measurement.

Adaptive iterative reconstruction based on relative total variation for low-intensity computed tomography

Low-intensity projections (high-level noise) may degrade computed tomography (CT) image quality in some applications. CT image reconstruction from projections with low intensity has therefore attracted attention from researchers, who aim to suppress noise and preserve structures. Selecting a regularization parameter is a significant step for iterative reconstruction. To address this problem, we propose a reconstruction model that employs relative total variation (RTV)-regularized projections onto convex sets (POCS-RTV). This reconstruction model can effectively suppress noise and preserve image structures. In POCS-RTV, the L-curve method can adaptively determine the regularization parameter, producing the adaptive POCS-RTV (POCS-ARTV) reconstruction model. A nested iterative algorithm is applied to solving the POCS-RTV and POCS-ARTV models. Compared to reconstruction models based on ℓ1 and ℓ0 pseudo norm, POCS-ARTV can acquire adaptive structure preservation and noise reduction, and it performs the best in terms of image quality assessment (IQA) indexes and graphic comparisons. Compared to the reconstruction model based on parallel primal-dual splitting algorithm for total generalized variation and shearlet (PPDS-TGVS) regularization, POCS-ARTV reconstructs images that are visually closer to the reference image. Experiments on digital phantoms and real CT projections verified superiorities of POCS-ARTV, and the ability of the L-curve method to determine an acceptable regularization parameter.

A general double-proximal gradient algorithm for d.c. programming

The possibilities of exploiting the special structure of d.c. programs, which consist of optimising the difference of convex functions, are currently more or less limited to variants of the DCA proposed by Pham Dinh Tao and Le Thi Hoai An in 1997. These assume that either the convex or the concave part, or both, are evaluated by one of their subgradients. In this paper we propose an algorithm which allows the evaluation of both the concave and the convex part by their proximal points. Additionally, we allow a smooth part, which is evaluated via its gradient. In the spirit of primal-dual splitting algorithms, the concave part might be the composition of a concave function with a linear operator, which are, however, evaluated separately. For this algorithm we show that every cluster point is a solution of the optimisation problem. Furthermore, we show the connection to the Toland dual problem and prove a descent property for the objective function values of a primal-dual formulation of the problem. Convergence of the iterates is shown if this objective function satisfies the Kurdyka–Łojasiewicz property. In the last part, we apply the algorithm to an image processing model.

Block-simultaneous direction method of multipliers: a proximal primal-dual splitting algorithm for nonconvex problems with multiple constraints

We introduce a generalization of the linearized Alternating Direction Method of Multipliers to optimize a real-valued function $f$ of multiple arguments with potentially multiple constraints $g_\circ$ on each of them. The function $f$ may be nonconvex as long as it is convex in every argument, while the constraints $g_\circ$ need to be convex but not smooth. If $f$ is smooth, the proposed Block-Simultaneous Direction Method of Multipliers (bSDMM) can be interpreted as a proximal analog to inexact coordinate descent methods under constraints. Unlike alternative approaches for joint solvers of multiple-constraint problems, we do not require linear operators $L$ of a constraint function $g(L\ \cdot)$ to be invertible or linked between each other. bSDMM is well-suited for a range of optimization problems, in particular for data analysis, where $f$ is the likelihood function of a model and $L$ could be a transformation matrix describing e.g. finite differences or basis transforms. We apply bSDMM to the Non-negative Matrix Factorization task of a hyperspectral unmixing problem and demonstrate convergence and effectiveness of multiple constraints on both matrix factors. The algorithms are implemented in python and released as an open-source package.

Space variant deconvolution of galaxy survey images

Removing the aberrations introduced by the Point Spread Function (PSF) is a fundamental aspect of astronomical image processing. The presence of noise in observed images makes deconvolution a nontrivial task that necessitates the use of regularisation. This task is particularly difficult when the PSF varies spatially as is the case for the Euclid telescope. New surveys will provide images containing thousand of galaxies and the deconvolution regularisation problem can be considered from a completely new perspective. In fact, one can assume that galaxies belong to a low-rank dimensional space. This work introduces the use of the low-rank matrix approximation as a regularisation prior for galaxy image deconvolution and compares its performance with a standard sparse regularisation technique. This new approach leads to a natural way to handle a space variant PSF. Deconvolution is performed using a Python code that implements a primal-dual splitting algorithm. The data set considered is a sample of 10 000 space-based galaxy images convolved with a known spatially varying Euclid-like PSF and including various levels of Gaussian additive noise. Performance is assessed by examining the deconvolved galaxy image pixels and shapes. The results demonstrate that for small samples of galaxies sparsity performs better in terms of pixel and shape recovery, while for larger samples of galaxies it is possible to obtain more accurate estimates of the galaxy shapes using the low-rank approximation.

A First-Order Stochastic Primal-Dual Algorithm with Correction Step

ABSTRACTIn this article, we investigate the convergence properties of a stochastic primal-dual splitting algorithm for solving structured monotone inclusions involving the sum of a cocoercive operator and a composite monotone operator. The proposed method is the stochastic extension to monotone inclusions of a proximal method studied in the literature for saddle point problems. It consists in a forward step determined by the stochastic evaluation of the cocoercive operator, a backward step in the dual variables involving the resolvent of the monotone operator, and an additional forward step using the stochastic evaluation of the cocoercive operator introduced in the first step. We prove weak almost sure convergence of the iterates by showing that the primal-dual sequence generated by the method is stochastic quasi-Fejér-monotone with respect to the set of zeros of the considered primal and dual inclusions. Additional results on ergodic convergence in expectation are considered for the special case of saddle point models.

Primal-dual splitting method for high-order model with application to image restoration

The total variation (TV) based iterative regularization method has been utilized to recover images degraded by blur and impulse noise. It is well-known that the TV regularization model preserves the edges well in the restored images while suffers from staircase effect. In this paper, we consider a high-order total variation minimization model which removes undesired artifacts for restoring blurry and noisy images. Then a primal-dual splitting algorithm is developed to solve the high-order minimization problem. The convergence of the proposed method is guaranteed. Numerical results illustrate that the proposed method is competitive with the state-of-the-art methods in terms of the peak signal-to-noise (PSNR) and the structural similarity index measurement (SSIM).

An inertial forward-backward-forward primal-dual splitting algorithm for solving monotone inclusion problems

We introduce and investigate the convergence properties of an inertial forward-backward-forward splitting algorithm for approaching the set of zeros of the sum of a maximally monotone operator and a single-valued monotone and Lipschitzian operator. By making use of the product space approach, we expand it to the solving of inclusion problems involving mixtures of linearly composed and parallel-sum type monotone operators. We obtain in this way an inertial forward-backward-forward primal-dual splitting algorithm having as main characteristic the fact that in the iterative scheme all operators are accessed separately either via forward or via backward evaluations. We present also the variational case when one is interested in the solving of a primal-dual pair of convex optimization problems with complexly structured objectives, which we also illustrate by numerical experiments in image processing.

On the convergence rate of a forward-backward type primal-dual splitting algorithm for convex optimization problems

In this paper, we analyse the convergence rate of the sequence of objective function values of a primal-dual proximal-point algorithm recently introduced in the literature for solving a primal convex optimization problem having as objective the sum of linearly composed infimal convolutions, nonsmooth and smooth convex functions and its Fenchel-type dual one. The theoretical part is illustrated by numerical experiments in image processing.