Parallel Splitting Method Research Articles

A proximal fully parallel splitting method with a relaxation factor for separable convex programming

In this paper, we propose a proximal fully parallel splitting method with a relaxation factor for solving separable convex minimization model with linear constraints, where the objective function is the sum of m individual functions without coupled variables. With a full Jacobian decomposition, we decompose the subproblem associated with the augmented Lagrangian method into m smaller subproblems and then add a quadratic proximal term to each decomposed subproblem, which makes the resulting ones easier to solve for many applications. In order to accelerate the numerical performance, we attach a positive relaxation factor to update the Lagrange multiplier, which also allows more flexibility in the design of algorithms. Moreover, we refine the step size of the underrelaxation step, which enlarges several existing ones in the literature. We prove that the proposed method is globally convergent, and show the worst-case O(1/k) convergence rate in a nonergodic sense. Finally, the efficiency and robustness of the proposed method are also demonstrated by solving the ℓ1 norm problem, the ℓ1-regularized least squares problem, the exchange problem and the total variation image restoration problem.

New Inexact Parallel Splitting Algorithm for a Class of Separable Convex Programming

In this paper, a new inexact parallel splitting algorithm based on the shrinkage algorithm is proposed. We bring in inexact terms, and new iteration points are obtained by the parallel splitting method. A new descent direction and a proper step length are derived. Under reasonable assumptions, the convergence of the algorithm is demonstrated. Some matrix correction problem experiments show that the algorithm is efficient and easy to implement.

Parallel Normal S-Iteration Methods with Applications to Optimization Problems

A large number of nonlinear and optimization problems can be reduced to altering point problems. This paper aims to introduce the parallel normal S-iteration technique and study its convergence rates for solving such problems in infinite-dimensional Hilbert spaces under practical assumptions. We place particular emphasis on the parallel splitting method for the sum of two maximal monotone operators and that can apply for solving a class of convex composite minimization problems. Moreover, we present applications of our iterative methods to some nonlinear problems, such as a system of variational inequalities and a system of inclusion problems. Finally, to demonstrate the applicability of the altering point technique, the performances of our proposed parallel normal S-iteration methods are presented through numerical experiments in signal recovery problems.

A customized low-rank prior model for structured cartoon–texture image decomposition

The mathematical characterization of the texture component plays an instrumental role in image decomposition. In this paper, we are concerned with a low-rank texture prior based cartoon–texture image decomposition model, which utilizes a total variation norm and a global nuclear norm to characterize the cartoon and texture components, respectively. It is promising that our decomposition model is not only extremely simple, but also works perfectly for globally well-patterned images in the sense that the model can recover cleaner texture (or details) than the other novel models. Moreover, such a model can be easily reformulated as a separable convex optimization problem, thereby enjoying a splitting nature so that we can employ a partially parallel splitting method (PPSM) to solve it efficiently. A series of numerical experiments on image restoration demonstrate that PPSM can recover slightly higher quality images than some existing algorithms in terms of taking less iterations or computing time in many cases.

Serial and Parallel Iterative Splitting Methods: Algorithms and Applications to Fractional Convection-Diffusion Equations

The benefits and properties of iterative splitting methods, which are based on serial versions, have been studied in recent years, this work, we extend the iterative splitting methods to novel classes of parallel versions to solve nonlinear fractional convection-diffusion equations. For such interesting partial differential examples with higher dimensional, fractional, and nonlinear terms, we could apply the parallel iterative splitting methods, which allow for accelerating the solver methods and reduce the computational time. Here, we could apply the benefits of the higher accuracy of the iterative splitting methods. We present a novel parallel iterative splitting method, which is based on the multi-splitting methods, The flexibilisation with multisplitting methods allows for decomposing large scale operator equations. In combination with iterative splitting methods, which use characteristics of waveform-relaxation (WR) methods, we could embed the relaxation behavior and deal better with the nonlinearities of the operators. We consider the convergence results of the parallel iterative splitting methods, reformulating the underlying methods with a summation of the individual convergence results of the WR methods. We discuss the numerical convergence of the serial and parallel iterative splitting methods with respect to the synchronous and asynchronous treatments. Furthermore, we present different numerical applications of fluid and phase field problems in order to validate the benefit of the parallel versions.

Computing the Resolvent of the Sum of Maximally Monotone Operators with the Averaged Alternating Modified Reflections Algorithm

The averaged alternating modified reflections algorithm is a projection method for finding the closest point in the intersection of closed convex sets to a given point in a Hilbert space. In this work, we generalize the scheme so that it can be used to compute the resolvent of the sum of two maximally monotone operators. This gives rise to a new splitting method, which is proved to be strongly convergent. A standard product space reformulation permits to apply the method for computing the resolvent of a finite sum of maximally monotone operators. Based on this, we propose two variants of such parallel splitting method.

On the convergence rate of the augmented Lagrangian-based parallel splitting method

The augmented Lagrangian method (ALM) is a well-regarded algorithm for solving convex optimization problems with linear constraints. Recently, in He et al. [On full Jacobian decomposition of the augmented Lagrangian method for separable convex programming, SIAM J. Optim. 25(4) (2015), pp. 2274–2312], it has been demonstrated that a straightforward Jacobian decomposition of ALM is not necessarily convergent when the objective function is the sum of functions without coupled variables. Then, Wang et al. [A note on augmented Lagrangian-based parallel splitting method, Optim. Lett. 9 (2015), pp. 1199–1212] proved the global convergence of the augmented Lagrangian-based parallel splitting method under the assumption that all objective functions are strongly convex. In this paper, we extend these results and derive the worst-case convergence rate of this method under both ergodic and non-ergodic conditions, where t represents the number of iterations. Furthermore, we show that the convergence rate can be improved from to , and finally, we also demonstrate that this method can achieve global linear convergence, when the involved functions satisfy some additional conditions.

A Partially Parallel Prediction-Correction Splitting Method for Convex Optimization Problems with Separable Structure

In this paper, we propose a partially parallel prediction-correction splitting method for solving block-separable linearly constrained convex optimization problems with three blocks. Unlike the extended alternating direction method of multipliers, the last two subproblems in the prediction step are solved parallelly, and a correction step is employed in the method to correct the dual variable and two blocks of the primal variables. The step size adapted in the correction step allows for major contribution from the latest solution point to the iteration point. Some numerical results are reported to show the effectiveness of the presented method.

A Proximal Fully Parallel Splitting Method for Stable Principal Component Pursuit

As a special three-block separable convex programming, the stable principal component pursuit (SPCP) arises in many different disciplines, such as statistical learning, signal processing, and web data ranking. In this paper, we propose a proximal fully parallel splitting method (PFPSM) for solving SPCP, in which the resulting subproblems all admit closed-form solutions and can be solved in distributed manners. Compared with other similar algorithms in the literature, PFPSM attaches a Glowinski relaxation factor η∈3/2,2/3 to the updating formula for its Lagrange multiplier, which can be used to accelerate the convergence of the generated sequence. Under mild conditions, the global convergence of PFPSM is proved. Preliminary computational results show that the proposed algorithm works very well in practice.

A proximal partially parallel splitting method for separable convex programs

In this paper, we propose a proximal partially parallel splitting method for solving convex minimization problems, where the objective function is separable into m individual operators without any coupled variables, and the structural constraint set comprises only linear functions. At each iteration of this algorithm, one selected subproblem is solved, and subsequently the remaining subproblems are solved in parallel, utilizing the new iterate information. Hence, the proposed method is a hybrid mechanism that combines the nice features of parallel decomposition methods and alternating direction methods, while simultaneously adopting the predictor–corrector strategy to ensure convergence of the algorithm. Our algorithmic framework is also amenable to admitting linearized versions of the subproblems, which frequently have closed-form solutions, thereby making the proposed method more implementable in practice. Furthermore, the worst-case convergence rate of the proposed method is obtained under both ergodic and nonergodic conditions. The efficiency of the proposed algorithm is also demonstrated by solving several instances of the robust PCA problem.

An augmented Lagrangian-based parallel splitting method for a one-leader-two-follower game

In this paper, we exploit a new parallel splitting method for the typical structured variational inequality problems which can be interpreted as a game with a leader and two followers. In the framework of this method, two followers decide their strategies simultaneously based on the instruction of the leader. Then, the leader improves his instruction by revising his own variable value according to the feedback information from the followers. The convergence of the method is established under some suitable conditions. Finally, we apply the proposed method to solve some application problems. Computational studies show that the method is reliable and efficient.

A partially parallel splitting method for multiple-block separable convex programming with applications to robust PCA

We consider a multiple-block separable convex programming problem, where the objective function is the sum of m individual convex functions without overlapping variables, and the constraints are li...

A note on augmented Lagrangian-based parallel splitting method

We consider the linearly constrained separable convex minimization problem, whose objective function consists of the sum of \(m\) individual convex functions in the absence of any coupling variables. While augmented Lagrangian-based decomposition methods have been well developed in the literature for solving such problems, a noteworthy requirement of these methods is that an additional correction step is a must to guarantee their convergence. This note shows that a straightforward Jacobian decomposition of the augmented Lagrangian method is globally convergent if the involved functions are further assumed to be strongly convex.

An augmented Lagrangian based parallel splitting method for separable convex minimization with applications to image processing

An augmented Lagrangian based parallel splitting method for separable convex minimization with applications to image processing

A proximal parallel splitting method for minimizing sum of convex functions with linear constraints

In this paper, we propose a proximal parallel decomposition algorithm for solving the optimization problems where the objective function is the sum of m separable functions (i.e., they have no crossed variables), and the constraint set is the intersection of Cartesian products of some simple sets and a linear manifold. The m subproblems are solved simultaneously per iterations, which are sum of the decomposed subproblems of the augmented Lagrange function and a quadratic term. Hence our algorithm is named as the ‘proximal parallel splitting method’. We prove the global convergence of the proposed algorithm under some mild conditions that the underlying functions are convex and the solution set is nonempty. To make the subproblems easier, some linearized versions of the proposed algorithm are also presented, together with their global convergence analysis. Finally, some preliminary numerical results are reported to support the efficiency of the new algorithms.

An inexact parallel splitting augmented Lagrangian method for monotone variational inequalities with separable structures

Splitting methods have been extensively studied in the context of convex programming and variational inequalities with separable structures. Recently, a parallel splitting method based on the augmented Lagrangian method (abbreviated as PSALM) was proposed in He (Comput. Optim. Appl. 42:195---212, 2009) for solving variational inequalities with separable structures. In this paper, we propose the inexact version of the PSALM approach, which solves the resulting subproblems of PSALM approximately by an inexact proximal point method. For the inexact PSALM, the resulting proximal subproblems have closed-form solutions when the proximal parameters and inexact terms are chosen appropriately. We show the efficiency of the inexact PSALM numerically by some preliminary numerical experiments.

A partial parallel splitting augmented Lagrangian method for solving constrained matrix optimization problems

Alternating directions methods (ADMs) are very effective for solving convex optimization problems with separable structure. However, when these methods are applied to solve convex optimization problems with three separable operators, their convergence results have not been established as yet. In this paper, we consider a class of constrained matrix optimization problems. The problem is first reformulated into a convex optimization problem with three separable operators, then it is solved by a proposed partial parallel splitting method. The proposed method combines the parallel splitting (augmented Lagrangian) method (PSALM) and the alternating directions method (ADM), and it is referred to as PADALM in short. The main difference between PADALM and PSALM is that in PADALM, two operators are handled first by a parallel method, then the third operator and the former two are dealt with by an alternating method. Finally, the convergence result for PADALM is established and numerical results are provided to show the efficacy of PADALM and its superiority over PSALM.

A Parallel Splitting Method for Coupled Monotone Inclusions

A parallel splitting method is proposed for solving systems of coupled monotone inclusions in Hilbert spaces, and its convergence is established under the assumption that solutions exist. Unlike ex...

A Parallel Splitting Method for Coupled Monotone Inclusions

A parallel splitting method is proposed for solving systems of coupled monotone inclusions in Hilbert spaces, and its convergence is established under the assumption that solutions exist. Unlike existing alternating algorithms, which are limited to two variables and linear coupling, our parallel method can handle an arbitrary number of variables as well as nonlinear coupling schemes. The breadth and flexibility of the proposed framework is illustrated through applications in the areas of evolution inclusions, variational problems, best approximation, and network flows.

A family of third-order parallel splitting methods for parabolic partial differential equations

A family of numerical methods, based upon a new rational approximation to the matrix exponential function, is developed for solving parabolic partial differential equations. These methods are L-acceptable, third-order accurate in space and time, and do not require the use of complex arithmetic. In these methods second-order spatial derivatives are approximated by third-order finite-difference approximations.- Parallel algorithms are developed and tested on the one-dimensional heat equation, with constant coefficients, subject to homogeneous boundary conditions and time-dependent boundary conditions. These methods are also extended to two- and three-dimensional heat equations, with constant coefficients, subject to homogeneous boundary conditions.