Block-Wise ADMM with a Relaxation Factor for Multiple-Block Convex Programming

Abstract

It has been shown that the alternating direction method of multipliers (ADMM) is not necessarily convergent when it is directly extended to a multiple-block linearly constrained convex minimization model with an objective function that is in the sum of more than two functions without coupled variables. Recently, we proposed the block-wise ADMM, which was obtained by regrouping the variables and functions of such a model as two blocks and then applying the original ADMM in block-wise. This note is a further study on this topic with the purpose of showing that a well-known relaxation factor proposed by Fortin and Glowinski for iteratively updating the Lagrangian multiplier of the original ADMM can also be used in the block-wise ADMM. We thus propose the block-wise ADMM with Fortin and Glowinski’s relaxation factor for the multiple-block convex minimization model. Like the block-wise ADMM, we also suggest further decomposing the resulting subproblems and regularizing them by proximal terms to ensure the convergence. For the block-wise ADMM with Fortin and Glowinski’s relaxation factor, its convergence and worst-case convergence rate measured by the iteration complexity in the ergodic sense are derived.