Convergence study on the proximal alternating direction method with larger step size

Abstract

The alternating direction method of multipliers (ADMM) is a popular method for solving separable convex programs with linear constraints, and its proximal version is an important variant. In the literature, Fortin and Glowinski proved that the step size for updating the Lagrange multiplier of the ADMM can be chosen in the open interval of zero to the golden ratio, and subsequently this result has been proved to be also valid for the proximal ADMM. In this paper, we demonstrate that the dual step size can be larger than the golden ratio when the proximal regularization is positive definite. Thus, the feasible interval of the dual step size can be further enlarged for the proximal ADMM. Moreover, we establish the exact relationship between the dual step size and the proximal parameter. We also prove global convergence and establish a worst case convergence rate in the ergodic sense for this proximal scheme with the enlarged step size. Finally, we present numerical results to demonstrate the practical performance of the method.