Modified hybrid decomposition of the augmented Lagrangian method with larger step size for three-block separable convex programming

Min Sun,Yiju Wang

doi:10.1186/s13660-018-1863-z

Abstract

The Jacobian decomposition and the Gauss–Seidel decomposition of augmented Lagrangian method (ALM) are two popular methods for separable convex programming. However, their convergence is not guaranteed for three-block separable convex programming. In this paper, we present a modified hybrid decomposition of ALM (MHD-ALM) for three-block separable convex programming, which first updates all variables by a hybrid decomposition of ALM, and then corrects the output by a correction step with constant step size alpha in(0,2-sqrt{2}) which is much less restricted than the step sizes in similar methods. Furthermore, we show that 2-sqrt{2} is the optimal upper bound of the constant step size α. The rationality of MHD-ALM is testified by theoretical analysis, including global convergence, ergodic convergence rate, nonergodic convergence rate, and refined ergodic convergence rate. MHD-ALM is applied to solve video background extraction problem, and numerical results indicate that it is numerically reliable and requires less computation.

Highlights

Many problems encountered in applied mathematics area can be formulated as separable convex programming, such as basis pursuit (BP) problem [1,2,3], video background extraction problem [4,5,6,7], image decomposition [8,9,10], and so on
To the best of our knowledge, He et al [12] first proposed a prediction-correction method with constant step size for solving (1), and they proved that the upper bound of the constant step size is 0.2679
In this paper, based on the methods in [12, 13, 25], we propose a modified hybrid decomposition of the augmented Lagrangian method with constant step size, whose upper bound is relaxed to 0.5858

Summary

Introduction

Many problems encountered in applied mathematics area can be formulated as separable convex programming, such as basis pursuit (BP) problem [1,2,3], video background extraction problem [4,5,6,7], image decomposition [8,9,10], and so on. Lemma 3.2 indicates that the matrices Q, M defined as in (17) and (19) satisfy Condition 2.1, and we get the following convergence results of MHD-ALM based on Theorems 3.3, 4.2, 4.5 in [28]. (1) Problem (1) with θ1 = 0, A1 = 0 reduces to two-block separable convex programming, which can be solved by MHD-ALM as follows:. (2) problem (1) with θ3 = 0, A3 = 0 reduces to two-block separable convex programming, which can be solved by MHD-ALM as follows:. (3) Extending MHD-ALM to solve four-block separable convex programming: min θi(xi) Aixi = b, xi ∈ Xi, i = 1, 2, 3, 4 , i=1 i=1 we can get the following iterative scheme:.

Application example

Conclusion

Methods