A forward–backward penalty scheme with inertial effects for monotone inclusions. Applications to convex bilevel programming

Radu Ioan Boţ,Dang-Khoa Nguyen

doi:10.1080/02331934.2018.1556662

Abstract

ABSTRACTWe investigate a forward–backward splitting algorithm of penalty type with inertial effects for finding the zeros of the sum of a maximally monotone operator and a cocoercive one and the convex normal cone to the set of zeroes of an another cocoercive operator. Weak ergodic convergence is obtained for the iterates, provided that a condition expressed via the Fitzpatrick function of the operator describing the underlying set of the normal cone is verified. Under strong monotonicity assumptions, strong convergence for the sequence of generated iterates is proved. As a particular instance we consider a convex bilevel minimization problem including the sum of a non-smooth and a smooth function in the upper level and another smooth function in the lower level. We show that in this context weak non-ergodic and strong convergence can be also achieved under inf-compactness assumptions for the involved functions.

Highlights

During the last couple years one can observe in the optimization community an increasing interest in numerical schemes for solving variational inequalities expressed as monotone inclusion problems of the form 0 ∈ Ax + NM (x), (1)
Where H is a real Hilbert space, A : H ⇒ H is a maximally monotone operator, M := arg min h is the set of global minima of the proper, convex and lower semicontinuous function h : R → R := R ∪ {±∞} and NM : H ⇒ H is the normal cone of the set M
The aim of this work is to endow the forward–backward penalty scheme for solving (4) from [7] with inertial effects, which means that the new iterate is defined in terms of the previous two iterates

Summary

Motivation and problems formulation

During the last couple years one can observe in the optimization community an increasing interest in numerical schemes for solving variational inequalities expressed as monotone inclusion problems of the form 0 ∈ Ax + NM (x) ,. One motivation for studying numerical algorithms for monotone inclusions of type (1) comes from the fact that, when A ≡ ∂f is the convex subdifferential of a proper, convex and lower semicontinuous function f : H → R , they furnish iterative methods for solving bilevel optimization problems of the form min f (x) : x ∈ arg min h. The aim of this work is to endow the forward–backward penalty scheme for solving (4) from [7] with inertial effects, which means that the new iterate is defined in terms of the previous two iterates Inertial algorithms have their roots in the time discretization of second-order differential systems [15]. We prove weak ergodic convergence of the sequence generated by the inertial forward–backward penalty algorithm to a solution of the monotone inclusion problem (4), under reasonable assumptions for the sequences of step sizes, penalty and inertial parameters. The weak non-ergodic theorem is an useful alternative to the one in [9], where a similar statement has been obtained for the inertial forward–backward penalty algorithm with constant inertial parameter under assumptions which are quite complicated and hard to verify (see [11,12])

Notations and preliminaries

Applications to convex bilevel programming