Proximal Subproblems Research Articles

An Inexact Proximal Path-Following Algorithm for Constrained Convex Minimization

Many scientific and engineering applications feature nonsmooth convex minimization problems over convex sets. In this paper, we address an important instance of this broad class where we assume that the nonsmooth objective is equipped with a tractable proximity operator and that the convex constraint set affords a self-concordant barrier. We provide a new joint treatment of proximal and self-concordant barrier concepts and illustrate that such problems can be efficiently solved, without the need of lifting the problem dimensions, as in disciplined convex optimization approach. We propose an inexact path-following algorithmic framework and theoretically characterize the worst-case analytical complexity of this framework when the proximal subproblems are solved inexactly. To show the merits of our framework, we apply its instances to both synthetic and real-world applications, where it shows advantages over standard interior point methods. As a byproduct, we describe how our framework can obtain points on t...

Iteration-Complexity of Block-Decomposition Algorithms and the Alternating Direction Method of Multipliers

In this paper, we consider the monotone inclusion problem consisting of the sum of a continuous monotone map and a point-to-set maximal monotone operator with a separable two-block structure and introduce a framework of block-decomposition prox-type algorithms for solving it which allows for each one of the single-block proximal subproblems to be solved in an approximate sense. Moreover, by showing that any method in this framework is also a special instance of the hybrid proximal extragradient (HPE) method introduced by Solodov and Svaiter, we derive corresponding convergence rate results. We also describe some instances of the framework based on specific and inexpensive schemes for solving the single-block proximal subproblems. Finally, we consider some applications of our methodology to establish for the first time (i) the iteration-complexity of an algorithm for finding a zero of the sum of two arbitrary maximal monotone operators and, as a consequence, the ergodic iteration-complexity of the Douglas-...

Iteration-Complexity of a Newton Proximal Extragradient Method for Monotone Variational Inequalities and Inclusion Problems

In a recent paper by Monteiro and Svaiter, a hybrid proximal extragradient (HPE) framework was used to study the iteration-complexity of a first-order (or, in the context of optimization, second-order) method for solving monotone nonlinear equations. The purpose of this paper is to extend this analysis to study a prox-type first-order method for monotone smooth variational inequalities and inclusion problems consisting of the sum of a smooth monotone map and a maximal monotone point-to-set operator. Each iteration of the method computes an approximate solution of a proximal subproblem obtained by linearizing the smooth part of the operator in the corresponding proximal equation for the original problem, which is then used to perform an extragradient step as prescribed by the HPE framework. Both pointwise and ergodic iteration-complexity results are derived for the aforementioned first-order method using corresponding results obtained here for a subfamily of the HPE framework.

An Accelerated Inexact Proximal Point Algorithm for Convex Minimization

The proximal point algorithm is classical and popular in the community of optimization. In practice, inexact proximal point algorithms which solve the involved proximal subproblems approximately subject to certain inexact criteria are truly implementable. In this paper, we first propose an inexact proximal point algorithm with a new inexact criterion for solving convex minimization, and show its O(1/k) iteration-complexity. Then we show that this inexact proximal point algorithm is eligible for being accelerated by some influential acceleration schemes proposed by Nesterov. Accordingly, an accelerated inexact proximal point algorithm with an iteration-complexity of O(1/k 2) is proposed.

An inexact parallel splitting augmented Lagrangian method for monotone variational inequalities with separable structures

Splitting methods have been extensively studied in the context of convex programming and variational inequalities with separable structures. Recently, a parallel splitting method based on the augmented Lagrangian method (abbreviated as PSALM) was proposed in He (Comput. Optim. Appl. 42:195---212, 2009) for solving variational inequalities with separable structures. In this paper, we propose the inexact version of the PSALM approach, which solves the resulting subproblems of PSALM approximately by an inexact proximal point method. For the inexact PSALM, the resulting proximal subproblems have closed-form solutions when the proximal parameters and inexact terms are chosen appropriately. We show the efficiency of the inexact PSALM numerically by some preliminary numerical experiments.

An Approximate Proximal Point Algorithm for Maximal Monotone Inclusion Problems

This paper presents and analyzes a strongly convergent approximate proximal point algorithm for finding zeros of maximal monotone operators in Hilbert spaces. The proposed method combines the proximal subproblem with a more general correction step which takes advantage of more information on the existing iterations. As applications, convex programming problems and generalized variational inequalities are considered. Some preliminary computational results are reported.

On the proximal point method for equilibrium problems in Hilbert spaces

We analyse a proximal point method for equilibrium problems in Hilbert spaces, improving upon previously known convergence results. We prove global weak convergence of the generated sequence to a solution of the problem, assuming existence of solutions and rather mild monotonicity properties of the bifunction which defines the equilibrium problem, and we establish existence of solutions of the proximal subproblems. We also present a new reformulation of equilibrium problems as variational inequalities ones.

Inexact Proximal Point Methods for Variational Inequality Problems

We present a new family of proximal point methods for solving monotone variational inequalities. Our algorithm has a relative error tolerance criterion in solving the proximal subproblems. Our convergence analysis covers a wide family of regularization functions, including double regularizations recently introduced by Silva, Eckstein, and Humes, Jr. [SIAM J. Optim., 12 (2001), pp. 238–261] and the Bregman distance induced by $h(x)=\sum_{i=1}^{n}x_{i}\log x_{i}$. We do not use in our analysis the assumption of paramonotonicity, which is standard in proving convergence of Bregman-based proximal methods.

A Proximal‐Projection Bundle Method for Lagrangian Relaxation, Including Semidefinite Programming

We give a proximal bundle method for minimizing a convex function $f$ over a convex set $C$. It requires evaluating $f$ and its subgradients with a fixed but possibly unknown accuracy $\epsilon>0$. Each iteration involves solving an unconstrained proximal subproblem and projecting a certain point onto $C$. The method asymptotically finds points that are $\epsilon$-optimal. In Lagrangian relaxation of convex programs, it allows for e-accurate solutions of Lagrangian subproblems and finds e-optimal primal solutions. For semidefinite programming problems, it extends the highly successful spectral bundle method to the case of inexact eigenvalue computations.

Proximal methods in reflexive Banach spaces without monotonicity

We introduce the concept of hypomonotone point-to-set operators in Banach spaces, with respect to a regularizing function. This notion coincides with the one given by Rockafellar and Wets in Hilbertian spaces, when the regularizing function is the square of the norm. We study the associated proximal mapping, which leads to a hybrid proximal–extragradient and proximal–projection methods for nonmonotone operators in reflexive Banach spaces. These methods allow for inexact solution of the proximal subproblems with relative error criteria. We then consider the notion of local hypomonotonicity and propose localized versions of the algorithms, which are locally convergent.

A strongly convergent hybrid proximal method in Banach spaces

This paper is devoted to the study of strong convergence in inexact proximal like methods for finding zeroes of maximal monotone operators in Banach spaces. Convergence properties of proximal point methods in Banach spaces can be summarized as follows: if the operator have zeroes then the sequence of iterates is bounded and all its weak accumulation points are solutions. Whether or not the whole sequence converges weakly to a solution and which is the relation of the weak limit with the initial iterate are key questions. We present a hybrid proximal Bregman projection method, allowing for inexact solutions of the proximal subproblems, that guarantees strong convergence of the sequence to the closest solution, in the sense of the Bregman distance, to the initial iterate.

Inexact Variants of the Proximal Point Algorithm without Monotonicity

This paper studies convergence properties of inexact variants of the proximal point algorithm when applied to a certain class of nonmonotone mappings. The presented algorithms allow for constant relative errors, in the line of the recently proposed hybrid proximal-extragradient algorithm. The main convergence result extends a recent work of the second author, where exact solutions for the proximal subproblems were required. We also show that the linear convergence property is preserved in the case when the inverse of the operator is locally Lipschitz continuous near the origin. As an application, we give a convergence analysis for an inexact version of the proximal method of multipliers for a rather general family of problems which includes variational inequalities and constrained optimization problems.

A New Hybrid Generalized Proximal Point Algorithm for Variational Inequality Problems

In this paper, we propose a modified Bregman-function-based proximal point algorithm for solving variational inequality problems. The algorithm adopts a similar constructive approximate criterion as the one developed by Solodov and Svaiter (Set Valued Analysis 7 (1999) 323) for solving the classical proximal subproblems. Under some suitable conditions, we can get an approximate solution satisfying the accuracy criterion via a single Newton-type step. We obtain the Fejer monotonicity to solutions of VIP for paramonotone operators. Some preliminary computational results are also reported to illustrate the method.

A new proximal-based globalization strategy for the Josephy‐Newton method for variational inequalities

We propose a new approach to globalizing the Josephy-Newton algorithm for solving the monotone variational inequality problem. Known globalization strategies rely either on minimization of a suitable merit function, or on a projection-type approach. The technique proposed here is based on a linesearch in the regularized Josephy-Newton direction which finds a trial point and a proximal point subproblem (i.e., subproblem with suitable parameters), for which this trial point is an acceptable approximate solution. We emphasize that this requires only checking a certain approximation criterion, and in particular, does not entail actually solving any nonlinear proximal point subproblems. The method converges globally under very mild assumptions. Furthermore, an easy modification of the method secures the local superlinear rate of convergence under standard conditions.

A UNIFIED FRAMEWORK FOR SOME INEXACT PROXIMAL POINT ALGORITHMS*

We present a unified framework for the design and convergence analysis of a class of algorithms based on approximate solution of proximal point subproblems. Our development further enhances the constructive approximation approach of the recently proposed hybrid projection–proximal and extragradient–proximal methods. Specifically, we introduce an even more flexible error tolerance criterion, as well as provide a unified view of these two algorithms. Our general method possesses global convergence and local (super)linear rate of convergence under standard assumptions, while using a constructive approximation criterion suitable for a number of specific implementations. For example, we show that close to a regular solution of a monotone system of semismooth equations, two Newton iterations are sufficient to solve the proximal subproblem within the required error tolerance. Such systems of equations arise naturally when reformulating the nonlinear complementarity problem. *Research of the first author is suppo...

Error bounds for proximal point subproblems and associated inexact proximal point algorithms

We study various error measures for approximate solution of proximal point regularizations of the variational inequality problem, and of the closely related problem of finding a zero of a maximal monotone operator. A new merit function is proposed for proximal point subproblems associated with the latter. This merit function is based on Burachik-Iusem-Svaiter’s concept of ε-enlargement of a maximal monotone operator. For variational inequalities, we establish a precise relationship between the regularized gap function, which is a natural error measure in this context, and our new merit function. Some error bounds are derived using both merit functions for the corresponding formulations of the proximal subproblem. We further use the regularized gap function to devise a new inexact proximal point algorithm for solving monotone variational inequalities. This inexact proximal point method preserves all the desirable global and local convergence properties of the classical exact/inexact method, while providing a constructive error tolerance criterion, suitable for further practical applications. The use of other tolerance rules is also discussed.