Inexact Proximal Point Method Research Articles

An inexact proximal point method with quasi-distance for quasi-convex multiobjective optimization

An inexact proximal point method with quasi-distance for quasi-convex multiobjective optimization

A new inexact gradient descent method with applications to nonsmooth convex optimization

The paper proposes and develops a novel inexact gradient method (IGD) for minimizing C 1 -smooth functions with Lipschitzian gradients, i.e. for problems of C 1 , 1 optimization. We show that the sequence of gradients generated by IGD converges to zero. The convergence of iterates to stationary points is guaranteed under the Kurdyka-Łojasiewicz (KL) property of the objective function with convergence rates depending on the KL exponent. The newly developed IGD is applied to designing two novel gradient-based methods of nonsmooth convex optimization such as the inexact proximal point methods (GIPPM) and the inexact augmented Lagrangian method (GIALM) for convex programs with linear equality constraints. These two methods inherit global convergence properties from IGD and are confirmed by numerical experiments to have practical advantages over some well-known algorithms of nonsmooth convex optimization.

An inexact algorithm for stochastic variational inequalities

We present a new Progressive Hedging Algorithm to solve Stochastic Variational Inequalities in the formulation introduced by Rockafellar and Wets in 2017, allowing the generated subproblems to be approximately solved with an implementable tolerance condition. Our scheme is based on Hybrid Inexact Proximal Point methods and generalizes the exact algorithm developed by Rockafellar and Sun in 2019, providing stronger convergence results. We also show some numerical experiments in two-stage Nash games.

Proximal point methods with possible unbounded errors for monotone operators in Hadamard spaces

In this paper, we investigate and analyse the strong convergence of the sequence generated by an inexact proximal point method with possible unbounded errors for finding zeros of monotone operators in Hadamard spaces. We show that the boundedness of the generated sequence is equivalent to the zero set of the operator to be nonempty. In this case, we prove the strong convergence of the generated sequence to a zero of the operator. We also provide some applications of our main results and give a numerical example to show the performance of the proposed algorithm.

Stochastic first-order methods for convex and nonconvex functional constrained optimization

Functional constrained optimization is becoming more and more important in machine learning and operations research. Such problems have potential applications in risk-averse machine learning, semisupervised learning and robust optimization among others. In this paper, we first present a novel Constraint Extrapolation (ConEx) method for solving convex functional constrained problems, which utilizes linear approximations of the constraint functions to define the extrapolation (or acceleration) step. We show that this method is a unified algorithm that achieves the best-known rate of convergence for solving different functional constrained convex composite problems, including convex or strongly convex, and smooth or nonsmooth problems with stochastic objective and/or stochastic constraints. Many of these rates of convergence were in fact obtained for the first time in the literature. In addition, ConEx is a single-loop algorithm that does not involve any penalty subproblems. Contrary to existing primal-dual methods, it does not require the projection of Lagrangian multipliers into a (possibly unknown) bounded set. Second, for nonconvex functional constrained problems, we introduce a new proximal point method which transforms the initial nonconvex problem into a sequence of convex problems by adding quadratic terms to both the objective and constraints. Under certain MFCQ-type assumption, we establish the convergence and rate of convergence of this method to KKT points when the convex subproblems are solved exactly or inexactly. For large-scale and stochastic problems, we present a more practical proximal point method in which the approximate solutions of the subproblems are computed by the aforementioned ConEx method. Under a strong feasibility assumption, we establish the total iteration complexity of ConEx required by this inexact proximal point method for a variety of problem settings, including nonconvex smooth or nonsmooth problems with stochastic objective and/or stochastic constraints. To the best of our knowledge, most of these convergence and complexity results of the proximal point method for nonconvex problems also seem to be new in the literature.

An Accelerated Inexact Proximal Point Method for Solving Nonconvex-Concave Min-Max Problems

Related DatabasesWeb of Science You must be logged in with an active subscription to view this.Article DataHistorySubmitted: 16 January 2020Accepted: 09 July 2021Published online: 25 October 2021Keywordsquadratic penalty method, composite nonconvex problem, iteration complexity, inexact proximal point method, first-order accelerated gradient method, minimax problemAMS Subject Headings47J22, 90C26, 90C30, 90C47, 90C60, 65K10Publication DataISSN (print): 1052-6234ISSN (online): 1095-7189Publisher: Society for Industrial and Applied MathematicsCODEN: sjope8

Inexact Proximal Point Methods for Multiobjective Quasiconvex Minimization on Hadamard Manifolds

In this paper, we present two inexact scalarization proximal point methods to solve quasiconvex multiobjective minimization problems on Hadamard manifolds. Under standard assumptions on the problem, we prove that the two sequences generated by the algorithms converge to a Pareto critical point of the problem and, for the convex case, the sequences converge to a weak Pareto solution. Finally, we explore an application of the method to demand theory in economy, which can be dealt with using the proposed algorithm.

On the complexity of a hybrid proximal extragradient projective method for solving monotone inclusion problems

In a series of papers (Solodov and Svaiter in J Convex Anal 6(1):59–70, 1999; Set-Valued Anal 7(4):323–345, 1999; Numer Funct Anal Optim 22(7–8):1013–1035, 2001) Solodov and Svaiter introduced new inexact variants of the proximal point method with relative error tolerances. Point-wise and ergodic iteration-complexity bounds for one of these methods, namely the hybrid proximal extragradient method (1999) were established by Monteiro and Svaiter (SIAM J Optim 20(6):2755–2787, 2010). Here, we extend these results to a more general framework, by establishing point-wise and ergodic iteration-complexity bounds for the inexact proximal point method studied by Solodov and Svaiter (2001). Using this framework we derive global convergence results and iteration-complexity bounds for a family of projective splitting methods for solving monotone inclusion problems, which generalize the projective splitting methods introduced and studied by Eckstein and Svaiter (SIAM J Control Optim 48(2):787–811, 2009).

An efficient adaptive accelerated inexact proximal point method for solving linearly constrained nonconvex composite problems

This paper proposes an efficient adaptive variant of a quadratic penalty accelerated inexact proximal point (QP-AIPP) method proposed earlier by the authors. Both the QP-AIPP method and its variant solve linearly set constrained nonconvex composite optimization problems using a quadratic penalty approach where the generated penalized subproblems are solved by a variant of the underlying AIPP method. The variant, in turn, solves a given penalized subproblem by generating a sequence of proximal subproblems which are then solved by an accelerated composite gradient algorithm. The main difference between AIPP and its variant is that the proximal subproblems in the former are always convex while the ones in the latter are not necessarily convex due to the fact that their prox parameters are chosen as aggressively as possible so as to improve efficiency. The possibly nonconvex proximal subproblems generated by the AIPP variant are also tentatively solved by a novel adaptive accelerated composite gradient algorithm based on the validity of some key convergence inequalities. As a result, the variant generates a sequence of proximal subproblems where the stepsizes are adaptively changed according to the responses obtained from the calls to the accelerated composite gradient algorithm. Finally, numerical results are given to demonstrate the efficiency of the proposed AIPP and QP-AIPP variants.

Complexity of a Quadratic Penalty Accelerated Inexact Proximal Point Method for Solving Linearly Constrained Nonconvex Composite Programs

This paper analyzes the iteration complexity of a quadratic penalty accelerated inexact proximal point method for solving linearly constrained nonconvex composite programs. More specifically, the objective function is of the form $f+h$, where $f$ is a differentiable function whose gradient is Lipschitz continuous and $h$ is a closed convex function with possibly unbounded domain. The method, basically, consists of applying an accelerated inexact proximal point method for approximately solving a sequence of quadratic penalized subproblems associated with the linearly constrained problem. Each subproblem of the proximal point method is in turn approximately solved by an accelerated composite gradient (ACG) method. It is shown that the proposed scheme generates a $\rho$-approximate stationary point in at most $\mathcal{O}(\rho^{-3})$ ACG iterations. Finally, numerical results showing the efficiency of the proposed method are also given.

A generalized proximal linearized algorithm for DC functions with application to the optimal size of the firm problem

The purpose of this paper is twofold. First, we examine convergence properties of an inexact proximal point method with a quasi distance as a regularization term in order to find a critical point (in the sense of Toland) of a DC function (difference of two convex functions). Global convergence of the sequence and some convergence rates are obtained with additional assumptions. Second, as an application and its inspiration, we study in a dynamic setting, the very important and difficult problem of the limit of the firm and the time it takes to reach it (maturation time), when increasing returns matter in the short run. Both the formalization of the critical size of the firm in term of a recent variational rationality approach of human dynamics and the speed of convergence results are new in Behavioral Sciences.

A proximal-Newton method for unconstrained convex optimization in Hilbert spaces

We propose and study the iteration-complexity of a proximal-Newton method for finding approximate solutions of the problem of minimizing a twice continuously differentiable convex function on a (possibly infinite dimensional) Hilbert space. We prove global convergence rates for obtaining approximate solutions in terms of function/gradient values. Our main results follow from an iteration-complexity study of an (large-step) inexact proximal point method for solving convex minimization problems.

An Inexact Proximal Method with Proximal Distances for Quasimonotone Equilibrium Problems

In this paper, we propose an inexact proximal point method to solve equilibrium problems using proximal distances and the diagonal subdifferential. Under some natural assumptions on the problem and the quasimonotonicity condition on the bifunction, we prove that the sequence generated by the method converges to a solution point of the problem.

Inexact Proximal Point Methods for Quasiconvex Minimization on Hadamard Manifolds

In this paper we present two inexact proximal point algorithms to solve minimization problems for quasiconvex objective functions on Hadamard manifolds. We prove that under natural assumptions the sequence generated by the algorithms are well defined and converge to critical points of the problem. We also present an application of the method to demand theory in economy.

Enlargement of Monotone Vector Fields and an Inexact Proximal Point Method for Variational Inequalities in Hadamard Manifolds

In this paper an inexact proximal point method for variational inequalities in Hadamard manifolds is introduced and studied its convergence properties. The main tool used for presenting the method is the concept of enlargement of monotone vector fields, which generalizes the concept of enlargement of monotone operators from the linear setting to the Riemannian context. As an application, an inexact proximal point method for constrained optimization problems is obtained.

An inexact proximal method for quasiconvex minimization

In this paper we propose an inexact proximal point method to solve constrained minimization problems with locally Lipschitz quasiconvex objective functions. Assuming that the function is also bounded from below, lower semicontinuous and using proximal distances, we show that the sequence generated for the method converges to a stationary point of the problem.

A Generalized Inexact Proximal Point Method for Nonsmooth Functions that Satisfies Kurdyka Lojasiewicz Inequality

In this paper, following the ideas presented in Attouch et al. Math. Program. Ser. A, 137: 91–129, (2013), we present an inexact version of the proximal point method for nonsmooth functions, whose regularization is given by a generalized perturbation term. More precisely, the new perturbation term is defined as a “curved enough” function of the quasi distance between two successive iterates, that appears to be a nice tool for Behavioral Sciences (Psychology, Economics, Management, Game theory, … ). Our convergence analysis is an extension, of the analysis due to Attouch and Bolte Math. Program. Ser. B, 116: 5–16, (2009) and, more generally, to Moreno et al. Optimization, 61:1383–1403, (2011), to an inexact setting of the proximal method which is more suitable from the point of view of applications. In a dynamic setting, (Bento and Soubeyran (2014)) present a striking application on the famous Nobel Prize (Kahneman and Tversky. Econometrica 47(2), 263–291 (1979); Tversky and Kahneman. Q. J. Econ. 106(4), 1039–1061 (38))“loss aversion effect” in Psychology and Management. This application shows how the strength of resistance to change can impact the speed of formation of a habituation/routinization process.

Accelerating Block-Decomposition First-Order Methods for Solving Composite Saddle-Point and Two-Player Nash Equilibrium Problems

This article considers the (two-player) composite Nash equilibrium (CNE) problem with a separable nonsmooth part, which is known to include the composite saddle-point (CSP) problem as a special case. Due to its two-block structure, this problem can be solved by any algorithm belonging to the block-decomposition hybrid proximal-extragradient (BD-HPE) framework proposed in [R. D. C. Monteiro and B. F. Svaiter, SIAM J. Optim., 23 (2013), pp. 475--507]. The framework consists of a family of inexact proximal point methods for solving a more general two-block structured monotone inclusion problem which, at every iteration, solves two prox subinclusions according to a certain relative error criterion. By exploiting the fact that the two prox subinclusions in the context of the CNE problem are equivalent to two composite convex programs, this article proposes a new instance of the BD-HPE framework that approximately solves them using an accelerated gradient method. It is shown that the new instance is able to take significantly larger prox stepsizes than other instances from this framework that perform single composite gradient steps for solving the subinclusions. As a result, it is shown that the first instance has better iteration-complexity than the latter ones. Finally, it is also shown that the new accelerated BD-HPE instance computationally outperforms several state-of-the-art algorithms on many relevant classes of CSP and CNE instances.

Convergence analysis of inexact proximal point algorithms on Hadamard manifolds

Inexact proximal point methods are extended to find singular points for multivalued vector fields on Hadamard manifolds. Convergence criteria are established under some mild conditions. In particular, in the case of proximal point algorithm, that is, $$\varepsilon _n=0$$ ? n = 0 for each $$n$$ n , our results improve sharply the corresponding results in Li et al. (2009). Applications to optimization problems, variational inequality problems and gradient methods are also given.

Bilevel Optimization as a Regularization Approach to Pseudomonotone Equilibrium Problems

We study properties of an inexact proximal point method for pseudomonotone equilibrium problems in real Hilbert spaces. Unlike monotone problems, in pseudomonotone problems, the regularized subproblems may not be strongly monotone, even not pseudomonotone. However, we show that every inexact proximal trajectory weakly converges to the same limit. We use these properties to extend a viscosity-proximal point algorithm developed in [28] to pseudomonotone equilibrium problems. Then we propose a hybrid extragradient-cutting plane algorithm for approximating the limit point by solving a bilevel strongly convex optimization problem. Finally, we show that by using this bilevel convex optimization, the proximal point method can be used for handling ill-possed pseudomonotone equilibrium problems.