Inexact Proximal Point Algorithm Research Articles

Convergence Rate of Inexact Proximal Point Algorithms for Operator with Hölder Metric Subregularity

Convergence Rate of Inexact Proximal Point Algorithms for Operator with Hölder Metric Subregularity

Iteration-Complexity of First-Order Augmented Lagrangian Methods for Convex Conic Programming

In this paper we consider a class of convex conic programming. In particular, we first propose an inexact augmented Lagrangian (I-AL) method that resembles the classical I-AL method for solving this problem, in which the augmented Lagrangian subproblems are solved approximately by a variant of Nesterov’s optimal first-order method. We show that the total number of first-order iterations of the proposed I-AL method for finding an -KKT solution is at most . We then propose an adaptively regularized I-AL method and show that it achieves a first-order iteration complexity , which significantly improves existing complexity bounds achieved by first-order I-AL methods for finding an -KKT solution. Our complexity analysis of the I-AL methods is based on a sharp analysis of the inexact proximal point algorithm (PPA) and the connection between the I-AL methods and inexact PPA. It is vastly different from existing complexity analyses of the first-order I-AL methods in the literature, which typically regard the I-AL methods as an inexact dual gradient method.

On finite termination of an inexact Proximal Point algorithm

The presence of sharp minima in nondifferentiable optimization models has been exploited, in the last decades, in the benefit of various subgradient or proximal methods. One of the long-lasting general proximal schemes of choice used to minimize nonsmooth functions is the Proximal Point Algorithm (PPA). Regarding the basic PPA, several well-known works proved finite convergence towards weak sharp minima, when supposedly each iteration is computed exactly. However, in this letter we show finite convergence of a common Inexact version of PPA (IPPA), under sufficiently low but persistent perturbations of the proximal operator. Moreover, when a simple Subgradient Method is recurrently called as an inner routine for computing each IPPA iterate, a suboptimal minimizer of the original problem lying at ϵ distance from the optimal set is obtained after a total Olog(1/ϵ) subgradient evaluations. Our preliminary numerical tests show improvements over existing restartation versions of Subgradient Method.

Bregman Proximal Point Algorithm Revisited: A New Inexact Version and Its Inertial Variant

We study a general convex optimization problem, which covers various classic problems in different areas and particularly includes many optimal transport related problems arising in recent years. To solve this problem, we revisit the classic Bregman proximal point algorithm (BPPA) and introduce a new inexact stopping condition for solving the subproblems, which can circumvent the underlying feasibility difficulty often appearing in existing inexact conditions when the problem has a complex feasible set. Our inexact condition also covers several existing inexact conditions as special cases and hence makes our inexact BPPA (iBPPA) more flexible to fit different scenarios in practice. As an application to the standard optimal transport (OT) problem, our iBPPA with the entropic proximal term can bypass some numerical instability issues that usually plague the popular Sinkhorn's algorithm in the OT community, since our iBPPA does not require the proximal parameter to be very small for obtaining an accurate approximate solution. The iteration complexity of $O(1/k)$ and the convergence of the sequence are also established for our iBPPA under some mild conditions. Moreover, inspired by Nesterov's acceleration technique, we develop an inertial variant of our iBPPA, denoted by V-iBPPA, and establish the iteration complexity of $O(1/k^{\lambda})$, where $\lambda\geq1$ is a quadrangle scaling exponent of the kernel function. In particular, when the proximal parameter is a constant and the kernel function is strongly convex with Lipschitz continuous gradient (hence $\lambda=2$), our V-iBPPA achieves a faster rate of $O(1/k^2)$ just as existing accelerated inexact proximal point algorithms. Some preliminary numerical experiments for solving the standard OT problem are conducted to show the convergence behaviors of our iBPPA and V-iBPPA under different inexactness settings. The experiments also empirically verify the potential of our V-iBPPA for improving the convergence speed.

Inexact proximal point algorithm for quasiconvex optimization problems on Hadamard manifolds

ABSTRACT In this paper, by using the inexact scalarization proximal methods, we solve quasiconvex multiobjective optimization problems on Hadamard manifolds. Under some assumptions on the multifunction of problem and vector fields, our methods are proved to be convergent to a Pareto critical point of the problem. For the convex case, the sequences generated by the methods converge to a weak Pareto solution.

Accelerated differential inclusion for convex optimization

This paper introduces a second-order differential inclusion for unconstrained convex optimization. In continuous level, solution existence in proper sense is obtained and exponential decay of a novel Lyapunov function along with the solution trajectory is derived as well. Then in discrete level, based on numerical discretizations of the continuous model, two inexact proximal point algorithms are proposed, and some new convergence rates are established via a discrete Lyapunov function.

Strong Convergence of an Inexact Proximal Point Algorithm in a Banach Space

By using our own approach, we study the strong convergence of an inexact proximal point algorithm with possible unbounded errors for a maximal monotone operator in a Banach space. We give a necessary and sufficient condition for the zero set of the operator to be nonempty and show that, in this case, this iterative sequence converges strongly to a zero of the operator. We present also some applications of our results to equilibrium problems and optimization. Our proximal point algorithm contains, as a special case, the one considered in Hilbert space by Djafari Rouhani and Moradi in (J Optim Theory Appl 172:222–235, 2017) and solves the open problem of extending it to a Banach space, which was stated in that paper and in Djafari Rouhani and Moradi in (J Optim Theory Appl 181:864–882, 2019) . Since the nonexpansiveness of the resolvent operator, which holds in Hilbert space, is not valid anymore in Banach space, our results require new methods of proofs, and significantly improve upon the previous results, both in theory and in applications.

Weak sharpness and finite termination for variational inequalities on Hadamard manifolds

We first introduce the notion of weak sharpness for the solution sets of variational inequality problems (in short, VIP) on Hadamard spaces. We then study the finite convergence property of sequences generated by the inexact proximal point algorithm with different error terms for solving VIP under weak sharpness of the solution set. We also give an upper bound on the number of iterations by which the sequence generated by the exact proximal point algorithm converges to a solution of VIP. An example is also given to illustrate our results.

Strong Convergence of the Hybrid Halpern Type Proximal Point Algorithm

Based on the proximal point algorithm, which is a widely used tool for solving a variety of convex optimization problems, there are many algorithms for finding zeros of maximally monotone operators. The algorithm works by applying successively so-called resolvent mappings with errors associated to the original object, and is weakly convergent in Hilbert space. In order to acquiring the strong convergence of the algorithm, in this paper, we construct a hybrid Halpern type proximal point algorithm with errors for approximating the zero of a maximal monotone operator, which is a combination of modified proximal point algorithm raised by Yao and Noor and Halpern inexact proximal point algorithm raised by Zhang, respectively. Then, we prove the strong convergence of our algorithm with weaker assumptions in Hilbert space. Finally, we present a numerical example to show the convergence and the convergence speed, which is not affected but accelerated by the projection in the algorithm. Our work improved and generalized some known results.

Regularization of proximal point algorithms in Hadamard manifolds

In this paper, we consider the regularization method for exact as well as for inexact proximal point algorithms for finding the singularities of maximal monotone set-valued vector fields. We prove that the sequences generated by these algorithms converge to an element of the set of singularities of a maximal monotone set-valued vector field. A numerical example is provided to illustrate the inexact proximal point algorithm with regularization. Applications of our results to minimization problems and saddle point problems are given in the setting of Hadamard manifolds.

An inexact algorithm with proximal distances for variational inequalities

In this paper we introduce an inexact proximal point algorithm using proximal distances for solving variational inequality problems when the mapping is pseudomonotone or quasimonotone. Under some natural assumptions we prove that the sequence generated by the algorithm is convergent for the pseudomonotone case and assuming an extra condition on the solution set we prove the convergence for the quasimonotone case. This approach unifies the results obtained by Auslender et al. [Math Oper. Res. 24 (1999) 644–688] and Brito et al. [J. Optim. Theory Appl. 154 (2012) 217–234] and extends the convergence properties for the class of φ-divergence distances and Bregman distances.

New results on the proximal point algorithm in non-positive curvature metric spaces

The subject of this paper is the inexact proximal point algorithm of usual and Halpern type in non-positive curvature metric spaces. We study the convergence of the sequences given by the inexact proximal point algorithm with non-summable errors. We also prove the strong convergence of the Halpern proximal point algorithm to a minimum point of the convex function. The results extend several results in Hilbert spaces, Hadamard manifolds and non-positive curvature metric spaces.

On the Convergence Rate of a Proximal Point Algorithm for Vector Function on Hadamard Manifolds

The proximal point algorithm has many interesting applications, such as signal recovery, signal processing and others. In recent years, the proximal point method has been extended to Riemannian manifolds. The main advantages of these extensions are that nonconvex problems in classic sense may become geodesic convex by introducing an appropriate Riemannian metric, constrained optimization problems may be seen as unconstrained ones. In this paper, we propose an inexact proximal point algorithm for geodesic convex vector function on Hadamard manifolds. Under the assumption that the objective function is coercive, the sequence generated by this algorithm converges to a Pareto critical point. When the objective function is coercive and strictly geodesic convex, the sequence generated by this algorithm converges to a Pareto optimal point. Furthermore, under the weaker growth condition, we prove that the inexact proximal point algorithm has linear/superlinear convergence rate.

On the Convergence Rate of an Inexact Proximal Point Algorithm for Quasiconvex Minimization on Hadamard Manifolds

In this paper, we present an analysis about the rate of convergence of an inexact proximal point algorithm to solve minimization problems for quasiconvex objective functions on Hadamard manifolds. We prove that under natural assumptions the sequence generated by the algorithm converges linearly or superlinearly to a critical point of the problem.

Inexact Proximal Point Methods for Quasiconvex Minimization on Hadamard Manifolds

In this paper we present two inexact proximal point algorithms to solve minimization problems for quasiconvex objective functions on Hadamard manifolds. We prove that under natural assumptions the sequence generated by the algorithms are well defined and converge to critical points of the problem. We also present an application of the method to demand theory in economy.

MONOTONE OPERATORS AND THE PROXIMAL POINT ALGORITHM IN COMPLETE CAT(0) METRIC SPACES

In this paper, we generalize monotone operators, their resolvents and the proximal point algorithm to complete CAT(0) spaces. We study some properties of monotone operators and their resolvents. We show that the sequence generated by the inexact proximal point algorithm $\unicode[STIX]{x1D6E5}$-converges to a zero of the monotone operator in complete CAT(0) spaces. A strong convergence (convergence in metric) result is also presented. Finally, we consider two important special cases of monotone operators and we prove that they satisfy the range condition (see Section 4 for the definition), which guarantees the existence of the sequence generated by the proximal point algorithm.

Proximal Point Algorithms on Hadamard Manifolds: Linear Convergence and Finite Termination

In the present paper, we consider inexact proximal point algorithms for finding singular points of multivalued vector fields on Hadamard manifolds. The rate of convergence is shown to be linear under the mild assumption of metric subregularity. Furthermore, if the sequence of parameters associated with the iterative scheme converges to $0$, then the convergence rate is superlinear. At the same time, the finite termination of the inexact proximal point algorithm is also provided under a weak sharp minima-like condition. Applications to optimization problems are provided. Some of our results are new even in Euclidean spaces, while others improve and/or extend some known results in Euclidean spaces. As a matter of fact, in the case of exact proximal point algorithm, our results improve the corresponding results in [G. C. Bento and J. X. Cruz Neto, Optim., 63 (2014), pp. 1281--1288]. Finally, several examples are provided to illustrate that our results are applicable while the corresponding results in the Hil...

A variational inequality in complete CAT(0) spaces

In this paper, first, existence of solutions for a variational inequality associated with a nonexpansive mapping in Hadamard spaces is studied. Then, the inexact proximal point algorithm for approximation of a solution of the variational inequality, which is also a fixed point of the nonexpansive mapping, is proposed. We prove the \({\Delta}\)-convergence of the generated sequence by the algorithm as well as the strong convergence of a Halpern-type regularization one to a fixed point of the nonexpansive mapping. Our motivation is to give a step toward investigation of variational inequalities in CAT(0) spaces.

Newton's Method for Solving Inclusions Using Set-Valued Approximations

Results on stability of both local and global metric regularity under set-valued perturbations are presented. As an application, we study (super)linear convergence of a Newton-type iterative process for solving generalized equations. We investigate several iterative schemes such as the inexact Newton's method, the nonsmooth Newton's method for semismooth functions, the inexact proximal point algorithm, etc. Moreover, we also cover a forward-backward splitting algorithm for finding a zero of the sum of two multivalued (not necessarily monotone) operators. Finally, a globalization of the Newton's method is discussed.

Finite Termination of Inexact Proximal Point Algorithms in Hilbert Spaces

In the present paper, we study the finite termination of sequences generated by inexact proximal point algorithms for finding zeroes of a maximal monotone (set-valued) operator $$T$$T on a Hilbert space. Under some mild conditions, we get that a sequence generated by inexact proximal point algorithm stops after a finite number of iterations. Our results extend the corresponding results in Rockafellar (SIAM J Control Optim 14:877---898, 1976). In particular, for optimization problems, our results improve corresponding results in Ferris (Math Progr 50:359---366, 1991). As applications, we obtain finite termination of projected gradient method.