Strong Convexity Assumption Research Articles

Proximal gradient method for huberized support vector machine

The Support Vector Machine (SVM) has been used in a wide variety of classification problems. The original SVM uses the hinge loss function, which is non-differentiable and makes the problem difficult to solve in particular for regularized SVMs, such as with $\ell_1$-regularization. This paper considers the Huberized SVM (HSVM), which uses a differentiable approximation of the hinge loss function. We first explore the use of the Proximal Gradient (PG) method to solving binary-class HSVM (B-HSVM) and then generalize it to multi-class HSVM (M-HSVM). Under strong convexity assumptions, we show that our algorithm converges linearly. In addition, we give a finite convergence result about the support of the solution, based on which we further accelerate the algorithm by a two-stage method. We present extensive numerical experiments on both synthetic and real datasets which demonstrate the superiority of our methods over some state-of-the-art methods for both binary- and multi-class SVMs.

On linear convergence of a distributed dual gradient algorithm for linearly constrained separable convex problems

In this paper we propose a fully distributed dual gradient algorithm for minimizing linearly constrained separable convex problems and analyze its rate of convergence. In particular, we prove that under the assumption of strong convexity and Lipschitz continuity of the gradient of the primal objective function we have a global error bound type property for the dual problem. Using this error bound property we devise a fully distributed dual gradient scheme, i.e. a gradient scheme based on a weighted step size, for which we derive global linear rate of convergence for both dual and primal suboptimality and for primal feasibility violation. Numerical simulations are also provided to confirm our theory.

Convergence Rate Analysis of the Forward-Douglas-Rachford Splitting Scheme

Operator splitting schemes are a class of powerful algorithms that solve complicated monotone inclusion and convex optimization problems that are built from many simpler pieces. They give rise to algorithms in which all simple pieces of the decomposition are processed individually. This leads to easily implementable and highly parallelizable or distributed algorithms, which often obtain nearly state-of-the-art performance. In this paper, we analyze the convergence rate of the forward-Douglas--Rachford splitting (FDRS) algorithm, which is a generalization of the forward-backward splitting (FBS) and Douglas--Rachford splitting algorithms. Under general convexity assumptions, we derive the ergodic and nonergodic convergence rates of the FDRS algorithm, and show that these rates are the best possible. Under Lipschitz differentiability assumptions, we show that the best iterate of FDRS converges as quickly as the last iterate of the FBS algorithm. Under strong convexity assumptions, we derive convergence rates...

On the Linear Convergence of the Approximate Proximal Splitting Method for Non-smooth Convex Optimization

Consider the problem of minimizing the sum of two convex functions, one being smooth and the other non-smooth. In this paper, we introduce a general class of approximate proximal splitting (APS) methods for solving such minimization problems. Methods in the APS class include many well-known algorithms such as the proximal splitting method, the block coordinate descent method (BCD), and the approximate gradient projection methods for smooth convex optimization. We establish the linear convergence of APS methods under a local error bound assumption. Since the latter is known to hold for compressive sensing and sparse group LASSO problems, our analysis implies the linear convergence of the BCD method for these problems without strong convexity assumption.

The Binet–Legendre Metric in Finsler Geometry

For every Finsler metric $F$ we associate a Riemannian metric $g_F$ (called the Binet-Legendre metric). The transformation $F \mapsto g_F$ is $C^0$-stable and has good smoothness properties, in contrast to previous constructions. The Riemannian metric $g_F$ also behaves nicely under conformal or bilipshitz deformation of the Finsler metric $F$. These properties makes it a powerful tool in Finsler geometry and we illustrate that by solving a number of named Finslerian geometric problems. We also generalize and give new and shorter proofs of a number of known results. In particular we answer a question of M. Matsumoto about local conformal mapping between two Minkowski spaces, we describe all possible conformal self maps and all self similarities on a Finsler manifold. We also classify all compact conformally flat Finsler manifolds, solve a conjecture of S. Deng and Z. Hou on the Berwaldian character of locally symmetric Finsler spaces, and extend the classic result of H.C. Wang about the maximal dimension of the isometry groups of Finsler manifolds to manifolds of all dimensions. Most proofs in this paper go along the following scheme: using the correspondence $F \mapsto g_F$we reduce the Finslerian problem to a similar problem for the Binet-Legendre metric, which is easier and is already solved in most cases we consider. The solution of the Riemannian problem provides us with the additional information that helps to solve the initial Finslerian problem. Our methods apply even in the absence of the strong convexity assumption usually assumed in Finsler geometry. The smoothness hypothesis can also be replaced by that of partial smoothness, a notion we introduce in the paper. Our results apply therefore to a vast class of Finsler metrics not usually considered in the Finsler literature.

A second-order differential system with Hessian-driven damping; Application to non-elastic shock laws

We consider the second-order differential system with Hessian-driven damping ¨ u + α ˙ u +β∇ 2 Φ(u)˙ u +∇Φ(u )+ ∇Ψ(u )= 0, where H is a real Hilbert space, Φ,Ψ : H → R are scalar potentials, and α,β are positive parameters. An interesting property of this system is that, after introduction of an auxiliary variable y, it can be equivalently written as a first- order system involving only the time derivatives ˙ u, ˙ y and the gradient operators ∇Φ, ∇Ψ. This allows to extend our analysis to the case of a convex lower semicontinuous function Φ : H → R ∪{ +∞}, and so to introduce constraints in our model. When Φ = δK is the indicator function of a closed convex set K ⊆ H , the subdifferential operator ∂Φ takes account of the contact forces, while ∇Ψ takes account of the driving forces. In this setting, by playing with the geometrical damping parameter β , we can describe nonelastic shock laws with restitution coefficient. Taking advantage of the infinite dimensional framework, we introduce a nonlinear hyperbolic PDE describing a damped oscillating system with obstacle. The first-order system is dissipative; each trajectory weakly converges to a minimizer of Φ + Ψ, provided that Φ and Φ +Ψ are convex functions. Exponential stabilization is obtained under strong convexity assumptions.

Modification of the Wolfe Line Search Rules to Satisfy the Descent Condition in the Polak-Ribière-Polyak Conjugate Gradient Method

This paper proposes a line search technique to satisfy a relaxed form of the strong Wolfe conditions in order to guarantee the descent condition at each iteration of the Polak-Ribiere-Polyak conjugate gradient algorithm. It is proved that this line search algorithm preserves the usual convergence properties of any descent algorithm. In particular, it is shown that the Zoutendijk condition holds under mild assumptions. It is also proved that the resulting conjugate gradient algorithm is convergent under a strong convexity assumption. For the nonconvex case, a globally convergent modification is proposed. Numerical tests are presented.

A Note on Line Search Termination Criteria for Collinear Scaling Algorithms

Ariyawansa [2] has introduced a line search termination criterion for collinear scaling minimization algorithms. However, he verified that this criterion provides sufficient decrease only in the case of objective functions that satisfy certain strong convexity assumptions. In this paper, we report results of our attempts to relax these assumptions. We present an example of a continuously differentiable, nonconvex function with bounded level sets on which the criterion of Ariyawansa [2] does not provide sufficient decrease. This example indicates that specifying line search termination criteria for collinear scaling algorithms is still an open problem. On a more positive note, we are able to show that the class of objective functions on which the criterion of Ariyawansa [2] provides sufficient decrease can be extended to include convex and strictly pseudo-convex functions.

Globally Convergent BFGS Method for Nonsmooth Convex Optimization1

We propose an implementable BFGS method for solving a nonsmooth convex optimization problem by converting the original objective function into a once continuously differentiable function by way of the Moreau–Yosida regularization. The proposed method makes use of approximate function and gradient values of the Moreau-Yosida regularization instead of the corresponding exact values. We prove the global convergence of the proposed method under the assumption of strong convexity of the objective function.

First-Order Algorithms for Generalized Semi-Infinite Min-Max Problems

We present a first-order algorithm for solving semi-infinite generalized min-max problems which consist of minimizing a function f^0(x) e F(ψ^1(x), …, ψ^m(x)), where F is a smooth function and each ψ^i is the maximum of an infinite number of smooth functions. In Section 3.3 of [17] Polak finds a methodology for solving infinite dimensional problems by expanding them into an infinite sequence of consistent finite dimensional approximating problems, and then using a master algorithm that selects an appropriate subsequence of these problems and applies a number of iterations of a finite dimensional optimization algorithm to each of these problems, sequentially. Our algorithm was constructed within this frameworks it calls an algorithm by Kiwiel as a subroutine. The number of iterations of the Kiwiel algorithm to be applied to the approximating problems is determined by a test that ensures that the overall scheme retains the rate of convergence of the Kiwiel algorithm. Under reasonable assumptions we show that all the accumulation points of sequences constructed by our algorithm are stationary, and, under an additional strong convexity assumption, that the Kiwiel algorithm converges at least linearly, and that our algorithm also converges at least linearly, with the same rate constant bounds as Kiwiel‘s.

A globally convergent version of the Polak-Ribière conjugate gradient method

In this paper we propose a new line search algorithm that ensures global convergence of the Polak-Ribiere conjugate gradient method for the unconstrained minimization of nonconvex differentiable functions. In particular, we show that with this line search every limit point produced by the Polak-Ribiere iteration is a stationary point of the objective function. Moreover, we define adaptive rules for the choice of the parameters in a way that the first stationary point along a search direction can be eventually accepted when the algorithm is converging to a minimum point with positive definite Hessian matrix. Under strong convexity assumptions, the known global convergence results can be reobtained as a special case. From a computational point of view, we may expect that an algorithm incorporating the step-size acceptance rules proposed here will retain the same good features of the Polak-Ribiere method, while avoiding pathological situations.

Symmetric hypersurfaces in Riemannian manifolds contracting to Lie-groups by their mean curvature

This paper concerns the deformation by mean curvature of hypersurfaces M in Riemannian spaces N that are invariant under a subgroup of the isometry-group on N. We show that the hypersurfaces contract to this subgroup, if the cross-section satisfies a strong convexity assumption.

Stability of Solutions for Stochastic Programs with Complete Recourse

Quantitative continuity of optimal solution sets to convex stochastic programs with (linear) complete recourse and random right-hand sides is investigated when the underlying probability measure varies in a metric space. The central result asserts that, under a strong-convexity condition for the expected recourse in the unperturbed problem, optimal tenders behave Hölder-continuous with respect to a Wasserstein metric. For linear stochastic programs this carries over to the Hausdorff distance of optimal solution sets A general sufficient condition for the crucial strong-convexity assumption is given and verified for recourse problems with separable and nonseparable objectives.

On finite convergence and constraint identification of subgradient projection methods

This paper deals with a continuous time, subgradient projection algorithm, shown to generate trajectories that accumulate to the solution set. Under a strong convexity assumption we show that convergence is exponential in norm. A sharpness condition yields convergence in finite time, and the necessary lapse is estimated. Invoking a constraint qualification and a non-degeneracy assumption, we demonstrate that optimally active constraints are identified in finite time.

Remarks on the first welfare theorem with nonordered preferences

A strong convexity assumption is used to establish the first fundamental theorem of welfare economics using the strong definition of Pareto optimality without assuming that the weak preference relations are complete or transitive. An example demonstrates that Debreu's ( Proc. Nat. Acad. Sci., 40 (1954), 588–592) nonsatiation and convexity assumptions are insufficient to establish the result in the absence of transitivity.

Decentralization with externalities

In this paper, we examine in a systematic way the problem of decentralization in an economy with all types of externalities. We show that decentralization with artificial markets requires strong convexity assumptions which are typically not fulfilled for negative externalities on producers. A milder form of decentralization with taxation is shown to obtain under weaker convexity assumptions. A new example is given to explain the role of a non-interiority assumption. Finally, new types of non-convexities are described and the problem of rights to externalities is clearly defined in the framework of this general theory.

Oka’s Heftungslemma and the Levi problem for complex spaces

The proof given here is based on a refined version of the Heftungslemma of Oka [11] which we give in ?2, and avoids the consideration of completely continuous mappings between Fr6chet spaces [4]. It also avoids a detour through the following result, which was required in the older proof. A holomorph-convex K-complete, space is a Stein space. In the second part, we consider the question of the extent to which the assumption of strong convexity can be dropped. Theorem 1 becomes false if we merely drop the assumption of strong convexity as shown by an example of H. Grauert (see L10a] where this example is given). In this direction we prove.