Strict Complementarity Condition Research Articles

On a Computationally Ill-Behaved Bilevel Problem with a Continuous and Nonconvex Lower Level

It is well known that bilevel optimization problems are hard to solve both in theory and practice. In this paper, we highlight a further computational difficulty when it comes to solving bilevel problems with continuous but nonconvex lower levels. Even if the lower-level problem is solved to varepsilon -feasibility regarding its nonlinear constraints for an arbitrarily small but positive varepsilon , the obtained bilevel solution as well as its objective value may be arbitrarily far away from the actual bilevel solution and its actual objective value. This result even holds for bilevel problems for which the nonconvex lower level is uniquely solvable, for which the strict complementarity condition holds, for which the feasible set is convex, and for which Slater’s constraint qualification is satisfied for all feasible upper-level decisions. Since the consideration of varepsilon -feasibility cannot be avoided when solving nonconvex problems to global optimality, our result shows that computational bilevel optimization with continuous and nonconvex lower levels needs to be done with great care. Finally, we illustrate that the nonlinearities in the lower level are the key reason for the observed bad behavior by showing that linear bilevel problems behave much better at least on the level of feasible solutions.

On the Efficient Implementation of the Matrix Exponentiated Gradient Algorithm for Low-Rank Matrix Optimization

Convex optimization over the spectrahedron, that is, the set of all real n × n positive semidefinite matrices with unit trace, has important applications in machine learning, signal processing, and statistics, mainly as a convex relaxation for optimization problems with low-rank matrices. It is also one of the most prominent examples in the theory of first order methods for convex optimization in which non-Euclidean methods can be significantly preferable to their Euclidean counterparts. In particular, the desirable choice is the matrix exponentiated gradient (MEG) method, which is based on the Bregman distance induced by the (negative) von Neumann entropy. Unfortunately, implementing MEG requires a full singular value decomposition (SVD) computation on each iteration, which is not scalable to high-dimensional problems. In this work, we propose efficient implementations of MEG, with both deterministic and stochastic gradients, which are tailored for optimization with low-rank matrices, and only use a single low-rank SVD computation on each iteration. We also provide efficiently computable certificates for the correct convergence of our methods. Mainly, we prove that, under a strict complementarity condition, the suggested methods converge from a warm-start initialization with similar rates to their full SVD–based counterparts. Finally, we bring empirical experiments that both support our theoretical findings and demonstrate the practical appeal of our methods.Funding: This work was supported by the Israel Science Foundation [Grant 1108/18].

Homogenization for polynomial optimization with unbounded sets

This paper considers polynomial optimization with unbounded sets. We give a homogenization formulation and propose a hierarchy of Moment-SOS relaxations to solve it. Under the assumptions that the feasible set is closed at infinity and the ideal of homogenized equality constraining polynomials is real radical, we show that this hierarchy of Moment-SOS relaxations has finite convergence, if some optimality conditions (i.e., the linear independence constraint qualification, strict complementarity and second order sufficient conditions) hold at every minimizer, including the one at infinity. Moreover, we prove extended versions of Putinar-Vasilescu type Positivstellensatz for polynomials that are nonnegative on unbounded sets. The classical Moment-SOS hierarchy with denominators is also studied. In particular, we give a positive answer to a conjecture of Mai, Lasserre and Magron in their recent work.

On the sensitivity of the optimal partition for parametric second-order conic optimization

In this paper, using an optimal partition approach, we study the parametric analysis of a second-order conic optimization problem, where the objective function is perturbed along a fixed direction. We characterize the notions of so-called invariancy set and nonlinearity interval, which serve as stability regions of the optimal partition. We then propose, under the strict complementarity condition, an iterative procedure to compute a nonlinearity interval of the optimal partition. Furthermore, under primal and dual nondegeneracy conditions, we show that a boundary point of a nonlinearity interval can be numerically identified from a nonlinear reformulation of the parametric second-order conic optimization problem. Our theoretical results are supported by numerical experiments.

Improved convergence analysis of a smoothing Newton method for the circular cone programming

ABSTRACT In this paper, we propose a new smoothing Newton method to solve the circular cone programming (denoted by CCP). The proposed method is designed based on a non-monotone derivative-free line search scheme. We show that any accumulation point of the iteration sequence generated by this method is a solution of the CCP. Moreover, we prove that the proposed method is locally quadratically convergent without requiring strict complementarity conditions. Compared with existing smoothing Newton methods for solving the CCP, our method has three new features: (i) the generated iteration sequence is bounded; (ii) the value of the merit function converges to zero; (iii) the whole iteration sequence converges to an accumulation point if this point is isolated. Some numerical results are reported.

The rate of convergence of proximal method of multipliers for second-order cone optimization problems

In this paper we consider a proximal method of multipliers (PMM) for a nonlinear second-order cone optimization problem. With the assumptions of constraint nondegeneracy, strict complementarity and second-order sufficient condition, we estimate the local convergence rate of PMM to be linear or superlinear, which depends on the strategy of parameter selection.

Relaxed constant positive linear dependence constraint qualification and its application to bilevel programs

Relaxed constant positive linear dependence constraint qualification (RCPLD) for a system of smooth equalities and inequalities is a constraint qualification that is weaker than the usual constraint qualifications such as Mangasarian Fromovitz constraint qualification and the linear constraint qualification. Moreover RCPLD is known to induce an error bound property. In this paper we extend RCPLD to a very general feasibility system which may include Lipschitz continuous inequality constraints, complementarity constraints and abstract constraints. We show that this RCPLD for the general system is a constraint qualification for the optimality condition in terms of limiting subdifferential and limiting normal cone and it is a sufficient condition for the error bound property under the strict complementarity condition for the complementarity system and Clarke regularity conditions for the inequality constraints and the abstract constraint set. Moreover we introduce and study some sufficient conditions for RCPLD including the relaxed constant rank constraint qualification. Finally we apply our results to the bilevel program.

The rate of convergence of proximal method of multipliers for nonlinear programming

We analyze the rate of convergence of the proximal method of multipliers for non-convex nonlinear programming problems. First, we prove, under the strict complementarity condition, that the rate of convergence of the proximal method of multipliers is linear and the ratio constant is proportional to 1/c when the ratio is small enough, which implies that the rate of convergence of the proximal method of multipliers is superlinear when the parameter c increases to . Second, we prove that, without strict complementarity condition, the rate of convergence of the proximal method of multipliers is proportional to 1/c when c exceeds a threshold.

2-factor Newton method for solving the constrained optimization problem with the singular Kuhn—Tucker system

A new method for solving the inequality constrained optimization problem is proposed for the case when the system of necessary optimality conditions of Kuhn—Tucker is degenerate. This situation occurs for example in the case when strict complementarity conditions fails in solution point. The reduction of the inequalities con- strained optimization problem to the equalities constrained problem is substantiated and the use of a new 2-fac- tor Newton method for the effective solution of the obtained degenerate system of optimality conditions is shown.

A new smoothing method for solving nonlinear complementarity problems

Abstract In this paper, a new improved smoothing Newton algorithm for the nonlinear complementarity problem was proposed. This method has two-fold advantages. First, compared with the classical smoothing Newton method, our proposed method needn’t nonsingular of the smoothing approximation function; second, the method also inherits the advantage of the classical smoothing Newton method, it only needs to solve one linear system of equations at each iteration. Without the need of strict complementarity conditions and the assumption of P0 property, we get the global and local quadratic convergence properties of the proposed method. Numerical experiments show that the efficiency of the proposed method.

2-Factor Newton Method for Solving Constrained Optimization Problems with a Degenerate Kuhn–Tucker System

A new method is proposed for solving constrained optimization problems with inequality constraints in the case when the system of Kuhn–Tucker necessary optimality conditions is degenerate. This situation arises, for example, when the strict complementarity conditions do not hold for the solution. A reduction of the inequality-constraint problem to an equality-constraint one and the use of a new 2 factor Newton method for efficiently solving the resulting degenerate system of optimality conditions are justified.

Mathematical Programs with Second-Order Cone Complementarity Constraints: Strong Stationarity and Approximation Method

The existence of complementarity constraints causes the difficulties for studying mathematical programs with second-order cone complementarity constraints, since the standard constraint qualification, such as Robinson’s constraint qualification, is invalid. Therefore, various stationary conditions including strong, Mordukhovich and Clarke stationary conditions have been proposed, according to different reformulations of the second-order cone complementarity constraints. In this paper, we present a new reformulation of this problem by taking into consideration the Jordan algebra associated with the second-order cone. It ensures that the classical Karush–Kuhn–Tucker condition coincides with the strong stationary condition of the original problem. Furthermore, we propose a class of approximation methods to solve mathematical programs with second-order cone complementarity constraints. Any accumulation point of the iterative sequences, generated by the approximation method, is Clarke stationary under the corresponding linear independence constraint qualification. This stationarity can be enhanced to strong stationarity with an extra strict complementarity condition. Preliminary numerical experiments indicate that the proposed method is effective.

Quadratic convergence to the optimal solution of second-order conic optimization without strict complementarity

ABSTRACTUnder primal and dual nondegeneracy conditions, we establish the quadratic convergence of Newton's method to the unique optimal solution of second-order conic optimization. Only very few approaches have been proposed to remedy the failure of strict complementarity, mostly based on nonsmooth analysis of the optimality conditions. Our local convergence result depends on the optimal partition of the problem, which can be identified from a bounded sequence of interior solutions. We provide a theoretical complexity bound for identifying the quadratic convergence region of Newton's method from the trajectory of central solutions. By way of experimentation, we illustrate quadratic convergence of Newton's method on some SOCO problems which fail strict complementarity condition.

Generalized conditioning based approaches to computing confidence intervals for solutions to stochastic variational inequalities

Stochastic variational inequalities (SVI) provide a unified framework for the study of a general class of nonlinear optimization and Nash-type equilibrium problems with uncertain model data. Often the true solution to an SVI cannot be found directly and must be approximated. This paper considers the use of a sample average approximation (SAA), and proposes a new method to compute confidence intervals for individual components of the true SVI solution based on the asymptotic distribution of SAA solutions. We estimate the asymptotic distribution based on one SAA solution instead of generating multiple SAA solutions, and can handle inequality constraints without requiring the strict complementarity condition in the standard nonlinear programming setting. The method in this paper uses the confidence regions to guide the selection of a single piece of a piecewise linear function that governs the asymptotic distribution of SAA solutions, and does not rely on convergence rates of the SAA solutions in probability. It also provides options to control the computation procedure and investigate effects of certain key estimates on the intervals.

Optimal Control of a Viscous Two‐Field Gradient Damage Model

AbstractThe paper deals with a class of optimal control problems governed by a viscous damage model including two damage variables which are coupled through a penalty term. The existence and directional differentiability of the associated control‐to‐state operator is established. Moreover, necessary optimality conditions are derived. It is shown that these are equivalent to an optimality system, provided that a strict complementarity condition is satisfied.

Local and global convergence of a general inertial proximal splitting scheme for minimizing composite functions

This paper is concerned with convex composite minimization problems in a Hilbert space. In these problems, the objective is the sum of two closed, proper, and convex functions where one is smooth and the other admits a computationally inexpensive proximal operator. We analyze a family of generalized inertial proximal splitting algorithms (GIPSA) for solving such problems. We establish weak convergence of the generated sequence when the minimum is attained. Our analysis unifies and extends several previous results. We then focus on $$\ell _1$$ -regularized optimization, which is the ubiquitous special case where the nonsmooth term is the $$\ell _1$$ -norm. For certain parameter choices, GIPSA is amenable to a local analysis for this problem. For these choices we show that GIPSA achieves finite “active manifold identification”, i.e. convergence in a finite number of iterations to the optimal support and sign, after which GIPSA reduces to minimizing a local smooth function. We prove local linear convergence under either restricted strong convexity or a strict complementarity condition. We determine the rate in terms of the inertia, stepsize, and local curvature. Our local analysis is applicable to certain recent variants of the Fast Iterative Shrinkage–Thresholding Algorithm (FISTA), for which we establish active manifold identification and local linear convergence. Based on our analysis we propose a momentum restart scheme in these FISTA variants to obtain the optimal local linear convergence rate while maintaining desirable global properties.

Generic Properties for Semialgebraic Programs

In this paper we study genericity for the class of semialgebraic optimization problems with equality and inequality constraints, in which every problem of the class is obtained by linear perturbations of the objective function, while the “core” objective function and the constraint functions are kept fixed. Assume that the linear independence constraint qualification is satisfied at every point in the constraint set. It is shown that almost all problems in the class are such that (i) the restriction of the objective function on the constraint set is coercive and regular at infinity; (ii) there is a unique optimal solution, lying on a unique active manifold, at which the strict complementarity and second order sufficiency conditions, the quadratic growth condition, and the Hölder type global error bound hold, and (iii) all minimizing sequences converge. Furthermore, the active manifold is constant, and the optimal solution and the optimal value function depend analytically under local perturbations of the objective function. These results are combined with a standard result about the existence of sums of squares certificates to prove that we can build a sequence of semidefinite programs whose solutions give rise to a sequence of points converging to the optimal solution of the original problem in finitely many steps. It is worth emphasizing that the results of this study hold globally and we do not require the problem to be convex or the constraint set to be compact.

Copositivity and Sparsity Relations Using Spectral Properties

A standard quadratic optimization problem consists in minimizing a quadratic form over the standard simplex. This problem has a closer connection with copositive optimization, where the copositivity appears in local optimality conditions. In this paper, we establish a relationship between the sparsity of a solution of the standard quadratic optimization problem, from one hand, and the copositivity of the Hessian matrix of the Lagrangian from other hand. More precisely, if the Hessian matrix is copositive we prove that the number of zero components, of a local minimizer associated with a standard quadratic optimization problem, is greater or equal to the number of negative eigenvalues counting multiplicities. Moreover, we show that if the number of zero components of a local minimizer is equal to the number of negative eigenvalues of the Hessian matrix, then the strict complementarity condition is satisfied and the critical cone is a linear subspace.

Generic Minimizing Behavior in Semialgebraic Optimization

We present a theorem of Sard type for semialgebraic set-valued mappings whose graphs have dimension no larger than that of their range space: the inverse of such a mapping admits a single-valued analytic localization around any pair in the graph, for a generic value parameter. This simple result yields a transparent and unified treatment of generic properties of semialgebraic optimization problems: “typical” semialgebraic problems have finitely many critical points, around each of which they admit a unique “active manifold” (analogue of an active set in nonlinear optimization); moreover, such critical points satisfy strict complementarity and second-order sufficient conditions for optimality are indeed necessary.

A one-parametric class of smoothing functions and an improved regularization Newton method for the NCP

In this paper, we introduce a one-parametric class of smoothing functions, which enjoys some favourable properties and includes two famous smoothing functions as special cases. Based on this class of smoothing functions, we propose a regularization Newton method for solving the non-linear complementarity problem. The main feature of the proposed method is that it uses a perturbed Newton equation to obtain the direction. This not only allows our method to have global and local quadratic convergences without strict complementarity conditions, but also makes the regularization parameter converge to zero globally Q-linearly. In addition, we use a new non-monotone line search scheme to obtain the step size. Some numerical results are reported which confirm the good theoretical properties of the proposed method.