Bounded Level Sets Research Articles

A New Complementarity Function and Applications in Stochastic Second-Order Cone Complementarity Problems

This paper considers the so-called expected residual minimization (ERM) formulation for stochastic second-order cone complementarity problems, which is based on a new complementarity function called termwise residual complementarity function associated with second-order cone. We show that the ERM model has bounded level sets under the stochastic weak $$R_0$$ -property. We further derive some error bound results under either the strong monotonicity or some kind of constraint qualifications. Then, we apply the Monte Carlo approximation techniques to solve the ERM model and establish a comprehensive convergence analysis. Furthermore, we report some numerical results on a stochastic second-order cone model for optimal power flow in radial networks.

Cubic overestimation and secant updating for unconstrained optimization of C2, 1 functions

The discrepancy between an objective function f and its local quadratic model f(x)+∇ f(x)⊤ s+s⊤ H(x) s/2 ≈ f(x+s) at the current iterate x is estimated using a cubic term q |s|3/3. Potential steps are chosen such that they minimize (or at least significantly reduce) the overestimating function ∇ f(x)⊤ s+s⊤ B s/2+q |s|3/3 with B ≈ H(x). This ensures f(x+s)<f(x) unless the approximating Hessian B=B⊤ differs significantly from H(x) or the scalar q>0 is too small. Either one or both quantities may be updated after unsuccessful and successful steps alike. For an algorithm employing both the symmetric rank one update and a shifted version of the BFGS formula we show that either∈f |∇ f|=0 or sup |B|=∞, provided the Hessian H(x) is Lipschitz on some neighbourhood of a bounded level set. Superlinear convergence is theoretically expected and numerically observed but not yet proven.

A Continuous Dynamical Systems Approach to Gauss-Newton Minimization

In this paper we show how the iterative Gauss-Newton method for minimizing a function can be reformulated as a solution to a continuous, autonomous dynamical system. We investigate the properties of the solutions to a one-parameter ODE initial value problem that involves the gradient and Hessian of the function. The equation incorporates an eigenvalue shift conditioner, which is a non-negative continuous function of the state. It enforces positive definiteness on a modified Hessian. Assuming the existence of a unique global minimum, the existence of a bounded connected sub-level set of the function and that the Hessian is non-zero in the interior of this set, our main results are: 1) existence of local solutions to the ODE initial value problem; 2) construction of a global solution by recursive extension of local solutions; 3) convergence of the global solution to the minimizing state for all initial values contained in the interior of the bounded level set; 4) eventual exact exponential decay of the gradient magnitude independent of the particular function and number of its variables. The results of a numerical experiment on the Rosenbrock Banana using a constant step-size 4th order Runge-Kutta method are presented and we point toward the direction of future research.

The divergence of the BFGS and Gauss Newton methods

We present examples of divergence for the BFGS and Gauss Newton methods. These examples have objective functions with bounded level sets and other properties concerning the examples published recently in this journal, like unit steps and convexity along the search lines. As these other examples, the iterates, function values and gradients in the new examples fit into the general formulation in our previous work Mascarenhas (Comput Appl Math 26(1), 2007), which also presents an example of divergence for Newton's method.

A Two-Parametric Class of Merit Functions for the Second-Order Cone Complementarity Problem

We propose a two-parametric class of merit functions for the second-order cone complementarity problem (SOCCP) based on the one-parametric class of complementarity functions. By the new class of merit functions, the SOCCP can be reformulated as an unconstrained minimization problem. The new class of merit functions is shown to possess some favorable properties. In particular, it provides a global error bound ifFandGhave the joint uniform CartesianP-property. And it has bounded level sets under a weaker condition than the most available conditions. Some preliminary numerical results for solving the SOCCPs show the effectiveness of the merit function method via the new class of merit functions.

Convergence of the Restricted Nelder--Mead Algorithm in Two Dimensions

The Nelder-Mead algorithm, a longstanding direct search method for unconstrained optimization published in 1965, is designed to minimize a scalar-valued function f of n real variables using only function values, without any derivative information. Each Nelder-Mead iteration is associated with a nondegenerate simplex defined by n+1 vertices and their function values; a typical iteration produces a new simplex by replacing the worst vertex by a new point. Despite the method's widespread use, theoretical results have been limited: for strictly convex objective functions of one variable with bounded level sets, the algorithm always converges to the minimizer; for such functions of two variables, the diameter of the simplex converges to zero, but examples constructed by McKinnon show that the algorithm may converge to a nonminimizing point. This paper considers the restricted Nelder-Mead algorithm, a variant that does not allow expansion steps. In two dimensions we show that, for any nondegenerate starting simplex and any twice-continuously differentiable function with positive definite Hessian and bounded level sets, the algorithm always converges to the minimizer. The proof is based on treating the method as a discrete dynamical system, and relies on several techniques that are non-standard in convergence proofs for unconstrained optimization.

On the Lorentz Cone Complementarity Problems in Infinite-Dimensional Real Hilbert Space

In this article, we consider the Lorentz cone complementarity problems in infinite-dimensional real Hilbert space. We establish several results that are standard and important when dealing with complementarity problems. These include proving the same growth of the Fishcher–Burmeister merit function and the natural residual merit function, investigating property of bounded level sets under mild conditions via different merit functions, and providing global error bounds through the proposed merit functions. Such results are helpful for further designing solution methods for the Lorentz cone complementarity problems in Hilbert space.

Two Classes of Merit Functions for Infinite-Dimensional Second Order Complimentary Problems

In this article, we extend two classes of merit functions for the second-order complementarity problem (SOCP) to infinite-dimensional SOCP. These two classes of merit functions include several popular merit functions, which are used in nonlinear complementarity problem, (NCP)/(SDCP) semidefinite complementarity problem, and SOCP, as special cases. We give conditions under which the infinite-dimensional SOCP has a unique solution and show that all these merit functions provide an error bound for infinite-dimensional SOCP and have bounded level sets. These results are very useful for designing solution methods for infinite-dimensional SOCP.

A new class of penalized NCP-functions and its properties

In this paper, we consider a class of penalized NCP-functions, which includes several existing well-known NCP-functions as special cases. The merit function induced by this class of NCP-functions is shown to have bounded level sets and provide error bounds under mild conditions. A derivative free algorithm is also proposed, its global convergence is proved and numerical performance compared with those based on some existing NCP-functions is reported.

The penalized Fischer-Burmeister SOC complementarity function

In this paper, we study the properties of the penalized Fischer-Burmeister (FB) second-order cone (SOC) complementarity function. We show that the function possesses similar desirable properties of the FB SOC complementarity function for local convergence; for example, with the function the second-order cone complementarity problem (SOCCP) can be reformulated as a (strongly) semismooth system of equations, and the corresponding nonsmooth Newton method has local quadratic convergence without strict complementarity of solutions. In addition, the penalized FB merit function has bounded level sets under a rather weak condition which can be satisfied by strictly feasible monotone SOCCPs or SOCCPs with the Cartesian R 01-property, although it is not continuously differentiable. Numerical results are included to illustrate the theoretical considerations.

A one-parametric class of merit functions for the symmetric cone complementarity problem

In this paper, we extend the one-parametric class of merit functions proposed by Kanzow and Kleinmichel [C. Kanzow, H. Kleinmichel, A new class of semismooth Newton-type methods for nonlinear complementarity problems, Comput. Optim. Appl. 11 (1998) 227–251] for the nonnegative orthant complementarity problem to the general symmetric cone complementarity problem (SCCP). We show that the class of merit functions is continuously differentiable everywhere and has a globally Lipschitz continuous gradient mapping. From this, we particularly obtain the smoothness of the Fischer–Burmeister merit function associated with symmetric cones and the Lipschitz continuity of its gradient. In addition, we also consider a regularized formulation for the class of merit functions which is actually an extension of one of the NCP function classes studied by [C. Kanzow, Y. Yamashita, M. Fukushima, New NCP functions and their properties, J. Optim. Theory Appl. 97 (1997) 115–135] to the SCCP. By exploiting the Cartesian P -properties for a nonlinear transformation, we show that the class of regularized merit functions provides a global error bound for the solution of the SCCP, and moreover, has bounded level sets under a rather weak condition which can be satisfied by the monotone SCCP with a strictly feasible point or the SCCP with the joint Cartesian R 02 -property. All of these results generalize some recent important works in [J.-S. Chen, P. Tseng, An unconstrained smooth minimization reformulation of the second-order cone complementarity problem, Math. Program. 104 (2005) 293–327; C.-K. Sim, J. Sun, D. Ralph, A note on the Lipschitz continuity of the gradient of the squared norm of the matrix-valued Fischer–Burmeister function, Math. Program. 107 (2006) 547–553; P. Tseng, Merit function for semidefinite complementarity problems, Math. Program. 83 (1998) 159–185] under a unified framework.

Regularized gap function as penalty term for constrained minimization problems

By using the regularized gap function for variational inequalities, Li and Peng introduced a new penalty function P α ( x ) for the problem of minimizing a twice continuously differentiable function in closed convex subset of the n-dimensional space R n . Under certain assumptions, they proved that the original constrained minimization problem is equivalent to unconstrained minimization of P α ( x ) . The main purpose of this paper is to give an in-depth study of those properties of the objective function that can be extended from the feasible set to the whole R n by P α ( x ) . For example, it is proved that the objective function has bounded level sets (or is strongly coercive) on the feasible set if and only if P α ( x ) has bounded level sets (or is strongly coercive) on R n . However, the convexity of the objective function does not imply the convexity of P α ( x ) when the objective function is not quadratic, no matter how small α is. Instead, the convexity of the objective function on the feasible set only implies the invexity of P α ( x ) on R n . Moreover, a characterization for the invexity of P α ( x ) is also given.

Spectral Finite-Element Methods for Parametric Constrained Optimization Problems

We present a method to approximate the solution mapping of parametric constrained optimization problems. The approximation, which is of the spectral finite element type, is represented as a linear combination of orthogonal polynomials. Its coefficients are determined by solving an appropriate finite-dimensional constrained optimization problem. We show that, under certain conditions, the latter problem is solvable because it is feasible for a sufficiently large degree of the polynomial approximation and has an objective function with bounded level sets. In addition, the solutions of the finite-dimensional problems converge for an increasing degree of the polynomials considered, provided that the solutions exhibit a sufficiently large and uniform degree of smoothness. Our approach solves, in the case of optimization problems with uncertain parameters, the most computationally intensive part of stochastic finite-element approaches. We demonstrate that our framework is applicable to parametric eigenvalue problems.

Sequential unconstrained minimization algorithms for constrained optimization

The problem of minimizing a function f(x):RJ → R, subject to constraints on the vector variable x, occurs frequently in inverse problems. Even without constraints, finding a minimizer of f(x) may require iterative methods. We consider here a general class of iterative algorithms that find a solution to the constrained minimization problem as the limit of a sequence of vectors, each solving an unconstrained minimization problem. Our sequential unconstrained minimization algorithm (SUMMA) is an iterative procedure for constrained minimization. At the kth step we minimize the function to obtain xk. The auxiliary functions gk(x):D ⊆ RJ → R+ are nonnegative on the set D, each xk is assumed to lie within D, and the objective is to minimize the continuous function f:RJ → R over x in the set , the closure of D. We assume that such minimizers exist, and denote one such by . We assume that the functions gk(x) satisfy the inequalities for k = 2, 3, …. Using this assumption, we show that the sequence {f(xk)} is decreasing and converges to . If the restriction of f(x) to D has bounded level sets, which happens if is unique and f(x) is closed, proper and convex, then the sequence {xk} is bounded, and , for any cluster point x*. Therefore, if is unique, and . When is not unique, convergence can still be obtained, in particular cases. The SUMMA includes, as particular cases, the well-known barrier- and penalty-function methods, the simultaneous multiplicative algebraic reconstruction technique (SMART), the proximal minimization algorithm of Censor and Zenios, the entropic proximal methods of Teboulle, as well as certain cases of gradient descent and the Newton–Raphson method. The proof techniques used for SUMMA can be extended to obtain related results for the induced proximal distance method of Auslander and Teboulle.

Conditions for Error Bounds and Bounded Level Sets of Some Merit Functions for the Second-Order Cone Complementarity Problem

Recently this author studied several merit functions systematically for the second-order cone complementarity problem. These merit functions were shown to enjoy some favorable properties, to provide error bounds under the condition of strong monotonicity, and to have bounded level sets under the conditions of monotonicity as well as strict feasibility. In this paper, we weaken the condition of strong monotonicity to the so-called uniform P ∗ -property, which is a new concept recently developed for linear and nonlinear transformations on Euclidean Jordan al- gebra. Moreover, we replace the monotonicity and strict feasibility by the so-called R01 or R02-functions to keep the property of bounded level sets.

Two classes of merit functions for the second-order cone complementarity problem

Recently Tseng (Math Program 83:159–185, 1998) extended a class of merit functions, proposed by Luo and Tseng (A new class of merit functions for the nonlinear complementarity problem, in Complementarity and Variational Problems: State of the Art, pp. 204–225, 1997), for the nonlinear complementarity problem (NCP) to the semidefinite complementarity problem (SDCP) and showed several related properties. In this paper, we extend this class of merit functions to the second-order cone complementarity problem (SOCCP) and show analogous properties as in NCP and SDCP cases. In addition, we study another class of merit functions which are based on a slight modification of the aforementioned class of merit functions. Both classes of merit functions provide an error bound for the SOCCP and have bounded level sets.

Global convergence of modified HS conjugate gradient method

In this paper, HS conjugate gradient method for minimizing a continuously differentiable functionf onR n is modified to have global convergence property. Firstly, it is shown that, using reverse modulus of continuity function and forcing function, the new method for solving unconstrained optimization can work for a continuously differentiable function with Curry-Altman’s step size rule and a bounded level set. Secondly, by using comparing technique, some general convergence properties of the new method with Armijo step size rule are established. Numerical results show that the new algorithms are efficient.

Global Convergence of an Elastic Mode Approach for a Class of Mathematical Programs with Complementarity Constraints

We prove that any accumulation point of an elastic mode approach, that approximately solves the relaxed subproblems, is a C-stationary point of the problem of optimizing a parametric mixed P variational inequality. If, in addition, the accumulation point satisfies the MPCC-LICQ constraint qualification, and if the solutions of the subproblem satisfy approximate second-order sufficient conditions, then the limiting point is an M-stationary point. Moreover, if the accumulation point satisfies the upper-level strict complementarity condition, the accumulation point will be a strongly stationary point. If we assume that the penalty function associated with the feasible set of the mathematical program with complementarity constraints has bounded level sets, and if the objective function is bounded below, we show that the algorithm will produce bounded iterates and will therefore have at least one accumulation point. We prove that the obstacle problem satisfies our assumptions for both a rigid and a deformable obstacle. The theoretical conclusions are validated by several numerical examples.

A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization

Let f be a continuous function on $\Rl^n$, and suppose f is continuously differentiable on an open dense subset. Such functions arise in many applications, and very often minimizers are points at which f is not differentiable. Of particular interest is the case where f is not convex, and perhaps not even locally Lipschitz, but is a function whose gradient is easily computed where it is defined. We present a practical, robust algorithm to locally minimize such functions, based on gradient sampling. No subgradient information is required by the algorithm. When f is locally Lipschitz and has bounded level sets, and the sampling radius $\eps$ is fixed, we show that, with probability 1, the algorithm generates a sequence with a cluster point that is Clarke $\eps$-stationary. Furthermore, we show that if f has a unique Clarke stationary point $\bar x$, then the set of all cluster points generated by the algorithm converges to $\bar x$ as $\eps$ is reduced to zero. Numerical results are presented demonstrating the robustness of the algorithm and its applicability in a wide variety of contexts, including cases where f is not locally Lipschitz at minimizers. We report approximate local minimizers for functions in the applications literature which have not, to our knowledge, been obtained previously. When the termination criteria of the algorithm are satisfied, a precise statement about nearness to Clarke $\eps$-stationarity is available. A MATLAB implementation of the algorithm is posted at http://www.cs.nyu.edu/overton/papers/gradsamp/alg.

The bounded smooth reformulation and a trust region algorithm for semidefinite complementarity problems

In this paper, we present a new merit function for the semidefinite complementarity problem (SDCP) by extending the bounded smooth reformulation for variational inequality problems. We prove that the merit function has a bounded level set and any stationary point of it is a global minimizer without the assumption of monotonicity. Moreover, we present a trust region algorithm for solving the minimization problem with semidefinite constraints. The trust region subproblem is solved by the truncated conjugate gradient method and the global convergence is established even without requiring the existence of an accumulation point of the generated sequence.