General Constrained Problems Research Articles

Scaled constraint qualifications for generalized equation constrained problems and application to nonsmooth mathematical programs with equilibrium constraints

In this paper, the notion of graphical derivatives is applied to define a new class of several well-known constraint qualifications for a nonconvex multifunction M at a point of its graph. This class is called as “scaled constraint qualifications”. The reason of this terminology is that these conditions ensure the existence of bounded KKT multiplier vectors with a proper upper bound. The relations between these constraint qualifications and stability properties of M are also investigated. New sharp necessary optimality conditions with bounded multiplier vectors are derived for an optimization problem with a generalized equation constraint. The results are adapted to nonsmooth general constrained problems and nonsmooth mathematical programs with equilibrium constraints.

A hybrid approach to constrained global optimization

In this paper, we propose a novel hybrid global optimization method to solve constrained optimization problems. An exact penalty function is first applied to approximate the original constrained optimization problem by a sequence of optimization problems with bound constraints. To solve each of these box constrained optimization problems, two hybrid methods are introduced, where two different strategies are used to combine limited memory BFGS (L-BFGS) with Greedy Diffusion Search (GDS). The convergence issue of the two hybrid methods is addressed. To evaluate the effectiveness of the proposed algorithm, 18 box constrained and 4 general constrained problems from the literature are tested. Numerical results obtained show that our proposed hybrid algorithm is more effective in obtaining more accurate solutions than those compared to.

A trust-region derivative-free algorithm for constrained optimization

We propose a trust-region algorithm for constrained optimization problems in which the derivatives of the objective function are not available. In each iteration, the objective function is approximated by a model obtained by quadratic interpolation, which is then minimized within the intersection of the feasible set with the trust region. Since the constraints are handled in the trust-region subproblems, all the iterates are feasible even if some interpolation points are not. The rules for constructing and updating the quadratic model and the interpolation set use ideas from the BOBYQA software, a largely used algorithm for box-constrained problems. The subproblems are solved by ALGENCAN, a competitive implementation of an Augmented Lagrangian approach for general-constrained problems. Some numerical results for the Hock–Schittkowski collection are presented, followed by a performance comparison between our proposal and three derivative-free algorithms found in the literature.

Mesh adaptive direct search algorithms for mixed variable optimization

This paper introduces a new derivative-free class of mesh adaptive direct search (MADS) algorithms for solving constrained mixed variable optimization problems, in which the variables may be continuous or categorical. This new class of algorithms, called mixed variable MADS (MV-MADS), generalizes both mixed variable pattern search (MVPS) algorithms for linearly constrained mixed variable problems and MADS algorithms for general constrained problems with only continuous variables. The convergence analysis, which makes use of the Clarke nonsmooth calculus, similarly generalizes the existing theory for both MVPS and MADS algorithms, and reasonable conditions are established for ensuring convergence of a subsequence of iterates to a suitably defined stationary point in the nonsmooth and mixed variable sense.

Efficient constrained optimization: from the deterministic past to the stochastic future

In our deterministic past is a complete, efficient theory to solve very general constrained problems in the calculus of variations/optimal control theory, given by the first author. This is done in two major steps. The first step involves numerical methods for calculus of variation problems which are efficient to apply and have a guaranteed, maximal, pointwise, a priori error of O ( h 2 ) , where h is the step size between node points. The second step is to properly reformulate/recast general constrained problems in the calculus of variations/optimal control theory as calculus of variations problems with the right boundary conditions. This theory includes inequality constraints and allows for the analytic solution of simpler problems and the numerical solution of more complex problems. For our stochastic future, we introduce a new stochastic theory of the calculus of variations/optimal control theory by using a restricted class of variations motivated by classical, deterministic methods. We consider boundary value problems for objective functionals, without averaging, and obtain necessary conditions in the form of stochastic differential systems with given boundary conditions. We emphasize that, as in the deterministic past, these stochastic systems come from critical point conditions, but unlike current stochastic methods, they yield random objective values. In general, these systems admit anticipating solutions but they can often also be reformulated as coupled forward–backward systems with non-anticipating solutions. Thus, when we, then, average the functionals as is done in the initial formulation of current stochastic control theory, our methods with anticipating solutions sometimes yield a smaller minimum for expected cost than classical stochastic methods while duplicating the classical results for the associated non-anticipatory problem. We conjecture that (at least) in the case of linear trajectories and quadratic cost, coupled forward–backward systems with non-anticipating solutions will always yield the classical stochastic averaged solution. Finally, we note that a major benefit of our variational approach is that we expect to duplicate the complete theory of solutions obtained by the first author for the deterministic case. In particular, we will obtain efficient methods for closed form solution for simpler problems and/or a new, efficient theory of numerical solutions, with a maximal, pointwise, a priori error estimate of O ( h 3 / 2 ) , for more difficult stochastic optimization problems. This includes a theory of constraint optimization with general equality and inequality constraints. An additional benefit of our approach is that major problems in the deterministic calculus of variations/optimal control theory can be compared to their associated stochastic problems since each can be efficiently solved by our methods.

O(h2 ) Global Pointwise Algorithms for Delay Quadratic Problems in the Calculus of Variations

The main purpose of this paper is to give numerical algorithms and the error analysis for delay quadratic problems in the calculus of variations. These methods are new, efficient, and accurate and have a global a priori error of O(h2 ), where h is the distance between any two successive node points. We also derive the results for the general, numerical, delay problem, but we focus on proving our results in the simpler quadratic case since the extra technical numerical details have been given previously by the second author. In addition, the authors have previously shown how to reformulate general delay constrained problems in optimal control theory/constrained calculus of variations as unconstrained delay problems. Thus our numerical results and methods will hold for these general constrained problems also. Finally, we note that our algorithm, which solves the stationary condition(s) numerically, avoids the more difficult problems of piecing the solution of second order equations together and requires less smoothness in the solution. Thus we replace difficult second order boundary value problems with the easier task of approximating definite integrals involving first order derivatives.

The Complete Solution for Constrained Delay Problems in the Calculus of Variations by Unconstrained Methods

This paper has two main ideas. The first idea is that general constrained problems with delay in the calculus of variations can be associated with unconstrained calculus of variations problems by using multipliers. This allows us to obtain a true Lagrange multiplier rule where the original variables, the multipliers, and the slack variables for inequality constraints can be determined. The second idea is that critical point solutions to the delay problem, which include the determination of the multipliers, immediately follow from Euler–Lagrange equations for the unconstrained problem. This critical point solution is a necessary condition for the original problem. In later work we will show that these methods can be combined with previous methods by the first author to obtain efficient and accurate numerical solutions to the original problem where no such general numerical methods currently seem to exist. This seems to be an important development since the lack of such methods has hindered the usefulness of the theory.

A Bliss-Type Multiplier Rule for Constrained Variational Problems with Time Delay

This paper gives a Bliss-type multiplier rule for general constrained variational problems with time delay. Our formulation allows the time delay terms and their derivatives to be present in both the objective functional and the equality and inequality constraint equations. Our major results include a Lagrange Multiplier Rule involving Euler–Lagrange equations for delay differential equations and transversality conditions. Of special note is that these results will be used in later work by the authors to obtain new analytical techniques to solve general constrained problems involving delay differential equations for the Calculus of Variations and for Optimal Control Theory. They will also lead to new general, accurate, and efficient numerical methods to solve these important problems where no such methods exist at this time. Thus, for the first time, we will be able to solve this important class of problems obtaining analytic solutions for simpler problems and accurate numerical results for more complex problems when such a solution exists.

Global optimization methods for engineering applications: A review

A review of the methods for global optimization reveals that most methods have been developed for unconstrained problems. They need to be extended to general constrained problems because most of the engineering applications have constraints. Some of the methods can be easily extended while others need further work. It is also possible to transform a constrained problem to an unconstrained one by using penalty or augmented Lagrangian methods and solve the problem that way. Some of the global optimization methods find all the local minimum points while others find only a few of them. In any case, all the methods require a very large number of calculations. Therefore, the computational effort to obtain a global solution is generally substantial. The methods for global optimization can be divided into two broad categories: deterministic and stochastic. Some deterministic methods are based on certain assumptions on the cost function that are not easy to check. These methods are not very useful since they are not applicable to general problems. Other deterministic methods are based on certain heuristics which may not lead to the true global solution. Several stochastic methods have been developed as some variation of the pure random search. Some methods are useful for only discrete optimization problems while others can be used for both discrete and continuous problems. Main characteristics of each method are identified and discussed. The selection of a method for a particular application depends on several attributes, such as types of design variables, whether or not all local minima are desired, and availability of gradients of all the functions.

A reduced Hessian method for constrained optimization

Reduced Hessian methods have been shown to be successful for equality constrained problems. However there are few results on reduced Hessian methods for general constrained problems. In this paper we propose a method for general constrained problems, based on Byrd and Schnabel's basis-independent algorithm. It can be regarded as a smooth extension of the standard reduced Hessian Method.

Discrete Variable Methods for the m–Dependent Variable Nonlinear, Extremal Problem in the Calculus of Variations II

In a previous work, Gregory and Wang gave efficient numerical methods to obtain global, pointwise, $O(h^2 )$ a priori error estimates for the basic fixed end point problem in the calculus of variations. This paper extends these numerical results to the more general situation involving a variety of variable end point problems in the calculus of variations. In particular, this paper gives new numerical transversality conditions (which correspond to classical transversality conditions) and shows that these numerical conditions along with the basic algorithm of the previous work lead to efficient numerical methods to obtain global, pointwise, $O(h^2 )$ a priori error estimates for a variety of variable end point problems. A nontrivial example is given to illustrate this theory. In addition, these results are used to obtain numerical solutions for constrained optimal control problems. To do this, this paper uses the result of a companion paper of the authors which shows that general constrained problems in the calculus of variations or optimal control theory can be reformulated as unconstrained problems in the Calculus of Variations and thus solved by the numerical methods of this paper. Thus, this paper obtains general, efficient and accurate methods for these constrained optimization problems where no such method currently seems to exist.

Avoiding the Maratos Effect by Means of a Nonmonotone Line Search I. General Constrained Problems

An essential condition for quasi-Newton optimization methods to converge super-linearly is that a full step of one be taken close to the solution. It is well known that, when dealing with constrained optimization problems, line search schemes ensuring global convergence of such methods may prevent this from occurring (the so-called “Maratos effect”). Two types of techniques have been used to circumvent this difficulty. In the watchdog technique, the full step of one is occasionally accepted even when the line search criterion is violated; subsequent backtracking is used if global convergence appears to be lost. In a “bending” technique proposed by Mayne and Polak [Math Programming Stud., 16 (1982), pp. 45–61], backtracking is avoided by performing a search along an arc whose construction requires evaluation of constraint functions at an auxiliary point; along this arc, the full step of one is accepted close to a solution. The main idea in the present paper is to combine Mayne and Polak’s technique with a nonmonotone line search proposed by Grippo, Lampariello, and Lucidi [SIAM J. Numer. Anal., 23 (1986), pp. 707–716] in the context of unconstrained optimization, in such a way that, after a few iterations, function evaluations are no longer performed at auxiliary points. In a companion paper (part II), it is shown that a refinement of this scheme can be used in the context of recently proposed SQP-based methods generating feasible iterates.