Exact penalty functions in nonlinear programming

On an exact penalty function method for nonlinear mixed discrete programming problems and its applications in search engine advertising problems

In this article, we develop a general theory of exact parametric penalty functions for constrained optimization problems. The main advantage of the method of parametric penalty functions is the fact that a parametric penalty function can be both smooth and exact unlike the standard (i.e. non-parametric) exact penalty functions that are always nonsmooth. We obtain several necessary and/or sufficient conditions for the exactness of parametric penalty functions, and for the zero duality gap property to hold true for these functions. We also prove some convergence results for the method of parametric penalty functions, and derive necessary and sufficient conditions for a parametric penalty function to not have any stationary points outside the set of feasible points of the constrained optimization problem under consideration. In the second part of the paper, we apply the general theory of exact parametric penalty functions to a class of parametric penalty functions introduced by Huyer and Neumaier, and to smoothing approximations of nonsmooth exact penalty functions. The general approach adopted in this article allowed us to unify and significantly sharpen many existing results on parametric penalty functions.

A new exact exponential penalty function method and nonconvex mathematical programming

Computational experience with an exact penalty function technique (EPT)

Many optimization problems in computational science and engineering (CS&E) are characterized by expensive objective and/or constraint function evaluations paired with a lack of derivative information. Direct search methods such as generating set search (GSS) are well understood and efficient for derivative-free optimization of unconstrained and linearly-constrained problems. This paper addresses the more difficult problem of general nonlinear programming where derivatives for objective or constraint functions are unavailable, which is the case for many CS&E applications. We focus on penalty methods that use GSS to solve the linearly-constrained problems, comparing different penalty functions. A classical choice for penalizing constraint violations is {ell}{sub 2}{sup 2}, the squared {ell}{sub 2} norm, which has advantages for derivative-based optimization methods. In our numerical tests, however, we show that exact penalty functions based on the {ell}{sub 1}, {ell}{sub 2}, and {ell}{sub {infinity}} norms converge to good approximate solutions more quickly and thus are attractive alternatives. Unfortunately, exact penalty functions are discontinuous and consequently introduce theoretical problems that degrade the final solution accuracy, so we also consider smoothed variants. Smoothed-exact penalty functions are theoretically attractive because they retain the differentiability of the original problem. Numerically, they are a compromise between exact and {ell}{sub 2}{sup 2}, i.e., they converge to a good solution somewhat quickly without sacrificing much solution accuracy. Moreover, the smoothing is parameterized and can potentially be adjusted to balance the two considerations. Since many CS&E optimization problems are characterized by expensive function evaluations, reducing the number of function evaluations is paramount, and the results of this paper show that exact and smoothed-exact penalty functions are well-suited to this task.

The purpose of the study was to investigate how effectively the penalty function methods are able to solve constrained optimization problems. The approach in these methods is to transform the constrained optimization problem into an equivalent unconstrained problem and solved using one of the algorithms for unconstrained optimization problems. Algorithms and matrix laboratory (MATLAB) codes are developed using Powell’s method for unconstrained optimization problems and then problems that have appeared frequently in the optimization literature, which have been solved using different techniques compared with other algorithms. It is found out in the research that the sequential transformation methods converge to at least to a local minimum in most cases without the need for the convexity assumptions and with no requirement for differentiability of the objective and constraint functions. For problems of non-convex functions it is recommended to solve the problem with different starting points, penalty parameters and penalty multipliers and take the best solution. But on the other hand for the exact penalty methods convexity assumptions and second-order sufficiency conditions for a local minimum is needed for the solution of unconstrained optimization problem to converge to the solution of the original problem with a finite penalty parameter. In these methods a single application of an unconstrained minimization technique as against the sequential methods is used to solve the constrained optimization problem. Key words: Penalty function, penalty parameter, augmented lagrangian penalty function, exact penalty function, unconstrained representation of the primal problem.

In this article, we consider a lower order penalty function and its ε-smoothing for an inequality constrained nonlinear programming problem. It is shown that any strict local minimum satisfying the second-order sufficiency condition for the original problem is a strict local minimum of the lower order penalty function with any positive penalty parameter. By using an ε-smoothing approximation to the lower order penalty function, we get a modified smooth global exact penalty function under mild assumptions.

In this paper we consider inequality constrained nonlinear optimization problems where the first order derivatives of the objective function and the constraints cannot be used. Our starting point is the possibility to transform the original constrained problem into an unconstrained or linearly constrained minimization of a nonsmooth exact penalty function. This approach shows two main difficulties: the first one is the nonsmoothness of this class of exact penalty functions which may cause derivative-free codes to converge to nonstationary points of the problem; the second one is the fact that the equivalence between stationary points of the constrained problem and those of the exact penalty function can only be stated when the penalty parameter is smaller than a threshold value which is not known a priori. In this paper we propose a derivative-free algorithm which overcomes the preceding difficulties and produces a sequence of points that admits a subsequence converging to a Karush–Kuhn–Tucker point of the constrained problem. In particular the proposed algorithm is based on a smoothing of the nondifferentiable exact penalty function and includes an updating rule which, after at most a finite number of updates, is able to determine a “right value” for the penalty parameter. Furthermore we present the results obtained on a real world problem concerning the estimation of parameters in an insulin-glucose model of the human body.

A new class of smooth exact penalty functions was recently introduced by Huyer and Neumaier. In this paper, we prove that the new smooth penalty function for a constrained optimization problem is exact if and only if the standard nonsmooth penalty function for this problem is exact. We also provide some estimates of the exact penalty parameter of the smooth penalty function, and, in particular, show that it asymptotically behaves as the square of the exact penalty parameter of the standard $\ell_1$ penalty function. We briefly discuss a simple way to reduce the exact penalty parameter of the smooth penalty function, and study the effect of nonlinear terms on the exactness of this function.

In this paper, we study a new exact and smooth penalty function forsemi-infinite programming problems with continuous inequalityconstraints. Through this exact penalty function, we can transform asemi-infinite programming problem into an unconstrained optimizationproblem. We find that, under some reasonable conditions when thepenalty parameter is sufficiently large, the local minimizer of thispenalty function is the local minimizer of the primal problem.Moreover, under some mild assumptions, the local exactness propertyis explored. The numerical results demonstrate that it is aneffective and promising approach for solving constrainedsemi-infinite programming problems.

In this paper we extend the theory of exact penalty functions for nonlinear programs whose objective functions and equality and inequality constraints are locally Lipschitz; arbitrary simple constraints are also allowed. Assuming a weak stability condition, we show that for all sufficiently large penalty parameter values an isolated local minimum of the nonlinear program is also an isolated local minimum of the exact penalty function. A tight lower bound on the parameter value is provided when certain first order sufficiency conditions are satisfied. We apply these results to unify and extend some results for convex programming. Since several effective algorithms for solving nonlinear programs with differentiable functions rely on exact penalty functions, our results provide a framework for extending these algorithms to problems with locally Lipschitz functions.

In this paper, the classical exact absolute value penalty function method is used for solving a new class of nonconvex nonsmooth optimization problems. Nonconvex nondifferentiable optimization problems with both inequality and equality constraints are considered here, in which not all functions constituting them have the fundamental property of convex functions and most classes of generalized convex functions—namely, a stationary point of such a function is its global minimum. It is proved for such nonconvex optimization problems that there exists a finite threshold of penalty parameters equal to the largest absolute value of a Lagrange multiplier such that, for every penalty parameter exceeding this lower bound, there is the equivalence between an optimal solution in the original constrained minimization problem with ‐invex functions and a minimizer in its associated penalized optimization problem with the exact l1 penalty function. Further, under nondifferentiable ‐invexity assumptions, a characterization of a saddle point of the Lagrange function, defined for the considered constrained optimization problem in terms of minimizers of its associated exact penalized optimization problem with the exact l1 penalty function, is presented.

Exactness of penalization for exact minimax penalty function method in nonconvex programming

A new characterization of the exact minimax penalty function method is presented. The exactness of the penalization for the exact minimax penalty function method is analyzed in the context of saddle point criteria of the Lagrange function in the nonconvex differentiable optimization problem with both inequality and equality constraints. Thus, new conditions for the exactness of the exact minimax penalty function method are established under assumption that the functions constituting considered constrained optimization problem are invex with respect to the same function $\\eta $ (exception with those equality constraints for which the associated Lagrange multipliers are negative - these functions should be assumed to be incave with respect to the same function $\\eta $). The threshold of the penalty parameter is given such that, for all penalty parameters exceeding this treshold, the equivalence holds between a saddle point of the Lagrange function in the considered constrained extremum problem and a minimizer in its associated penalized optimization problem with the exact minimax penalty function.

Automatic decrease of the penalty parameter in exact penalty function methods