Extended duality for nonlinear programming

Duality is an important notion for constrained optimization which provides a theoretical foundation for a number of constraint decomposition schemes such as separable programming and for deriving lower bounds in space decomposition algorithms such as branch and bound. However, the conventional duality theory has the fundamental limit that it leads to duality gaps for nonconvex optimization problems, especially discrete and mixed-integer problems where the feasible sets are nonconvex. In this paper, we propose a novel extended duality theory for nonlinear optimization that overcomes some limitations of previous dual methods. Based on a new dual function, the extended duality theory leads to zero duality gap for general nonconvex problems defined in discrete, continuous, and mixed-integer spaces under mild conditions.

On an exact penalty function method for nonlinear mixed discrete programming problems and its applications in search engine advertising problems

Previous article Next article Extensions of Lagrange Multipliers in Nonlinear ProgrammingF. J. GouldF. J. Gouldhttps://doi.org/10.1137/0117120PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAbout[1] K. J. Arrow, , Leonid Hurwicz and , Hirofumi Uzawa, Studies in linear and non-linear programming, With contributions by H. B. Chenery, S. M. Johnson, S. Karlin, T. Marschak, R. M. Solow. Stanford Mathematical Studies in the Social Sciences, vol. II, Stanford University Press, Stanford, Calif., 1958vii+229, Chap. 3 MR0108399 (21:7115) 0091.16002 Google Scholar[2] R. Brooks and , A. Geoffrion, Finding Everett's Lagrange multipliers by linear programming, Operations Res., 14 (1966), 1149–1153 MR0210449 (35:1342) CrossrefISIGoogle Scholar[3] G. Eppen and , F. J. Gould, A Lagrangian application to production models, Operations Res., 16 (1968), 819–829 CrossrefISIGoogle Scholar[4] Hugh Everett, III, Generalized Lagrange multiplier method for solving problems of optimum allocation of resources, Operations Res., 11 (1963), 399–417 MR0152360 (27:2340) 0113.14202 CrossrefISIGoogle Scholar[5] A. V. Fiacco and , G. Mccormick, The sequential unconstrained minimization technique for nonlinear programming, A primal-dual method, Management Sci., 10 (1964), 360–366 CrossrefISIGoogle Scholar[6] A. V. Fiacco and , G. P. McCormick, Computational algorithm for the sequential unconstrained minimization technique for nonlinear programming, Management Sci., 10 (1964), 601–617 CrossrefISIGoogle Scholar[7] F. J. Gould, A class of inside-out algorithms for general programs, Management Sci., 16 (1969/70), 350–356 MR0302193 (46:1346) 0191.48902 CrossrefISIGoogle Scholar[8] F. J. Gould, Nonlinear duality theorems, Rep., 6817, Center for Mathematical Studies in Business and Economics, University of Chicago, Chicago, 1968 Google Scholar[9] S. Kaplan, Solution of the Lone-Savage and similar integer programming problems by the generalized Lagrange multiplier method, Operations Res., 14 (1966), 1130–1136 0158.38406 CrossrefISIGoogle Scholar[10] H. W. Kuhn and , A. W. Tucker, Nonlinear programming, Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, 1950, University of California Press, Berkeley and Los Angeles, 1951, 481–492 MR0047303 (13,855f) 0044.05903 Google Scholar[11] G. Pugh, Lagrange multipliers and the optimal allocation of defense resources, Operations Res., 12 (1964), 543–567 0129.34207 CrossrefISIGoogle Scholar[12] R. T. Rockafellar, Duality in nonlinear programmingMathematics of the Decision Sciences, Part 1 (Seminar, Stanford, Calif., 1967), Amer. Math. Soc., Providence, R.I., 1968, 401–422, Lectures in Applied Mathematics, vol. II MR0242491 (39:3822) 0231.90037 Google Scholar[13] Philip Wolfe, R. L. Graves and , P. Wolfe, Methods of nonlinear programmingRecent advances in mathematical programming, McGraw-Hill, New York, 1963, 67–86 MR0155683 (27:5617) 0225.90042 Google Scholar[14] Willard I. Zangwill, Non-linear programming via penalty functions, Management Sci., 13 (1967), 344–358 MR0252040 (40:5265) 0171.18202 CrossrefGoogle Scholar Previous article Next article FiguresRelatedReferencesCited ByDetails Economics of Externalities: An OverviewHandbook of Production Economics | 3 June 2022 Cross Ref Evolutionary economics under nonconvexity and externalitiesOxford Economic Papers, Vol. 73, No. 3 | 26 March 2021 Cross Ref Parametric Stochastic Programming with One Chance Constraint: Gaining Insights from Response Space AnalysisHarvey J. Greenberg | 30 July 2020 Cross Ref Economics of Externalities: An OverviewHandbook of Production Economics | 2 June 2020 Cross Ref Generalized Lagrange Multiplier Method and KKT Conditions With an Application to Distributed OptimizationIEEE Transactions on Circuits and Systems II: Express Briefs, Vol. 66, No. 2 | 1 Feb 2019 Cross Ref Glossary of Mathematical Optimization TerminologyMathematical Optimization Terminology | 1 Jan 2018 Cross Ref A Maximum Entropy Method for a Robust Portfolio ProblemEntropy, Vol. 16, No. 6 | 20 June 2014 Cross Ref A theoretical and empirical investigation on the Lagrangian capacities of the 0-1 multidimensional knapsack problemEuropean Journal of Operational Research, Vol. 218, No. 2 | 1 Apr 2012 Cross Ref A new duality theory for mathematical programmingOptimization, Vol. 60, No. 8-9 | 1 Aug 2011 Cross Ref Effective method for constrained minimum — reverse bridge theoremComputers & Mathematics with Applications, Vol. 56, No. 10 | 1 Nov 2008 Cross Ref DualityMathematics of Optimization | 1 Jan 2004 Cross Ref LiteraturverzeichnisOperations Research | 1 Jan 2001 Cross Ref Johri's general dual, the Lagrangian dual, and the surrogate dualEuropean Journal of Operational Research, Vol. 117, No. 1 | 1 Aug 1999 Cross Ref Maximum-Entropy Approach in Linear ProgramsOPSEARCH, Vol. 36, No. 2 | 14 December 2017 Cross Ref CONSTRAINED NON-LINEAR PROGRAMMING: A MINIMUM CROSS-ENTROPY ALGORITHMEngineering Optimization, Vol. 31, No. 4 | 1 Mar 1999 Cross Ref LiteraturverzeichnisOperations Research | 1 Jan 1998 Cross Ref THE USE OF ENTROPY IN HYDROLOGY AND WATER RESOURCESHydrological Processes, Vol. 11, No. 6 | 1 May 1997 Cross Ref Duality in nonlinear multiobjective programming using augmented Lagrangian functionsJournal of Optimization Theory and Applications, Vol. 88, No. 3 | 1 Mar 1996 Cross Ref Additively separable duality theoryJournal of Optimization Theory and Applications, Vol. 88, No. 2 | 1 Feb 1996 Cross Ref Vector optimization and generalized Lagrangian dualityAnnals of Operations Research, Vol. 51, No. 1 | 1 Jan 1994 Cross Ref Some characterizations of perturbational dual problemsOptimization, Vol. 29, No. 2 | 1 Jan 1994 Cross Ref A parallel iterative algorithm for structural optimizationComputer Methods in Applied Mechanics and Engineering, Vol. 96, No. 1 | 1 Apr 1992 Cross Ref On a Lagrangian sufficient optimality conditionJournal of Optimization Theory and Applications, Vol. 68, No. 1 | 1 Jan 1991 Cross Ref Some general Lagrangian duality theoremsJournal of Mathematical Analysis and Applications, Vol. 144, No. 1 | 1 Nov 1989 Cross Ref Semidifferentiable functions and necessary optimality conditionsJournal of Optimization Theory and Applications, Vol. 60, No. 2 | 1 Feb 1989 Cross Ref Necessary Optimality Conditions via Image ProblemNonsmooth Optimization and Related Topics | 1 Jan 1989 Cross Ref Maximum Entropy and Constrained OptimizationMaximum Entropy and Bayesian Methods | 1 Jan 1989 Cross Ref Entropy-based optimum sizing of trussesCivil Engineering Systems, Vol. 5, No. 3 | 1 Sep 1988 Cross Ref Programmation mathématique et théorie économiqueL'Actualité économique, Vol. 64, No. 2 | 1 Jan 1988 Cross Ref A MAXIMUM ENTROPY APPROACH TO CONSTRAINED NON-LINEAR PROGRAMMINGEngineering Optimization, Vol. 12, No. 3 | 1 Oct 1987 Cross Ref Symmetric duality for disjunctive programming with absolute value functionalsEuropean Journal of Operational Research, Vol. 31, No. 1 | 1 Jul 1987 Cross Ref Duality in nonlinear programs using augmented Lagrangian functionsJournal of Mathematical Analysis and Applications, Vol. 121, No. 1 | 1 Jan 1987 Cross Ref A general theory of dual optimization problemsJournal of Mathematical Analysis and Applications, Vol. 116, No. 1 | 1 May 1986 Cross Ref Non-linear separation theorems, duality and optimality conditionsOptimization and Related Fields | 16 September 2006 Cross Ref LiteraturverzeichnisOperations Research | 1 Jan 1985 Cross Ref Lagrangean relaxation and decomposition in an uncapacitated 2-hierarchal location-allocation problemComputers & Operations Research, Vol. 12, No. 2 | 1 Jan 1985 Cross Ref On duality-separability-relationsOptimization, Vol. 16, No. 6 | 27 June 2007 Cross Ref Theorems of the alternative and optimality conditionsJournal of Optimization Theory and Applications, Vol. 42, No. 3 | 1 Mar 1984 Cross Ref Comments on the general duality survey by J. Tind and L.A. WolseyMathematical Programming, Vol. 25, No. 2 | 1 Jun 1983 Cross Ref Selected Bibliography of Works Not CitedIntroduction to Sensitivity and Stability Analysis in Nonlinear Programming | 1 Jan 1983 Cross Ref Méthodes d'optimisation pour un problème de théorie des nombresRAIRO. Informatique théorique, Vol. 17, No. 3 | 8 January 2011 Cross Ref On abstract duality in mathematical programmingZeitschrift für Operations Research, Vol. 26, No. 1 | 1 Dec 1982 Cross Ref An elementary survey of general duality theory in mathematical programmingMathematical Programming, Vol. 21, No. 1 | 1 Dec 1981 Cross Ref Multi-Item Inventory Management with Monetary Objective FunctionA I I E Transactions, Vol. 13, No. 1 | 6 July 2007 Cross Ref Decomposition in large system optimization using the method of multipliersJournal of Optimization Theory and Applications, Vol. 25, No. 2 | 1 Jun 1978 Cross Ref On $\Phi $-Convexity in Extremal ProblemsSzymon Dolecki and Stanisław KurcyuszSIAM Journal on Control and Optimization, Vol. 16, No. 2 | 18 July 2006AbstractPDF (2290 KB)Projection on a Cone, Penalty Functionals and Duality Theory for Problems with Inequaltity Constraints in Hilbert SpaceAndrzej P. Wierzbicki and Stanislaw KurcyuszSIAM Journal on Control and Optimization, Vol. 15, No. 1 | 18 July 2006AbstractPDF (3182 KB)A generalization of Lagrange's method of undetermined multipliers using zero-zone functionalsJournal of Optimization Theory and Applications, Vol. 19, No. 4 | 1 Aug 1976 Cross Ref Nested Decomposition of Multistage Convex ProgramsRichard P. O'NeillSIAM Journal on Control and Optimization, Vol. 14, No. 3 | 18 July 2006AbstractPDF (717 KB)A four-variable world systemOptimization Techniques Modeling and Optimization in the Service of Man Part 1 | 21 May 2005 Cross Ref Some remarks on generalized lagrangiansOptimization Techniques Modeling and Optimization in the Service of Man Part 2 | 21 May 2005 Cross Ref A generalized Lagrangian function and multiplier methodJournal of Optimization Theory and Applications, Vol. 17, No. 3-4 | 3 August 2013 Cross Ref The use of Hestenes' method of multipliers to resolve dual gaps in engineering system optimizationJournal of Optimization Theory and Applications, Vol. 15, No. 3 | 1 Mar 1975 Cross Ref Constructive Duality in Integer ProgrammingMarshall L. Fisher and Jeremy F. ShapiroSIAM Journal on Applied Mathematics, Vol. 27, No. 1 | 12 July 2006AbstractPDF (2205 KB)Optimal design of a multi-item, multi-location, multi-repair type repair and supply systemNaval Research Logistics Quarterly, Vol. 21, No. 2 | 1 Jun 1974 Cross Ref A Saddle-Point Optimality Criterion for Nonconvex Programming in Normed SpacesRichard H. Smith, Jr. and V. David VandeLindeSIAM Journal on Applied Mathematics, Vol. 23, No. 2 | 12 July 2006AbstractPDF (919 KB)Penalty function versus non-penalty function methods for constrained nonlinear programming problemsMathematical Programming, Vol. 1, No. 1 | 1 Dec 1971 Cross Ref The Progressive Support Method for Convex ProgrammingS. N. AfriatSIAM Journal on Numerical Analysis, Vol. 7, No. 3 | 14 July 2006AbstractPDF (912 KB)IntroductionOptimization of Discrete Time Systems Cross Ref One stage systemsOptimization of Discrete Time Systems Cross Ref Volume 17, Issue 6| 1969SIAM Journal on Applied Mathematics1017-1325 History Submitted:21 July 1967Published online:17 February 2012 InformationCopyright © 1969 © Society for Industrial and Applied MathematicsPDF Download Article & Publication DataArticle DOI:10.1137/0117120Article page range:pp. 1280-1297ISSN (print):0036-1399ISSN (online):1095-712XPublisher:Society for Industrial and Applied Mathematics

It is shown that the existence of a strict local minimum satisfying the constraint qualification of [16] or McCormick's [12] second order sufficient optimality condition implies the existence of a class of exact local penalty functions (that is ones with a finite value of the penalty parameter) for a nonlinear programming problem. A lower bound to the penalty parameter is given by a norm of the optimal Lagrange multipliers which is dual to the norm used in the penalty function.

Restricted accessMoreSectionsView PDF ToolsAdd to favoritesDownload CitationsTrack Citations ShareShare onFacebookTwitterLinked InRedditEmail Cite this article Yang Gao David 2001Complementarity, polarity and triality in non‐smooth, non–convex and non–conservative Hamilton systemsPhil. Trans. R. Soc. A.3592347–2367http://doi.org/10.1098/rsta.2001.0855SectionRestricted accessComplementarity, polarity and triality in non‐smooth, non–convex and non–conservative Hamilton systems David Yang Gao David Yang Gao Department of Mathematics, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, USA () Google Scholar Find this author on PubMed Search for more papers by this author David Yang Gao David Yang Gao Department of Mathematics, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, USA () Google Scholar Find this author on PubMed Search for more papers by this author Published:15 December 2001https://doi.org/10.1098/rsta.2001.0855AbstractThis paper presents a unified critical–point theory in non–smooth, non–convex and dissipative Hamilton systems. The canonical dual/polar transformation methods and the associated bi–duality and triality theories proposed recently in non–convex variational problems are generalized into fully nonlinear dissipative dynamical systems governed by non–smooth constitutive laws and boundary conditions. It is shown that, by this method, non–smooth and non–convex Hamilton systems can be reformulated into certain smooth dual, complementary and polar variational problems. Based on a newly proposed polar Hamiltonian, a nice bipolarity variational principle is established for three–dimensional non–smooth elastodynamical systems, and a potentially powerful complementary variational principle can be used for solving unilateral variational inequality problems governed by non–smooth boundary conditions. Previous ArticleNext Article VIEW FULL TEXT DOWNLOAD PDF FiguresRelatedReferencesDetailsCited by Qiu Z and Xia H (2021) Symplectic perturbation series methodology for non-conservative linear Hamiltonian system with damping, Acta Mechanica Sinica, 10.1007/s10409-021-01076-0, 37:6, (983-996), Online publication date: 1-Jun-2021. Gao D and Ali E (2019) A Novel Canonical Duality Theory for Solving 3-D Topology Optimization Problems Advances in Mathematical Methods and High Performance Computing, 10.1007/978-3-030-02487-1_13, (209-246), . Gao D (2019) Canonical Duality-Triality Theory: Unified Understanding for Modeling, Problems, and NP-Hardness in Global Optimization of Multi-Scale Systems Advances in Mathematical Methods and High Performance Computing, 10.1007/978-3-030-02487-1_1, (3-50), . Gao D (2018) On topology optimization and canonical duality method, Computer Methods in Applied Mechanics and Engineering, 10.1016/j.cma.2018.06.027, 341, (249-277), Online publication date: 1-Nov-2018. Gao D, Ruan N and Latorre V (2017) Canonical Duality-Triality Theory: Bridge Between Nonconvex Analysis/Mechanics and Global Optimization in Complex System Canonical Duality Theory, 10.1007/978-3-319-58017-3_1, (1-47), . Gao D (2015) Analytical solutions to general anti-plane shear problems in finite elasticity, Continuum Mechanics and Thermodynamics, 10.1007/s00161-015-0412-y, 28:1-2, (175-194), Online publication date: 1-Mar-2016. Gao D (2009) Canonical duality theory: Unified understanding and generalized solution for global optimization problems, Computers & Chemical Engineering, 10.1016/j.compchemeng.2009.06.009, 33:12, (1964-1972), Online publication date: 1-Dec-2009. Gao D and Sherali H (2009) Canonical Duality Theory: Connections between Nonconvex Mechanics and Global Optimization Advances in Applied Mathematics and Global Optimization, 10.1007/978-0-387-75714-8_8, (257-326), . Penot J (2009) Ekeland Duality as a Paradigm Advances in Applied Mathematics and Global Optimization, 10.1007/978-0-387-75714-8_10, (349-376), . Yuan Y (2008) Optimal solutions to a class of nonconvex minimization problems with linear inequality constraints, Applied Mathematics and Computation, 10.1016/j.amc.2008.04.016, 203:1, (142-152), Online publication date: 1-Sep-2008. Gao D and Yu H (2008) Multi-scale modelling and canonical dual finite element method in phase transitions of solids, International Journal of Solids and Structures, 10.1016/j.ijsolstr.2007.08.027, 45:13, (3660-3673), Online publication date: 1-Jun-2008. Gao D (2007) Duality-Mathematics Wiley Encyclopedia of Electrical and Electronics Engineering, 10.1002/047134608X.W2412.pub2 Gao D (2016) Complementary Principle, Algorithm, and Complete Solutions to Phase Transitions in Solids Governed by Landau-Ginzburg Equation, Mathematics and Mechanics of Solids, 10.1177/1081286504038455, 9:3, (285-305), Online publication date: 1-Jun-2004. Gao D (2003) Perfect duality theory and complete solutions to a class of global optimization problems*, Optimization, 10.1080/02331930310001611501, 52:4-5, (467-493), Online publication date: 1-Aug-2003. Gao D (2003) Nonconvex Semi-Linear Problems and Canonical Duality Solutions Advances in Mechanics and Mathematics, 10.1007/978-1-4613-0247-6_5, (261-312), . This Issue15 December 2001Volume 359Issue 1789Theme Issue ‘Non-smooth mechanics’ compiled by F. G. Pfeiffer Article InformationDOI:https://doi.org/10.1098/rsta.2001.0855Published by:Royal SocietyPrint ISSN:1364-503XOnline ISSN:1471-2962History: Published online15/12/2001Published in print15/12/2001 License: Citations and impact Keywordsnon–convex variational problemsnon–smooth elastodynamicstrialitypolaritynon–conservative Hamilton systemscomplementarity

Nonlinear optimization is a critical branch in applied mathematics and has attracted wide attention due to its popularity in practical applications. In this work, we present two methods which use first-order information to solve two typical classes of nonlinear structured optimization problems. For a class of unconstrained nonconvex composite optimization problems where the objective is the sum of a smooth but possibly nonconvex function and a convex but possibly nonsmooth function, we propose a unified proximal gradient method with extrapolation, which provides unified treatment to convex and nonconvex problems. The method achieves the best-known convergence rate for first-order methods when solving convex optimization problems. In the case that the problem is nonconvex, the method performs as a proximal gradient method with extrapolation, and a linear convergence rate of the objective values and the generated iterates is obtained under additional proper assumptions. The efficiency of the algorithm is shown by numerical experiments. For a family of nonconvex separable optimization problems with linear constraints where the objective function is the sum of a smooth but possibly nonconvex function and a possibly nonsmooth nonconvex function, an inexact alternating direction method of multipliers is designed. The method solves subproblems to adaptive error criteria. An expansion step and more flexible dual stepsize are exploited to accelerate the convergence of the algorithm. A linear convergence rate of the generated iterates is guaranteed under proper conditions. Numerical examples illustrate the better performance of the method compared with state-of-the-art ADMM algorithms.

A new exact exponential penalty function method and nonconvex mathematical programming

The first key objective of this thesis is to estimate a set of econometric models of Australian broadacre agricultural production by applying the duality theory in production economics. Models of Australian broadacre agriculture are estimated using alternative formulations of econometric modelling of producer decision-making, in which cost minimisation, revenue maximisation and profit maximisation are assumed. A unique, large quasi-micro pooled cross-sectional farm dataset drawn from the Australian Agricultural and Grazing Industries Survey over a sixteen year period, from 1990 to 2005, is used for model estimation. Key policy-relevant outcomes of this investigation are estimates of price elasticities and elasticities of transformation and substitution between broadacre inputs and outputs in Australia. Multi-product dual cost, revenue and profit functions are specified for Australian broadacre agriculture. The multi-product functions are specified to accommodate the prevalent multi-enterprise operation on Australian broadacre farms. The restricted versions of the dual functions are chosen to account for quasi-fixity and the lumpiness of some capital used in broadacre production. The translog and normalised quadratic, the two most popular flexible functional forms in empirical duality applications, are used to specify the dual functions. In addition, impacts of climatic conditions, production focuses, production scales and rainfall on production are also allowed for in the estimated models. Systems of demand and/or supply equations are derived from the specified dual cost, revenue and profit functions and estimated using the quasi-micro dataset available. The estimated systems obtained have reasonable statistical goodness-of-fit with high percentage of statistically significant system coefficients. Price variables are also found to significantly influence input demand and output supply. Importantly, using the normalised quadratic form, the estimated system of input demand derived from the cost function and the estimated system of output supply derived from revenue functions satisfy the theoretical curvature conditions implied by rational economic behaviour. The estimated supply and demand system derived from the profit function violates the curvature condition but the violation is not severe. Input demand and output supply in broadacre farming are found to be fairly inelastic with respect to price changes in the short run. However, demand for fertilisers and crop and pasture chemicals are found to be sensitive to changes in their own prices and in general production costs. The second key objective of this thesis is to investigate three significant issues concerning the application of the duality theory in empirical agricultural production research using data from Australian broadacre agriculture. First, the results obtained from the dual cost, revenue and profit functions are contrasted according to goodness-of-fit, the satisfaction of theoretical regularity conditions and the sensibility of the generated elasticity estimates. The estimation result obtained under the cost minimisation assumption conforms more to economic theory than results obtained under revenue maximisation or profit maximisation. Second, the estimation results are more in line with expectations based on prior economic reasoning when the normalised quadratic functional form is used than when the translog is used. Third, results using data of Australian broadacre farming at two aggregate levels indicate that data aggregation across firms can have a significant impact on research findings, depending on the assumption made regarding the economic behaviour of producers

Nonorthogonal multiple access (NOMA) has been recognized as a key solution to fulfill the demands of 5G wireless communication. In this paper, our aim is to maximize the fairness in the data rates of different users in a multiuser NOMA system. We optimize the downlink transmission subject to minimum rate requirement of each user, limited power budget at the transmitter, and the successive interference cancelation constraint. First, we solve the problem for two‐user scenario where the nonconvex problem is transformed into a standard convex minimization problem and the duality theory is exploited to find the solution. The optimal power allocation is obtained from the Karush‐Kuhn‐Tucker (KKT) conditions, whereas the dual problem is solved via subgradient algorithm. As a next step, we consider the general multiuser optimization problem where more than two users can share the same channel under NOMA transmission. We design efficient solution techniques to solve the nonconvex optimization problem with sequential quadratic programming (SQP). Furthermore, two suboptimal low complexity solutions are also presented. We found that, under the proposed schemes, the fairness increases with increasing the available transmit power and decreases with the increasing the number of users. We show a complexity comparison of the dual‐based solution and the SQP algorithm. It is observed that power optimization through KKT conditions exhibits much lower computational complexity as compared to the SQP‐based solution.

In this paper a nonlinear penalty method via a nonlinear Lagrangian function is introduced for semi-infinite programs. A convergence result is established which shows that the sequence of optimal values of nonlinear penalty problems converges to that of semi-infinite programs. Moreover a conceptual convergence result of a discretization method with an adaptive scheme for solving semi-infinite programs is established. Preliminary numerical experiments show that better optimal values for some nonlinear semi-infinite programs can be obtained using the nonlinear penalty method.

In this paper we extend the theory of exact penalty functions for nonlinear programs whose objective functions and equality and inequality constraints are locally Lipschitz; arbitrary simple constraints are also allowed. Assuming a weak stability condition, we show that for all sufficiently large penalty parameter values an isolated local minimum of the nonlinear program is also an isolated local minimum of the exact penalty function. A tight lower bound on the parameter value is provided when certain first order sufficiency conditions are satisfied. We apply these results to unify and extend some results for convex programming. Since several effective algorithms for solving nonlinear programs with differentiable functions rely on exact penalty functions, our results provide a framework for extending these algorithms to problems with locally Lipschitz functions.

In this paper, an infeasible interior-point technique is proposed to generate the nondominated set of nonlinear multi-objective optimization problems with the help of the direction-based cone method. We derive the proposed method for both convex and nonconvex problems. In order to solve the parametric optimization problems of the cone method, the infeasible interior-point method starts with an initial iterate outside the feasible region, and then gradually reduces the primal and dual infeasibility measures and the objective function value across the iterations with the help of a merit function. Estimates of the reduction of primal and dual infeasibility parameters per iteration are given. The convergence analysis of the method and an estimate of the number of iterations to reach an ϵ-precise solution are also provided. We provide the performance of the proposed methods on a variety of convex and nonconvex multi-objective test problems. Performance comparison between the proposed method and popular existing solvers is provided with respect to two performance measures and the corresponding relative efficiency measures. The reduction of a combined infeasibility measure, as the iterations progress, on the test problems is also shown graphically.

Chapter 26 - Duality in nonlinear programming

Computational experience with an exact penalty function technique (EPT)

Many optimization problems in computational science and engineering (CS&E) are characterized by expensive objective and/or constraint function evaluations paired with a lack of derivative information. Direct search methods such as generating set search (GSS) are well understood and efficient for derivative-free optimization of unconstrained and linearly-constrained problems. This paper addresses the more difficult problem of general nonlinear programming where derivatives for objective or constraint functions are unavailable, which is the case for many CS&E applications. We focus on penalty methods that use GSS to solve the linearly-constrained problems, comparing different penalty functions. A classical choice for penalizing constraint violations is {ell}{sub 2}{sup 2}, the squared {ell}{sub 2} norm, which has advantages for derivative-based optimization methods. In our numerical tests, however, we show that exact penalty functions based on the {ell}{sub 1}, {ell}{sub 2}, and {ell}{sub {infinity}} norms converge to good approximate solutions more quickly and thus are attractive alternatives. Unfortunately, exact penalty functions are discontinuous and consequently introduce theoretical problems that degrade the final solution accuracy, so we also consider smoothed variants. Smoothed-exact penalty functions are theoretically attractive because they retain the differentiability of the original problem. Numerically, they are a compromise between exact and {ell}{sub 2}{sup 2}, i.e., they converge to a good solution somewhat quickly without sacrificing much solution accuracy. Moreover, the smoothing is parameterized and can potentially be adjusted to balance the two considerations. Since many CS&E optimization problems are characterized by expensive function evaluations, reducing the number of function evaluations is paramount, and the results of this paper show that exact and smoothed-exact penalty functions are well-suited to this task.