Engineering Optimization Theory and Practice

In contributing to the body of knowledge for decision-based design, the work reported in this paper has involved steps towards building a hybrid genetic algorithm to address systems design. Highlighted is a work in progress at the Australian Defence Force Academy (ADFA). A genetic algorithm (GA) is proposed to deal with discrete aspects of a design model (e.g., allocation of space to function) and a sequential linear programming (SLP) method for the continuous aspects (e.g., sizing). Our historical Decision Based Design (DBD) tool has been the code DSIDES (Decision Support In the Design of Engineering Systems). The original functionality of DSIDES was to solve linear and non-linear goal programming styled problems using linear programming (LP) and sequential (adaptive) linear programming (SLP/ALP). We seek to enhance DSIDES’s solver capability by the addition of genetic algorithms. We will also develop the appropriate tools to deal with the decomposition and synthesis implied. The foundational paradigm for DSIDES, which remains unchanged, is the Decision Support Problem Technique (DSPT). Through introducing genetic algorithms as solvers in DSIDES, the intention is to improve the likelihood of finding the global minimum (for the formulated model) as well as the ability of dealing more effectively with nonlinear problems which have discrete variables, undifferentiable objective functions or undifferentiable constraints. Using some numerical examples and a practical ship design case study, the proposed GA based method is demonstrated to be better in maintaining diversity of populations, preventing premature convergence, compared with other similar GAs. It also has similar effectiveness in finding the solutions as the original ALP DSIDES solver.

Inner approximation algorithms have had two major roles in the mathematical programming literature. Their first role was in the construction of algorithms for the decomposition of large-scale mathematical programs, such as in the Dantzig-Wolfe decomposition principle. However, recently they have been used in the creation of algorithms that locate Kuhn-Tucker solutions to nonconvex programs. Avriel and Williams' (Avriel, M., A. C. Williams. 1970. Complementary geometric programming. SIAM J. Appl. Math. 19 125–141.) complementary geometric programming algorithm, Duffin and Peterson's (Duffin, R. J., E. L. Peterson. 1972. Reversed geometric programs treated by harmonic means. Indiana Univ. Math. J. 22 531–550.) reversed geometric programming algorithms, Reklaitis and Wilde’s (Reklaitis, G. V., D. J. Wilde. 1974. Geometric programming via a primal auxiliary problem. AIIE Trans. 6 308–317.) primal reversed geometric programming algorithm, and Bitran and Novaes' (Bitran, G. R., A. G. Novaes. 1973. Linear programming with a fractional objective function. Opns. Res. 21 22–29.) linear fractional programming algorithm are all examples of this class of inner approximation algorithms. A sequence of approximating convex programs are solved in each of these algorithms. Rosen's (Rosen, J. B. 1966. Iterative solution of nonlinear optimal control problems. SIAM J. Control 4 223–244.) inner approximation algorithm is a special case of the general inner approximation algorithm presented in this note.

In this paper, we combined Langrage Multiplier, Penalty Function and Conjugate Gradient Methods (CPLCGM), to enable Conjugate Gradient Method (CGM) to be employed for solving constrained optimization problems. In the year past, Langrage Multiplier Method (LMM) has been used extensively to solve constrained optimization problems likewise Penalty Function Method (PFM). However, with some special features in CGM, which makes it unique in solving unconstrained optimization problems, we see that this features we be advantageous to solve constrained optimization problems if it can be properly amended. This, then call for the CPLCGM that is aimed at taking care of some constrained optimization problems, either with equality or inequality constraint but in this paper, we focus on equality constraints. The authors of this paper desired that, with the construction of the new Algorithm, it will circumvent the difficulties undergone using only LMM and as well as PFM to solve constrained optimization problems and its application will further improve the result of the Conjugate Gradient Method in solving this class of optimization problem. We applied the new algorithm to some constrained optimization problems and compared the results with the LMM and PFM.

A dual convex programming approach to solving linear programs with inequality constraints through entropic perturbation is derived. The amount of perturbation required depends on the desired accuracy of the optimum. The dual program contains only non‐positivity constraints. An ϵ‐optimal solution to the linear program can be obtained effortlessly from the optimal solution of the dual program. Since cross‐entropy minimization subject to linear inequality constraints is a special case of the perturbed linear program, the duality result becomes readily applicable. Many standard constrained optimization techniques can be specialized to solve the dual program. Such specializations, made possible by the simplicity of the constraints, significantly reduce the computational effort usually incurred by these methods. Immediate applications of the theory developed include an entropic path‐following approach to solving linear semi‐infinite programs with an infinite number of inequality constraints and the widely used entropy optimization models with linear inequality and/or equality constraints.

A new general-purpose optimization program for engineering design is described. ADS (Automated Design Synthesis - Version 1.10) is a FORTRAN program for solution of nonlinear constrained optimization problems. The program is segmented into three levels: strategy, optimizer, and one-dimensional search. At each level, several options are available so that a total of over 100 possible combinations can be created. Examples of available strategies are sequential unconstrained minimization, the Augmented Lagrange Multiplier method, and Sequential Linear Programming. Available optimizers include variable metric methods and the Method of Feasible Directions as examples, and one-dimensional search options include polynomial interpolation and the Golden Section method as examples. Emphasis is placed on ease of use of the program. All information is transferred via a single parameter list. Default values are provided for all internal program parameters such as convergence criteria, and the user is given a simple means to over-ride these, if desired.

Chapter 4 - Optimum Design Concepts: Optimality Conditions

This thesis proposes a new active-set method for large-scale nonlinearly con strained optimization. The method solves a sequence of linear programs to generate search directions. The typical approach for globalization is based on damping the search directions with a trust-region constraint; our proposed ap proach is instead based on using a 2-norm regularization term in the objective. Numerical evidence is presented which demonstrates scaling inefficiencies in current sequential linear programming algorithms that use a trust-region constraint. Specifically, we show that the trust-region constraints in the trustregion subproblems significantly reduce the warm-start efficiency of these subproblem solves, and also unnecessarily induce infeasible subproblems. We also show that the use of a regularized linear programming (RLP) step largely elim inates these inefficiencies and, additionally, that the dual problem to RLP is a bound-constrained least-squares problem, which may allow for very efficient subproblem solves using gradient-projection-type algorithms. Two new algorithms were implemented and are presented in this thesis, based on solving sequences of RLPs and trust-region constrained LPs. These algorithms are used to demonstrate the effectiveness of each type of subproblem, which we extrapolate onto the effectiveness of an RLP-based algorithm with the addition of Newton-like steps. All of the source code needed to reproduce the figures and tables presented in this thesis is available online at http: //www.cs.ubc.ca/labs/scl/thesis/lOcrowe/

Previous article Next article Complementarity Theorems for Linear ProgrammingA. C. WilliamsA. C. Williamshttps://doi.org/10.1137/1012015PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAbout[1] Frank Eugene Clark, Mathematical Notes: Remark on the Constraint Sets in Linear Programming, Amer. Math. Monthly, 68 (1961), 351–352 MR1531192 0109.38204 CrossrefISIGoogle Scholar[2] A. J. Goldman, Resolution and separation theorems for polyhedral convex setsLinear inequalities and related systems, Annals of Mathematics Studies, no. 38, Princeton University Press, Princeton, N. J., 1956, 41–51, in [4] MR0089113 0072.37505 Google Scholar[3] A. J. Goldman and , A. W. Tucker, Theory of linear programmingLinear inequalities and related systems, Annals of Mathematics Studies, no. 38, Princeton University Press, Princeton, N.J., 1956, 53–97, in [4] MR0101826 0072.37601 Google Scholar[4] H. W. Kuhn and , A. W. Tucker, Linear Inequalities and Related Systems, Princeton University Press, Princeton, 1956 0072.37502 Google Scholar[5] A. C. Williams, Boundedness relations for linear constraint sets, Linear Algebra and Appl., 3 (1970), 129–141 10.1016/0024-3795(70)90009-1 MR0266612 0201.22003 CrossrefGoogle Scholar Previous article Next article FiguresRelatedReferencesCited byDetails Sparse solutions to an underdetermined system of linear equations via penalized Huber loss6 November 2020 | Optimization and Engineering, Vol. 22, No. 3 Cross Ref Dynamic Non-diagonal Regularization in Interior Point Methods for Linear and Convex Quadratic Programming26 February 2019 | Journal of Optimization Theory and Applications, Vol. 181, No. 3 Cross Ref Necessary and Sufficient Conditions for Noiseless Sparse Recovery via Convex Quadratic SplinesMustafa Ç Pinar12 February 2019 | SIAM Journal on Matrix Analysis and Applications, Vol. 40, No. 1AbstractPDF (416 KB)A labeling algorithm for the sensitivity ranges of the assignment problemApplied Mathematical Modelling, Vol. 35, No. 10 Cross Ref Bibliography15 August 2011 Cross Ref Sensitivity Analysis and Dynamic Programming15 February 2011 Cross Ref Bibliography Cross Ref Condition measures and properties of the central trajectory of a linear programMathematical Programming, Vol. 83, No. 1-3 Cross Ref New characterizations of ℓ1 solutions to overdetermined systems of linear equationsOperations Research Letters, Vol. 16, No. 3 Cross Ref Limiting behavior of weighted central paths in linear programmingMathematical Programming, Vol. 65, No. 1-3 Cross Ref Stability of linearly constrained convex quadratic programsJournal of Optimization Theory and Applications, Vol. 64, No. 1 Cross Ref Marginal values in mixed integer linear programmingMathematical Programming, Vol. 44, No. 1-3 Cross Ref A theory of linear inequality systemsLinear Algebra and its Applications, Vol. 106 Cross Ref Boundedness relations in linear semi-infinite programmingAdvances in Applied Mathematics, Vol. 8, No. 1 Cross Ref A Variable-Complexity Norm Maximization ProblemO. L. Mangasarian and T. -H. Shiau17 July 2006 | SIAM Journal on Algebraic Discrete Methods, Vol. 7, No. 3AbstractPDF (698 KB)Simple computable bounds for solutions of linear complementarity problems and linear programs26 February 2009 Cross Ref On polyhedral extension of some LP theoremsMathematical Programming, Vol. 30, No. 1 Cross Ref Polyhedral extensions of some theorems of linear programmingMathematical Programming, Vol. 24, No. 1 Cross Ref Optimal simplex tableau characterization of unique and bounded solutions of linear programsJournal of Optimization Theory and Applications, Vol. 35, No. 1 Cross Ref Projection and Restriction Methods in Geometric Programming and Related Problems Cross Ref Representation of Convex Sets Cross Ref Projection and restriction methods in geometric programming and related problemsJournal of Optimization Theory and Applications, Vol. 26, No. 1 Cross Ref The complementary unboundedness of dual feasible solution sets in convex programmingMathematical Programming, Vol. 12, No. 1 Cross Ref Theorems on the dimension of convex setsLinear Algebra and its Applications, Vol. 12, No. 1 Cross Ref On the primal and dual constraint sets in geometric programmingJournal of Mathematical Analysis and Applications, Vol. 32, No. 3 Cross Ref Volume 12, Issue 1| 1970SIAM Review History Submitted:23 December 1968Published online:18 July 2006 InformationCopyright © 1970 Society for Industrial and Applied MathematicsPDF Download Article & Publication DataArticle DOI:10.1137/1012015Article page range:pp. 135-137ISSN (print):0036-1445ISSN (online):1095-7200Publisher:Society for Industrial and Applied Mathematics

Sequential linear goal programming: Implementation via MPSX

<p><span>The duality principle provides that optimization problems may be viewed from either of two perspectives, the primal problem or the dual problem. The solution to the dual problem provides a lower bound to the solution of the primal (minimization) problem. However the optimal values of the primal and dual problems need not be equal. Their difference is called the duality gap. For convex optimization problems, the duality gap is zero under a constraint qualification condition.<span> </span>In other words given any linear program, there is another related linear program called the dual. In this paper, an understanding of the dual linear program will be developed. This understanding will give important insights into the algorithm and solution of optimization problem in linear programming. <span> </span>Thus the main concepts of duality will be explored by the solution of simple optimization problem.</span></p>

A method of random search in constrained minimization problems

This paper presents a new Successive Fuzzy Linear Programming (SFLP) method for Reactive Power Optimization (RPO) to minimize the transmission loss and improve the voltage profile. The SFLP method uses Fuzzy LP (FLP) model in the Successive Linear Programming (SLP) framework to solve the RPO problem. In the FLP model, the objective and each of the constraints are assigned a satisfaction parameter. The satisfaction parameter corresponding to the objective quantifies the degree of closeness of the objective in the current state to the optimum. The satisfaction parameter corresponding to a constraint describes the degree of enforcement of that constraint. By maximizing the minimum of these satisfaction parameters, the objective as well as the constraint enforcements are maximized. The SFLP method of RPO is efficient and reliable in scheduling systems having severe under-voltages and insufficient reactive power, as the method is capable of maximizing the enforcement of the violated load bus voltage constraints, minimizing the transmission loss while simultaneously enforcing all the other constraints strictly. This method uses compactly stored, factorized constant matrices in all the LP steps, both for the construction of the FLP model as well as for the power flow solution. The control variables used in this method are generator bus voltage magnitudes, reactive powers of switchable VAR sources, and on-load tap changer (OLTC) settings of transformers. The method was tested on IEEE test systems and on a practical electric utility system. The merits of the proposed method compared to the SLP method using non-fuzzy approach are brought out.

Integral methods -such as the Finite Element Method (FEM) and the Boundary Element Method (BEL1)are frequently used in structural optimization problems to solve systems of partial differential equations. Therefore: one must take into account the large computational requirements of these sophisticated techniques at the time of choosing a suitable Mathematical Programming (MP) algorithm for this kind of problems. Among the currently available M P algorithms, Sequential Linear Programming (SLP) seems t o be one of the most adequate to structural optimization. Basically, SLP consist in constructing succesive linear approximations to the original non linear optimization problem within each step. However, the application of SLP may involve important malfunctions. Thus, the solution to the approximated linear problems can fail to exist, or may lead to a highly unfeasible point of the original non linear problem; also, large oscillations often occur near the optimum, precluding the algorithm to converge. In this paper, we present an improved SLP algorithm with line-search. specially designed for structural optimization problems. In each iteration) an approximated linear problem with aditional side constraints is solved by Linear Programming. The solution to this linear problem defines a search direction. Then, the objective function and the non linear constraints are quadratically approximated in the search direction, and a line-search is performed. The algorithm includes strategies t o avoid stalling in the boundary of the feasible region, and to obtain alternate search directions in the case of incompatible linearized constraints. Techniques developed by the authors for efficient high-order shape sensitivity analysis are referenced. Transactions on the Built Environment vol 52, © 2001 WIT Press, www.witpress.com, ISSN 1743-3509

The purpose of the study was to investigate how effectively the penalty function methods are able to solve constrained optimization problems. The approach in these methods is to transform the constrained optimization problem into an equivalent unconstrained problem and solved using one of the algorithms for unconstrained optimization problems. Algorithms and matrix laboratory (MATLAB) codes are developed using Powell’s method for unconstrained optimization problems and then problems that have appeared frequently in the optimization literature, which have been solved using different techniques compared with other algorithms. It is found out in the research that the sequential transformation methods converge to at least to a local minimum in most cases without the need for the convexity assumptions and with no requirement for differentiability of the objective and constraint functions. For problems of non-convex functions it is recommended to solve the problem with different starting points, penalty parameters and penalty multipliers and take the best solution. But on the other hand for the exact penalty methods convexity assumptions and second-order sufficiency conditions for a local minimum is needed for the solution of unconstrained optimization problem to converge to the solution of the original problem with a finite penalty parameter. In these methods a single application of an unconstrained minimization technique as against the sequential methods is used to solve the constrained optimization problem. Key words: Penalty function, penalty parameter, augmented lagrangian penalty function, exact penalty function, unconstrained representation of the primal problem.

In this paper we show how the standard goal programming approach to engineering design can be enhanced by the use of multiple objective linear and geometric programming formulations. For simple formulations a logarithmic transformation may be adequate to allow problems to be solved using linear programming and multiple objective linear programming methods. For some more complex forms, geometric programming can be used.