Large Optimization Problems Research Articles

제약조건 처리기법에 따른 하모니써치 알고리즘의 효율성 평가 : 관로 최소비용설계 문제의 적용

제약조건이 있는 공학 최적화 문제에서 보다 좋은 결과를 얻기 위해서는 효율적인 제약조건 처리기법의 적용은 필수적이다. 본 연구에서는 네 가지의 제약조건 처리기법을 적용하여 메타휴리스틱 최적화 기법으로 널리 사용되고 있는 Harmony Search 알고리즘의 최적화 효율성을 평가하였다. 평가를 위해 대표적인 이산형 최적화 문제 중 하나인 상수관로 최소비용설계 문제를 적용하였다. 적용결과 전통적인 제약조건 처리방법으로 사용되던 벌칙함수에 비해 제안된 제약조건 처리기법의 결과가 효율적임을 확인하였다. 특히, <TEX>${\varepsilon}$</TEX>-Constrained Method의 경우 기존방법에 비하여 효율적이고 안정적인 결과를 도출하였다. 제안된 방법은 새로운 최적화 알고리즘의 개발 필요 없이 HS의 성능을 증가시킬 수 있다는 점에서 의의가 있다고 판단된다. 또한 400개 이상의 결정변수를 가지는 대규모 문제의 적용을 통하여, 제안된 방법이 대규모 공학 최적화 문제에서도 활용이 가능함을 확인하였다. The application of efficient constraint handling technique is fundamental method to find better solutions in engineering optimization problems with constraints. In this research four of constraint handling techniques are used with a meta-heuristic optimization method, harmony search algorithm, and the efficiency of algorithm is evaluated. The sample problem for evaluation of effectiveness is one of the typical discrete problems, optimal pipe size design problem of water distribution system. The result shows the suggested constraint handling technique derives better solutions than classical constraint handling technique with penalty function. Especially, the case of <TEX>${\varepsilon}$</TEX>-constrained method derives solutions with efficiency and stability. This technique is meaningful method for improvement of harmony search algorithm without the need for development of new algorithm. In addition, the applicability of suggested method for large scale engineering optimization problems is verified with application of constraint handling technique to big size problem has over 400 of decision variables.

Hyperopt: a Python library for model selection and hyperparameter optimization

Sequential model-based optimization (also known as Bayesian optimization) is one of the most efficient methods (per function evaluation) of function minimization. This efficiency makes it appropriate for optimizing the hyperparameters of machine learning algorithms that are slow to train. The Hyperopt library provides algorithms and parallelization infrastructure for performing hyperparameter optimization (model selection) in Python. This paper presents an introductory tutorial on the usage of the Hyperopt library, including the description of search spaces, minimization (in serial and parallel), and the analysis of the results collected in the course of minimization. This paper also gives an overview of Hyperopt-Sklearn, a software project that provides automatic algorithm configuration of the Scikit-learn machine learning library. Following Auto-Weka, we take the view that the choice of classifier and even the choice of preprocessing module can be taken together to represent a single large hyperparameter optimization problem. We use Hyperopt to define a search space that encompasses many standard components (e.g. SVM, RF, KNN, PCA, TFIDF) and common patterns of composing them together. We demonstrate, using search algorithms in Hyperopt and standard benchmarking data sets (MNIST, 20-newsgroups, convex shapes), that searching this space is practical and effective. In particular, we improve on best-known scores for the model space for both MNIST and convex shapes. The paper closes with some discussion of ongoing and future work.

Large-Scale Optimization of Polymer Electrolyte Membrane Fuel Cells

The design of low-temperature polymer electrolyte membrane fuel cells (PEMFCs) consists of two separate but related problems: (1) geometry optimization, which focuses on finding the optimum channel configuration and overall cell dimensions; and (2) microstructure optimization, which focuses on finding the optimum distributions of the porosity, catalyst, and electrolyte that maximize the power density of the cell [1]. The number of design parameters in each of the above problems is relatively large, which makes the design of PEMFCs a difficult mathematical problem. For instance, let us consider problem (2): if we assume that the fuel cell is discretized using a 3-D finite element discretization with 106 nodes, the design variables consist of the mesh-point values of the porosity, catalyst concentration, and electrolyte density at each mesh point. Hence, the number of design variables is 3 ×106, which make the design of PEMFC a large-scale optimization problem. Usually, such large optimization problems are solved on parallel computer clusters, with a large number of concurrent processors. In this presentation we focus on the large-scale design of the microstructure of the PEMFC by developing a gradient-based method for the computation of the optimum (mathematically exact) distributions of the porosity, catalyst, and electrolyte that maximize the power density of the cell. It should be noted that traditional heuristic techniques such as genetic algorithms, swarm-optimization or other evolutionary algorithms are inappropriate for the large-scale optimization of PEMFCs. Indeed, the number of optimization variables in non-heuristic techniques in of the order or 5-10 variables, which is a few orders of magnitude smaller than the total number of design variables in PEMFCs. Since the computation time of heuristic techniques increases approximately exponentially with the number of design variables, the total optimization time of heuristic techniques becomes unpractical in large-scale optimization problems. The technique presented in this work is based on evaluating the sensitivity functions of the cell voltage and discharge current and applying a gradient-based maximization method to optimize the PEMFC. The sensitivity functions are computed efficiently using an adjoint space approach suitable for large-scale systems that can be simulated using 2-D or 3-D finite element models. The computational cost required to compute the sensitivity functions is relatively small since our technique requires solving only one sparse system of linear equations instead of performing multiple device simulations. For this reason, the numerical algorithm presented in this work can be implemented on single processor computers even in the case of 3-D finite element problems. Using the proposed algorithm we are able to predict the optimum distribution of platinum particles and electrolyte inside the catalyst layers, as well as the optimum porosity profile that maximizes the power density of the cell. The optimum distributions and porosity profiles depends on the positions of landings and openings, and on the geometry and dimensions of the layers. In agreement with existing experimental data, we obtain that the catalyst density should be distributed non-uniformly inside the cell in order to increase the power density of the cell (see figure). In the case of the porosity profile it is more efficient to increase the porosity towards the gas diffusion layer side in order to increase the flow of the oxygen and water vapors. In the case of the electrolyte distribution in the catalyst layers it is more efficient to increase the fraction of the electrolyte towards the membrane in order to decrease the ohmic losses of the protons. Finally, it should be noted that our numerical algorithm can also show how the optimum distributions of the catalyst, porosity and electrolyte vary with the discharge conditions. The results describe the complete profiles of the optimal catalyst and porosity by providing the full distribution of optimum locations and concentration of platinum particles in the catalyst layer, as well as the “ideal” porosity distribution of the PEMFC under the specified operating conditions. More details about the technique, the numerical implementation, and the final optimized structures will be presented at the conference.

A New Hybrid Simulated Annealing Algorithm for Large Scale Global Optimization

The present paper focus on the improvement of the efficiency of structural optimization, in typical structural optimization problems there may be many locally minimum configurations. For that reason, the application of a global method, which may escape from the locally minimum points, remain essential. In this paper, a new hybrid simulated annealing algorithm for large scale global optimization problems with constraints is proposed. The authors have developed a stochastic algorithm called SAPSPSA that uses Simulated Annealing algorithm (SA). In addition, the Simultaneous Perturbation Stochastic Approximation method (SPSA) is used to refine the solution. Commonly, structural analysis problems are constrained. For the reason that SPSA method involves penalizing constraints a penalty method is used to design a new method, called Penalty SPSA (PSPSA) method. The combination of both methods (Simulated Annealing algorithm and Penalty Simultaneous Perturbation Stochastic Approximation algorithm) provides a powerful hybrid stochastic optimization method (SAPSPSA), the proposed method is applicable for any problem where the topology of the structure is not fixed. It is simple and capable of handling problems subject to any number of constraints which may not be necessarily linear. Numerical results demonstrate the applicability, accuracy and efficiency of the suggested method for structural optimization. It is found that the best results are obtained by SAPSPSA compared to the results provided by the commercial software ANSYS.

Simultaneous integrated optimal energy flow of electricity, gas, and heat

In this paper, an integrated approach to optimize electrical, natural gas, and district heating networks simultaneously is studied. Several interdependencies between these infrastructures are considered in details including a nonlinear part-load performance for boilers and CHPs besides the valve-point effect for generators. A novel approach based on selecting an appropriate set of state-variables for the problem is proposed that eliminates the addition of any new variable to convert irregular equations into a regular set while the optimization problem is still solvable. As a large optimization problem, the optimal solution cannot be achieved by conventional mathematical techniques. Hence, it is better to use evolutionary algorithms instead. In this paper, the well-known modified teaching–learning based optimization algorithm is utilized to solve the multi-period optimal power flow problem of multi-carrier energy networks. The proposed scheme is implemented and applied to a typical multi-carrier energy network. Results are compared with some other conventional heuristic algorithms and the applicability and superiority of the proposed methodology is verified.

A new regularized limited memory BFGS-type method based on modified secant conditions for unconstrained optimization problems

In the framework of large-scale optimization problems, the standard BFGS method is not affordable due to memory constraints. The so-called limited-memory BFGS (L-BFGS) method is an adaption of the BFGS method for large-scale settings. However, the standard BFGS method and therefore the standard L-BFGS method only use the gradient information of the objective function and neglect function values. In this paper, we propose a new regularized L-BFGS method for solving large scale unconstrained optimization problems in which more available information from the function and gradient values are employed to approximate the curvature of the objective function. The proposed method utilizes a class of modified quasi-Newton equations in order to achieve higher order accuracy in approximating the second order curvature of the objective function. Under some standard assumptions, we provide the global convergence property of the new method. In order to provide an efficient method for finding global minima of a continuously differentiable function, a hybrid algorithm that combines a genetic algorithm (GA) with the new proposed regularized L-BFGS method has been proposed. This combination leads the iterates to a stationary point of the objective function with higher chance of being global minima. Numerical results show the efficiency and robustness of the new proposed regularized L-BFGS and its hybridized version with GA in practice.

An Efficient and Effective Algorithm for Large Scale Global Optimization Problems

Invasive weed optimization (IWO) algorithm and quantum-behaved particle swarm optimization (QPSO) algorithm are inclined to fall into local optimum with lower convergence accuracy when separately used to deal with large scale global optimization (LSGO) problems. In order to fully utilize the advantages of these two intelligent algorithms and complement each other, following the idea of portfolio optimization, this paper correspondingly adjusts and improves the quantum models of IWO and QPSO, organically integrates the two algorithms, and proposes the quantum-behaved invasive weed optimization (QIWO) algorithm. This mixed algorithm can achieve the purpose of information exchange and cooperative search through alternate search enables the make algorithm converge to the optimal solution quickly, properly overcoming the defects of falling into local optimum and premature convergence. Test results of 20 LSGO functions show that compared with other algorithms, QIWO has stronger global optimization capability, faster convergence speed and higher convergence accuracy.

Evolutionary Approaches for the Multi-objective Reservoir Operation Problem

The Operation Planning of Hydroelectric Systems is a large, time-coupled, stochastic, space-coupled and nonlinear optimization problem. The formulation of such problem can have several conflicting objectives in the representation of different aspects of the problem. In this work, we propose two approaches for the study and resolution of this problem. The proposals are based on two Evolutionary Metaheuristics—Genetic Algorithms and Differential Evolution. The methods work simultaneously with a set of solutions in order to perform exploration and exploitation of the search space. The intent is to find a set of solutions obtained in a single round of the algorithm, considering explicitly the different criteria of the problem. The proposed algorithms are applied to Brazilian hydropower case studies and successfully generated a wide range of cost-benefit solutions that can be applied in real planning.

Correlation matrices in kriging assisted optimisation of electromagnetic devices

Kriging surrogate modelling facilitates efficient decision making regarding where to place the next point for evaluation during optimisation. This is particularly helpful in the design of electromagnetic devices where computationally expensive numerical field modelling needs to be used. The disadvantage, however, is that correlation matrices are required which, for problems with many design variables and multiple objectives, may grow in size leading to the need for page swapping and thus slowing down of what in principle should be a very fast process. In this study several methodologies to reduce the computational resources required in such problems are proposed. The efficiency of the proposed approach is demonstrated using an example of a large multi-parameter optimisation problem where kriging coupled with the average gradient value method is employed.

Performance Analysis of Sparse Data Structure Implementations

Abstract Sparse linear algebraic data structures are widely used during the solution of large scale linear optimization problems. The efficiency of the solver is significantly influenced by the used data structures. The implementations of such data structures are not trivial. A performance analysis of the available data structures can provide valuable information to improve efficiency. In the talk we present our software that supports this task as well as our new, special vector representation. We also report results covering the solution for numerical issues affecting the performance of sparse linear algebraic operations.

A combined SQP–IPM algorithm for solving large-scale nonlinear optimization problems

We consider a combined IPM–SQP method to solve smooth nonlinear optimization problems, which may possess a large number of variables and a sparse Jacobian matrix of the constraints. Basically, the algorithm is a sequential quadratic programming (SQP) method, where the quadratic programming subproblem is solved by a primal-dual interior point method (IPM). A special feature of the algorithm is that the quadratic programming subproblem does not need to become exactly solved. To solve large optimization problems, either a limited-memory BFGS update to approximate the Hessian of the Lagrangian function is applied or the user specifies the Hessian by himself. Numerical results are presented for the 306 small and dense Hock-Schittkowski problems, for 13 large semi-linear elliptic control problems after a suitable discretization, and for 35 examples of the CUTEr test problem collection with more than 5000 variables.

Reliability Based Optimum Reservoir Design by Hybrid ACO-LP Algorithm

Optimal design of irrigation and water supply reservoirs under reliability constraints may be categorized as large combinatorial optimization problems. In this paper, the reliability based optimum design of a single water supply reservoir is formulated as a mixed integer programming and a hybrid algorithm is introduced for its solution. To eliminate iterative procedures in reliability-based reservoir design and operation, the reliability requirements are directly embedded into the modeling framework and treated as different sets of constraints. Adaptive penalty method is used for constraint handling in the solution methodology. The proposed algorithm couples an ant colony optimization (ACO) optimizer with a virtual linear programing (LP) model for the solution of the resulted NP-hard mixed integer nonlinear programming problem. Dez reservoir for irrigation water supply with 480 months of inflow is used to demonstrate the method and its performance. The structure and solution methodology is verified by the solution to the inverse problem. It is shown that the proposed hybrid model can efficiently solve the problem for various combinations of reliability measures in a multiple period modeling scheme. It is illustrated that under some circumstances and specific reliability values, the mixed integer nonlinear programming (MINLP) solver may even fail to address a feasible and local optimal solution. Although operating rule is not included in the operational scheme, the procedure is capable of identifying coefficients for decision rules with any proposed structure.

An interior point method and Sherman--Morrison formula for solving large scale convex quadratic problems with diagonal Hessians

We develop an approach for solving large scale convex quadratic problems with quadratic matrices subject to linear equalities and box-constraints. These problems appear in real-life applications. At first glance, this is a simple convex optimisation problem. However, the size of this problem (\(10^9~\)variables and \(10^6~\)constraints for some applications) makes it very challenging to apply traditional convex optimisation techniques. Therefore, one needs to develop a specific algorithm for solving such kind of problems. We apply a combination of the Interior Point method and Sherman--Morrison formula to solve this problem. We test our approach on smaller size datasets (\(1000~\)variables and \(100~\)constraints). Our numerical experiments show that this combination is efficient, fast and computationally stable. This approach is suitable for large scale convex quadratic optimisation problems. References E. Anderson, Z. Bai, C. Bischof, S. Blackford, J. Demmel, J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, and D. Sorensen. LAPACK Users' Guide . Society for Industrial and Applied Mathematics, Philadelphia, third edition, 1999. M. S. Bartlett. An inverse matrix adjustment arising in discriminant analysis. Ann. Math. Statist. , 22(1):107–111, 03 1951. doi:10.1214/aoms/1177729698 S. Boyd and L. Vandenberghe. Convex Optimization . Cambridge University Press, New York, NY, USA, 2010. Jack Dongarra, Mathieu Faverge, Hatem Ltaief, and Piotr Luszczek. High performance matrix inversion based on lu factorization for multicore architectures. In Proceedings of the 2011 ACM international workshop on Many task computing on grids and supercomputers , pages 33–42. ACM, 2011. G. E. Forsythe, M. A. Malcolm, and C. B. Moler. Computer Methods for Mathematical Computations . Prentice-Hall, Englewood Cliffs, N.J., 1977. Cited in Ake Bjorck's bibliography on least squares, ftp://math.liu.se/pub/references Gene H. Golub and Robert J. Plemmons. Large-scale geodetic least-squares adjustment by dissection and orthogonal decomposition. Linear Algebra and Its Applications , 34:3–28, 1980. doi:10.1016/0024-3795980)90156-1 G. H. Golub and C. F. Van Loan. Matrix Computations . Johns Hopkins Studies in the Mathematical Sciences. Johns Hopkins University Press, 1996. Jacek Gondzio and Andreas Grothey. Exploiting structure in parallel implementation of interior point methods for optimization. Technical report, School of Mathematics, University of Edinburgh, Edinburgh, 2004. Magnus R. Hestenes. Multiplier and gradient methods. J. Optimization Theory Appl. , 4:303–320, 1969. doi:10.1007/BF00927673 Magnus R. Hestenes and Eduard Stiefel. Methods of conjugate gradients for solving linear systems. J. Res. Nat. Bur. Standards , 49:409–436, 1952. http://www.sam.math.ethz.ch/ mhg/Latsis2002/jr49-6.pdf Narendra Karmarkar. A new polynomial-time algorithm for linear programming. Combinatorica , 4(4):373–395, 1984. doi:10.1145/800057.808695 Seung-Jean Kim, Kwangmoo Koh, Michael Lustig, Stephen Boyd, and Dimitry Gorinevsky. An interior-point method for large-scale L1-regularized least squares. Selected Topics in Signal Processing, IEEE Journal of , 1(4):606–617, 2007. doi:10.1109/JSTSP.2007.910971 Yurii Nesterov and Arkadii Nemirovskii. Interior-Point Polynomial Algorithms in Convex Programming , volume 13 of SIAM Studies in Applied mathematics . 1993. J. Nocedal and S. Wright. Numerical Optimization . Springer-Verlag, Inc., New York, 2006. M. J. D. Powell. A method for nonlinear constraints in minimization problems. In R. Fletcher, editor, Optimization , pages 283–298. New York, NY, 1969. http://www.ams.org/mathscinet-getitem?mr=42:7284 Sherman Robinson, Andrea Cattaneo, and Moataz El-Said. Updating and estimating a social accounting matrix using cross entropy methods. TMD discussion papers 58, International Food Policy Research Institute, 2000. http://ideas.repec.org/p/fpr/tmddps/58.html Jack Sherman and Winifred J. Morrison. Adjustment of an inverse matrix corresponding to a change in one element of a given matrix. Ann. Math. Statistics , 21:124–127, 1950. doi:10.1214/aoms/1177729893

English

This paper reviews the optimal power flow solutions under variable load conditions. In this article we review the recent trend towards non-deterministic (random) search techniques and hybrid methods for OPF and give the review conclusions. These methods have become popular because they have a theoretical advantage over the deterministic methods with respect to handling of non convexity, dynamics, and discrete variables. Present commercial OPF programs can solve very large and complex power systems optimization problems in a relatively less time. In recent years many different solution methods have been suggested to solve OPF problems. The review contributes a comprehensive discussion of specific optimization techniques that can be applied to OPF Solution methodology.

A Graph Based Framework for Route Optimization in Sea-Trade Logistics

How to find the optimal transportation route in sea-trade is very important for the logistics industry. The traditional routing problem is solved by performing the combinatorial optimization over a specified transportation network. Facing the huge network extracted from the foreign trading industry as well as the complex constraints, it is impossible for the traditional optimization methods to find the solution in a short time, which motivates our work. In this paper, we first carefully study the property of foreign trade network, and then convert the transportation network into a hierarchical one and propose a novel framework based on graphical model to solve this large scale network optimization problem. The experimental results demonstrate that our approach is superior to the famous ant colony optimization algorithm (ACO) in terms of accuracy and the time spent.

An elucidation for unit commitment problem viaImperialistic Competition Algorithm

Unit Commitment is large scale short-term optimization problem, in which main objective is to schedule generation to minimize the total fuel cost, subjected to large number of constraints that must be satisfied. In this paper unit commitment problem is solved by using a new evolutionary algorithm known as imperialistic algorithm. In ICA the initial populations individuals (countries) are the countries are in two types: imperialistic and colonies that all together form some empires .Imperialistic competitions among these empires converge to state in their exist only one empire. In the proposed ICA for the unit commitment problem, the scheduling variables are coded as integers; therefore the some constraints are handled are handled directly as minimum up and down type constraints. A new method for initializing the countries is proposed. To verify the performance of the Imperialistic Competition Algorithm (ICA), it is applied to systems with ten number of generating units in one-day scheduling period.

The performance research of artificial bee colony algorithm on the large scale global optimisation problems

Swarm intelligent optimisation algorithms have been researched widely in recent years. Artificial Bee Colony ABC algorithm is a kind of swarm intelligent algorithm with briefness and efficiency for global optimisation problems. This paper focuses on the performance of ABC algorithm on the large-scale global optimisation problems. The algorithm has been tested on 12 benchmark functions from relevant literature. The experiment results of ABC algorithm on the large-scale global optimisation problems have been shown in this paper. The experiment results indicated that as the dimensions of the test functions increase, the performance of ABC on most test problems deteriorates gradually.

A modified nonlinear conjugate gradient method for unconstrained optimization

Nonlinear conjugate gradient method holds an important role in solving large scale unconstrained optimization problems. Their simplicity, low memory requirement, and global convergence stimulated a massive study on the method. Numerous modifications have been done recently to improve its performance. In this paper, we proposed a new formula for the conjugate gradient coefficient k  that generates the descent search direction. In addition, we establish the global convergence result under exact line search. The outcome of our numerical experiment show that the proposed formula is very efficient and more reliable when compare to other conjugate gradient methods.

An adaptive preconditioned conjugate gradient approach for extremely large optimisation problems arising in computational geomechanics

The feasibility of preconditioned conjugate gradient (PCG) for solving extremely large sparse linear systems arising in three-dimensional finite element limit analysis of geotechnical stability problems is studied. With problem size increasing beyond few millions of variables, the use of direct solvers becomes prohibitive (both in storage and computational times) even on modern computers. This motivates the study on possible employment of iterative methods to solve systems of equations resulting from interior point method-based limit analysis formulations. In this paper, an adaptive preconditioning technique based on incomplete Cholesky (IC) preconditioner, as well as a novel implementation scheme for preconditioned conjugate gradient method, is proposed to effectively deal with highly ill-conditioned systems, such as those which occur at last iterations of IPM methods. Numerical tests on some ultra-large scale sample geotechnical problems, along with comparison with other potential solution methods, verify the robustness and efficiency of the proposed approach.

Serial Genetic Algorithm Decoder for Low Density Parity Check Codes

Genetic algorithms are successfully used for decoding some classes of error correcting codes, and offer very good performances for solving large optimization problems. This article proposes a new decoder based on Serial Genetic Algorithm Decoder (SGAD) for decoding Low Density Parity Check (LDPC) codes. The results show that the proposed algorithm gives large gains over sum-product decoder, which proves its efficiency.