Population-based heuristic algorithms for continuous and mixed discrete-continuous optimization problems

Abstract

Continuous optimization problems are optimization problems where all variableshave a domain that typically is a subset of the real numbers; mixed discrete-continuousoptimization problems have additionally other types of variables, sothat some variables are continuous and others are on an ordinal or categoricalscale. Continuous and mixed discrete-continuous problems have a wide rangeof applications in disciplines such as computer science, mechanical or electricalengineering, economics and bioinformatics. These problems are also often hard tosolve due to their inherent difficulties such as a large number of variables, manylocal optima or other factors making problems hard. Therefore, in this thesis ourfocus is on the design, engineering and configuration of high-performing heuristicoptimization algorithms.We tackle continuous and mixed discrete-continuous optimization problemswith two classes of population-based heuristic algorithms, ant colony optimization(ACO) algorithms and evolution strategies. In a nutshell, the main contributionsof this thesis are that (i) we advance the design and engineering of ACO algorithms to algorithms that are competitive or superior to recent state-of-the-artalgorithms for continuous and mixed discrete-continuous optimization problems,(ii) we improve upon a specific state-of-the-art evolution strategy, the covariancematrix adaptation evolution strategy (CMA-ES), and (iii) we extend CMA-ES totackle mixed discrete-continuous optimization problems.More in detail, we propose a unified ant colony optimization (ACO) frameworkfor continuous optimization (UACOR). This framework synthesizes algorithmiccomponents of two ACO algorithms that have been proposed in the literatureand an incremental ACO algorithm with local search for continuous optimization,which we have proposed during my doctoral research. The design of UACORallows the usage of automatic algorithm configuration techniques to automaticallyderive new, high-performing ACO algorithms for continuous optimization. We alsopropose iCMAES-ILS, a hybrid algorithm that loosely couples IPOP-CMA-ES, aCMA-ES variant that uses a restart schema coupled with an increasing populationsize, and a new iterated local search (ILS) algorithm for continuous optimization.The hybrid algorithm consists of an initial competition phase, in which IPOP-CMA-ES and the ILS algorithm compete for further deployment during a secondphase. A cooperative aspect of the hybrid algorithm is implemented in the formof some limited information exchange from IPOP-CMA-ES to the ILS algorithmduring the initial phase. Experimental studies on recent benchmark functionssuites show that UACOR and iCMAES-ILS are competitive or superior to otherstate-of-the-art algorithms.To tackle mixed discrete-continuous optimization problems, we extend ACOMV and propose CESMV, an ant colony optimization algorithm and a covariance matrix adaptation evolution strategy, respectively. In ACOMV and CESMV , the decision variables of an optimization problem can be declared as continuous, ordinal, or categorical, which allows the algorithm to treat them adequately. ACOMV andCESMV include three solution generation mechanisms: a continuous optimizationmechanism, a continuous relaxation mechanism for ordinal variables, and a categorical optimization mechanism for categorical variables. Together, these mechanisms allow ACOMV and CESMV to tackle mixed variable optimization problems.We also propose a set of artificial, mixed-variable benchmark functions, which cansimulate discrete variables as ordered or categorical. We use them to automatically tune ACOMV and CESMV's parameters and benchmark their performance.Finally we test ACOMV and CESMV on various real-world continuous and mixed-variable engineering optimization problems. Comparisons with results from theliterature demonstrate the effectiveness and robustness of ACOMV and CESMVon mixed-variable optimization problems.Apart from these main contributions, during my doctoral research I have accomplished a number of additional contributions, which concern (i) a note on thebound constraints handling for the CEC'05 benchmark set, (ii) computational results for an automatically tuned IPOP-CMA-ES on the CEC'05 benchmark set and(iii) a study of artificial bee colonies for continuous optimization. These additionalcontributions are to be found in the appendix to this thesis.