Automatic Algorithm Configuration Techniques Research Articles

Automatic design of hybrid stochastic local search algorithms for permutation flowshop problems

Stochastic local search methods are at the core of many effective heuristics for tackling different permutation flowshop problems (PFSPs). Usually, such algorithms require a careful, manual algorithm engineering effort to reach high performance. An alternative to the manual algorithm engineering is the automated design of effective SLS algorithms through building flexible algorithm frameworks and using automatic algorithm configuration techniques to instantiate high-performing algorithms. In this paper, we automatically generate new high-performing algorithms for some of the most widely studied variants of the PFSP. More in detail, we (i) developed a new algorithm framework, EMILI, that implements algorithm-specific and problem-specific building blocks; (ii) define the rules of how to compose algorithms from the building blocks; and (iii) employ an automatic algorithm configuration tool to search for high performing algorithm configurations. With these ingredients, we automatically generate algorithms for the PFSP with the objectives makespan, total completion time and total tardiness, which outperform the best algorithms obtained by a manual algorithm engineering process.

Deep optimization for spectrum repacking

Over 13 months in 2016--17 the U.S. Federal Communications Commission conducted an "incentive auction" to repurpose radio spectrum from broadcast television to wireless internet. In the end, the auction yielded $19.8 bn, $10.05 bn of which was paid to 175 broadcasters for voluntarily relinquishing their licenses across 14 Ultra High Frequency (UHF) channels. Stations that continued broadcasting were assigned potentially new channels to fit as densely as possible into the channels that remained. The government netted more than $7 bn (used to pay down the national debt) after covering costs (including retuning). A crucial element of the auction design was the construction of a solver, dubbed SAT-based Feasibility Checker (SATFC), that determined whether sets of stations could be "repacked" in this way; it needed to run every time a station was given a price quote. This paper describes the process by which we built SATFC. We adopted an approach we dub "deep optimization," taking a data-driven, highly parametric, and computationally intensive approach to solver design. More specifically, to build SATFC we designed software that could pair both complete and local-search SAT-encoded feasibility checking with a wide range of domain-specific techniques, such as constraint graph decomposition and novel caching mechanisms that allow for reuse of partial solutions from related, solved problems. We then used automatic algorithm configuration techniques to construct a portfolio of 8 complementary algorithms to be run in parallel, aiming to achieve good performance on instances that arose in proprietary auction simulations. To evaluate the impact of our solver in this paper, we built an open-source reverse auction simulator. We found that within the short time budget required in practice, SATFC solved more than 95% of the problems it encountered. Furthermore, the incentive auction paired with SATFC produced nearly optimal allocations in a restricted setting and substantially outperformed other alternatives at national scale.

ABC-X: a generalized, automatically configurable artificial bee colony framework

The artificial bee colony (ABC) algorithm is a popular metaheuristic that was originally conceived for tackling continuous function optimization tasks. Over the last decade, a large number of variants of ABC have been proposed, making it by now a well-studied swarm intelligence algorithm. Typically, in a paper on algorithmic variants of ABC algorithms, one or at most two of its algorithmic components are modified. Possible changes include variations on the search equations, the selection of candidate solutions to be explored, or the adoption of features from other algorithmic techniques. In this article, we propose to follow a different direction and to build a generalized ABC algorithm, which we call ABC-X. ABC-X collects algorithmic components available from known ABC algorithms into a common algorithm framework that allows not only to instantiate known ABC variants but, more importantly, also many ABC algorithm variants that have never been explored before in the literature. Automatic algorithm configuration techniques can generate from this template new ABC variants that perform better than known ABC algorithms, even when their numerical parameters are fine-tuned using the same automatic configuration process.

Population-based heuristic algorithms for continuous and mixed discrete-continuous optimization problems

Continuous optimization problems are optimization problems where all variableshave a domain that typically is a subset of the real numbers; mixed discrete-continuousoptimization problems have additionally other types of variables, sothat some variables are continuous and others are on an ordinal or categoricalscale. Continuous and mixed discrete-continuous problems have a wide rangeof applications in disciplines such as computer science, mechanical or electricalengineering, economics and bioinformatics. These problems are also often hard tosolve due to their inherent difficulties such as a large number of variables, manylocal optima or other factors making problems hard. Therefore, in this thesis ourfocus is on the design, engineering and configuration of high-performing heuristicoptimization algorithms.We tackle continuous and mixed discrete-continuous optimization problemswith two classes of population-based heuristic algorithms, ant colony optimization(ACO) algorithms and evolution strategies. In a nutshell, the main contributionsof this thesis are that (i) we advance the design and engineering of ACO algorithms to algorithms that are competitive or superior to recent state-of-the-artalgorithms for continuous and mixed discrete-continuous optimization problems,(ii) we improve upon a specific state-of-the-art evolution strategy, the covariancematrix adaptation evolution strategy (CMA-ES), and (iii) we extend CMA-ES totackle mixed discrete-continuous optimization problems.More in detail, we propose a unified ant colony optimization (ACO) frameworkfor continuous optimization (UACOR). This framework synthesizes algorithmiccomponents of two ACO algorithms that have been proposed in the literatureand an incremental ACO algorithm with local search for continuous optimization,which we have proposed during my doctoral research. The design of UACORallows the usage of automatic algorithm configuration techniques to automaticallyderive new, high-performing ACO algorithms for continuous optimization. We alsopropose iCMAES-ILS, a hybrid algorithm that loosely couples IPOP-CMA-ES, aCMA-ES variant that uses a restart schema coupled with an increasing populationsize, and a new iterated local search (ILS) algorithm for continuous optimization.The hybrid algorithm consists of an initial competition phase, in which IPOP-CMA-ES and the ILS algorithm compete for further deployment during a secondphase. A cooperative aspect of the hybrid algorithm is implemented in the formof some limited information exchange from IPOP-CMA-ES to the ILS algorithmduring the initial phase. Experimental studies on recent benchmark functionssuites show that UACOR and iCMAES-ILS are competitive or superior to otherstate-of-the-art algorithms.To tackle mixed discrete-continuous optimization problems, we extend ACOMV and propose CESMV, an ant colony optimization algorithm and a covariance matrix adaptation evolution strategy, respectively. In ACOMV and CESMV , the decision variables of an optimization problem can be declared as continuous, ordinal, or categorical, which allows the algorithm to treat them adequately. ACOMV andCESMV include three solution generation mechanisms: a continuous optimizationmechanism, a continuous relaxation mechanism for ordinal variables, and a categorical optimization mechanism for categorical variables. Together, these mechanisms allow ACOMV and CESMV to tackle mixed variable optimization problems.We also propose a set of artificial, mixed-variable benchmark functions, which cansimulate discrete variables as ordered or categorical. We use them to automatically tune ACOMV and CESMV's parameters and benchmark their performance.Finally we test ACOMV and CESMV on various real-world continuous and mixed-variable engineering optimization problems. Comparisons with results from theliterature demonstrate the effectiveness and robustness of ACOMV and CESMVon mixed-variable optimization problems.Apart from these main contributions, during my doctoral research I have accomplished a number of additional contributions, which concern (i) a note on thebound constraints handling for the CEC'05 benchmark set, (ii) computational results for an automatically tuned IPOP-CMA-ES on the CEC'05 benchmark set and(iii) a study of artificial bee colonies for continuous optimization. These additionalcontributions are to be found in the appendix to this thesis.

A unified ant colony optimization algorithm for continuous optimization

In this article, we propose UACOR, a unified ant colony optimization (ACO) algorithm for continuous optimization. UACOR includes algorithmic components from ACOR,DACOR and IACOR-LS, three ACO algorithms for continuous optimization that have been proposed previously. Thus, it can be used to instantiate each of these three earlier algorithms; in addition, from UACOR we can also generate new continuous ACO algorithms that have not been considered before in the literature. In fact, UACOR allows the usage of automatic algorithm configuration techniques to automatically derive new ACO algorithms. To show the benefits of UACOR’s flexibility, we automatically configure two new ACO algorithms, UACOR-s and UACOR-c, and evaluate them on two sets of benchmark functions from a recent special issue of the Soft Computing (SOCO) journal and the IEEE 2005 Congress on Evolutionary Computation (CEC’05), respectively. We show that UACOR-s is competitive with the best of the 19 algorithms benchmarked on the SOCO benchmark set and that UACOR-c performs superior to IPOP-CMA-ES and statistically significantly better than five other algorithms benchmarked on the CEC’05 set. These results show the high potential ACO algorithms have for continuous optimization and suggest that automatic algorithm configuration is a viable approach for designing state-of-the-art continuous optimizers.

The Automatic Design of Multiobjective Ant Colony Optimization Algorithms

Multiobjective optimization problems are problems with several, typically conflicting, criteria for evaluating solutions. Without any a priori preference information, the Pareto optimality principle establishes a partial order among solutions, and the output of the algorithm becomes a set of nondominated solutions rather than a single one. Various ant colony optimization (ACO) algorithms have been proposed in recent years for solving such problems. These multiobjective ACO (MOACO) algorithms exhibit different design choices for dealing with the particularities of the multiobjective context. This paper proposes a formulation of algorithmic components that suffices to describe most MOACO algorithms proposed so far. This formulation also shows that existing MOACO algorithms often share equivalent design choices, but they are described in different terms. Moreover, this formulation is synthesized into a flexible algorithmic framework, from which not only existing MOACO algorithms may be instantiated, but also combinations of components that were never studied in the literature. In this sense, this paper goes beyond proposing a new MOACO algorithm, but it rather introduces a family of MOACO algorithms. The flexibility of the proposed MOACO framework facilitates the application of automatic algorithm configuration techniques. The experimental results presented in this paper show that the automatically configured MOACO framework outperforms the MOACO algorithms that inspired the framework itself. This paper is also among the first to apply automatic algorithm configuration techniques to multiobjective algorithms.