Algorithm Configuration Methods Research Articles

Instance-specific algorithm configuration via unsupervised deep graph clustering

Instance-specific Algorithm Configuration (AC) methods are effective in automatically generating high-quality algorithm parameters for heterogeneous NP-hard problems from multiple sources. However, existing works rely on manually designed features to describe training instances, which are simple numerical attributes and cannot fully capture structural differences. Targeting at Mixed-Integer Programming (MIP) solvers, this paper proposes a novel instances-specific AC method based on end-to-end deep graph clustering. By representing an MIP instance as a bipartite graph, a random walk algorithm is designed to extract raw features with both numerical and structural information from the instance graph. Then an auto-encoder is designed to learn dense instance embeddings unsupervisedly, which facilitates clustering heterogeneous instances into homogeneous clusters for training instance-specific configurations. Experimental results on multiple benchmarks show that the proposed method can improve the solving efficiency of CPLEX on highly heterogeneous instances, and outperform existing instance specific AC methods.

Automated Dynamic Algorithm Configuration

The performance of an algorithm often critically depends on its parameter configuration. While a variety of automated algorithm configuration methods have been proposed to relieve users from the tedious and error-prone task of manually tuning parameters, there is still a lot of untapped potential as the learned configuration is static, i.e., parameter settings remain fixed throughout the run. However, it has been shown that some algorithm parameters are best adjusted dynamically during execution. Thus far, this is most commonly achieved through hand-crafted heuristics. A promising recent alternative is to automatically learn such dynamic parameter adaptation policies from data. In this article, we give the first comprehensive account of this new field of automated dynamic algorithm configuration (DAC), present a series of recent advances, and provide a solid foundation for future research in this field. Specifically, we (i) situate DAC in the broader historical context of AI research; (ii) formalize DAC as a computational problem; (iii) identify the methods used in prior art to tackle this problem; and (iv) conduct empirical case studies for using DAC in evolutionary optimization, AI planning, and machine learning.

A Survey of Methods for Automated Algorithm Configuration

Algorithm configuration (AC) is concerned with the automated search of the most suitable parameter configuration of a parametrized algorithm. There is currently a wide variety of AC problem variants and methods proposed in the literature. Existing reviews do not take into account all derivatives of the AC problem, nor do they offer a complete classification scheme. To this end, we introduce taxonomies to describe the AC problem and features of configuration methods, respectively. We review existing AC literature within the lens of our taxonomies, outline relevant design choices of configuration approaches, contrast methods and problem variants against each other, and describe the state of AC in industry. Finally, our review provides researchers and practitioners with a look at future research directions in the field of AC.

An efficient parameter tuning method based on the Latin Hypercube Hammersley Sampling and fuzzy C-Means clustering methods

Metaheuristic algorithms, which are developed to find near-optimal solutions to optimization problems within acceptable times, have specific parameters that have a significant impact on their performance. And fine-tuning these parameters can lead to an effective and good-performing version of such algorithms. We propose a novel algorithm configuration method based on Latin Hypercube Hammersley Sampling (LHHS) and Fuzzy C-means Clustering (FCM) methods. Usage of these methods is first in the algorithm configuration literature. We evaluated the proposed tuning method in two experiments and four cases and compared its performance with the current state-of-the-art automatic parameter tuning methods. In the first experiment, we tune three numerical parameters of the Standard Genetic Algorithm (SGA) and in the second experiment, we tune two numerical parameters of the Artificial Bee Colony (ABC) algorithm. Our experimental results show the proposed tuning method showed better performance than other state-of-the-art tuning methods in two cases. And demonstrated competitive performance with the other algorithm configuration methods for the other two cases. The most important result of our experiments is that not only the best configuration but also other configurations that remained in the final configuration set demonstrated competitive results with the best configuration found with other state-of-the-art parameter tuning methods. However, because the proposed method is designed to find the best-performing configurations among a large initial configuration set using all the available tuning budgets, it requires more computational time.

Comparison of sampling methods for algorithm configuration problem: A case for tuning differential ant-stigmergy (DASA) algorithm parameters

Metaheuristic and heuristic methods, which are effective tools for solving difficult computational problems, have many design parameters, and selecting their values affects their performance. A good selection of the parameter values can increase their performance. In this study, we compared eight different sampling methods, used for tuning the five control parameters of the Differential Ant-Stigmergy Algorithm (DASA). Although most of the proposed parameter tuning methods in the literature use sampling methods to initialize or select candidate parameter vectors, we used them as individual tuning methods in this study. This is the first study to use Latin Hypercube Hammersley Sampling (LHHS) for algorithm configuration problems. In the first part of this study, the performance of a large set of DASA parameter settings, obtained using different sampling methods, was evaluated on the Sphere function optimization problem with respect to two dimensions of this function. In the second part of the experiment, we compared the performance of these differently tuned versions of DASA on the first seven noiseless benchmark test functions of the Black-Box Optimization Benchmarking 2010 workshop. The results of our research demonstrated that the best parameter vector obtained with the LHHS method had a better performance than the default parameter value of the DASA and other best parameter values obtained with other methods (it has the best-ranking amongst the all compared method). Furthermore, according to our results, LHHS can produce promising results when used for the initialization of other state-of-art algorithm configuration methods.

Generative Adversarial Construction of Parallel Portfolios.

Since automatic algorithm configuration methods have been very effective, recently there is increasing research interest in utilizing them for automatic solver construction, resulting in several notable approaches. For these approaches, a basic assumption is that the given training set could sufficiently represent the target use cases such that the constructed solvers can generalize well. However, such an assumption does not always hold in practice since in some cases, we might only have scarce and biased training data. This article studies effective construction approaches for the parallel algorithm portfolios that are less affected in these cases. Unlike previous approaches, the proposed approach simultaneously considers instance generation and portfolio construction in an adversarial process, in which the aim of the former is to generate instances that are challenging for the current portfolio, while the aim of the latter is to find a new component solver for the portfolio to better solve the newly generated instances. Applied to two widely studied problem domains, that is, the Boolean satisfiability problems (SAT) and the traveling salesman problems (TSPs), the proposed approach identified parallel portfolios with much better generalization than the ones generated by the existing approaches when the training data were scarce and biased. Moreover, it was further demonstrated that the generated portfolios could even rival the state-of-the-art manually designed parallel solvers.

Pitfalls and Best Practices in Algorithm Configuration

Good parameter settings are crucial to achieve high performance in many areas of artificial intelligence (AI), such as propositional satisfiability solving, AI planning, scheduling, and machine learning (in particular deep learning). Automated algorithm configuration methods have recently received much attention in the AI community since they replace tedious, irreproducible and error-prone manual parameter tuning and can lead to new state-of-the-art performance. However, practical applications of algorithm configuration are prone to several (often subtle) pitfalls in the experimental design that can render the procedure ineffective. We identify several common issues and propose best practices for avoiding them. As one possibility for automatically handling as many of these as possible, we also propose a tool called GenericWrapper4AC.

On automatic algorithm configuration of vehicle routing problem solvers

Many of the algorithms for solving vehicle routing problems expose parameters that strongly influence the quality of obtained solutions and the performance of the algorithm.Finding good values for these parameters is a tedious task that requires experimentation and experience. Therefore, methods that automate the process of algorithm configuration have received growing attention. In this paper, we present a comprehensive study to critically evaluate and compare the capabilities and suitability of seven state-of-the-art methods in configuring vehicle routing metaheuristics. The configuration target is the solution quality of eight metaheuristics solving two vehicle routing problem variants. We show that the automatic algorithm configuration methods find good parameters for the vehicle route optimization metaheuristics and clearly improve the solutions obtained over default parameters. Our comparison shows that despite some observable differences in configured performance there is no single configuration method that always outperforms the others. However, largest gains in performance can be made by carefully selecting the right configurator. The findings of this paper may give insights on how to effectively choose and extend automatic parameter configuration methods and how to use them to improve vehicle routing solver performance.

Warmstarting of Model-Based Algorithm Configuration

The performance of many hard combinatorial problem solvers depends strongly on their parameter settings, and since manual parameter tuning is both tedious and suboptimal the AI community has recently developed several algorithm configuration (AC) methods to automatically address this problem. While all existing AC methods start the configuration process of an algorithm A from scratch for each new type of benchmark instances, here we propose to exploit information about A's performance on previous benchmarks in order to warmstart its configuration on new types of benchmarks. We introduce two complementary ways in which we can exploit this information to warmstart AC methods based on a predictive model. Experiments for optimizing a flexible modern SAT solver on twelve different instance sets show that our methods often yield substantial speedups over existing AC methods (up to 165-fold) and can also find substantially better configurations given the same compute budget.

A Large-Scale Experimental Evaluation of High-Performing Multi- and Many-Objective Evolutionary Algorithms.

Research on multi-objective evolutionary algorithms (MOEAs) has produced over the past decades a large number of algorithms and a rich literature on performance assessment tools to evaluate and compare them. Yet, newly proposed MOEAs are typically compared against very few, often a decade older MOEAs. One reason for this apparent contradiction is the lack of a common baseline for comparison, with each subsequent study often devising its own experimental scenario, slightly different from other studies. As a result, the state of the art in MOEAs is a disputed topic. This article reports a systematic, comprehensive evaluation of a large number of MOEAs that covers a wide range of experimental scenarios. A novelty of this study is the separation between the higher-level algorithmic components related to multi-objective optimization (MO), which characterize each particular MOEA, and the underlying parameters-such as evolutionary operators, population size, etc.-whose configuration may be tuned for each scenario. Instead of relying on a common or "default" parameter configuration that may be low-performing for particular MOEAs or scenarios and unintentionally biased, we tune the parameters of each MOEA for each scenario using automatic algorithm configuration methods. Our results confirm some of the assumed knowledge in the field, while at the same time they provide new insights on the relative performance of MOEAs for many-objective problems. For example, under certain conditions, indicator-based MOEAs are more competitive for such problems than previously assumed. We also analyze problem-specific features affecting performance, the agreement between performance metrics, and the improvement of tuned configurations over the default configurations used in the literature. Finally, the data produced is made publicly available to motivate further analysis and a baseline for future comparisons.

Automatic Component-Wise Design of Multiobjective Evolutionary Algorithms

Multiobjective evolutionary algorithms (MOEAs) are typically proposed, studied, and applied as monolithic blocks with a few numerical parameters that need to be set. Few works have studied how the algorithmic components of these evolutionary algorithms can be classified and combined to produce new algorithmic designs. The motivation for studies of this latter type stem from the development of flexible software frameworks and the usage of automatic algorithm configuration methods to find novel algorithm designs. In this paper, we propose anMOEA template and a new conceptual view of its components that surpasses existing frameworks in both number of algorithms that can be instantiated from the template and flexibility to produce novel algorithmic designs. We empirically demonstrate the flexibility of our proposed framework by automatically designing MOEAs for continuous and combinatorial optimization problems. The automatically designed algorithms are often able to outperform six traditional MOEAs from the literature, even after tuning their numerical parameters.