Unbalanced budget distribution for automatic algorithm configuration

Abstract

Optimization algorithms often have several critical setting parameters and the improvement of the empirical performance of these algorithms depends on tuning them. Manually configuration of such parameters is a tedious task that results in unsatisfactory outputs. Therefore, several automatic algorithm configuration frameworks have been proposed to regulate the parameters of a given algorithm for a series of problem instances. Although the developed frameworks perform very well to deal with various problems, however, there is still a trade-off between the accuracy and budget requirements that need to be addressed. This work investigates the performance of unbalanced distribution of budget for different configurations to deal with the automatic algorithm configuration problem. Inspired by the bandit-based approaches, the main goal is to find a better configuration that substantially improves the performance of the target algorithm while using a smaller run time budget. In this work, non-dominated sorting genetic algorithm II is employed as a target algorithm using jMetalPy software platform and the multimodal multi-objective optimization (MMO) test suite of CEC’2020 is used as a set of test problems. We did a comprehensive comparison with other known methods including random search, Bayesian optimization, sequential model-based algorithm configuration (SMAC), iterated local search in parameter configuration space (ParamILS), iterated racing for automatic algorithm configuration (irace), and many-objective automatic algorithm configuration (MAC) methods. In order to characterize, validate and evaluate the performance of these methods, hypervolume (HV), generational distance, and epsilon indicator (\(I_{{\epsilon }^{+}}\)) are used as performance indicators. The experimental results interestingly proved the efficiency of the proposed approach for automatic algorithm configuration with a minimum time budget in comparison with other competitors.