Assessing Algorithm Parameter Importance Using Global Sensitivity Analysis

Abstract

In general, biologically-inspired multi-objective optimization algorithms comprise several parameters which values have to be selected ahead of running the algorithm. In this paper we describe a global sensitivity analysis framework that enables a better understanding of the effects of parameters on algorithm performance. For this work, we tested NSGA-III and MOEA/D on multi-objective optimization testbeds, undertaking our proposed sensitivity analysis techniques on the relevant metrics, namely Generational Distance, Inverted Generational Distance, and Hypervolume. Experimental results show that both algorithms are most sensitive to the cardinality of the population. In all analyses, two clusters of parameter usually appear: (1) the population size (Pop) and (2) the Crossover Distribution Index, Crossover Probability, Mutation Distribution Index and Mutation Probability; where the first cluster, Pop, is the most important (sensitive) parameter with respect to the others. Choosing the correct population size for the tested algorithms has a significant impact on the solution accuracy and algorithm performance. It was already known how important the population of an evolutionary algorithm was, but it was not known its importance compared to the remaining parameters. The distance between the two clusters shows how crucial the size of the population is, compared to the other parameters. Detailed analysis clearly reveals a hierarchy of parameters: on the one hand the size of the population, on the other the remaining parameters that are always grouped together (in a single cluster) without a possible significant distinction. In fact, the other parameters all have the same importance, a secondary relevance for the performance of the algorithms, something which, to date, has not been observed in the evolutionary algorithm literature. The methodology designed in this paper can be adopted to evaluate the importance of the parameters of any algorithm.