Effects of the Random Forests Hyper-Parameters in Surrogate Models for Multi-Objective Combinatorial Optimization: A Case Study using MOEA/D-RFTS

Abstract

Surrogate models are techniques to approximate the objective functions of expensive optimization problems. Recently, Random Forests have been studied as a surrogate model technique for combinatorial optimization problems. Nonetheless, Random Forests contain several hyper-parameters that are used to control the prediction process. Despite their importance, research on the effects of these hyper-parameters is scarce. Therefore, this paper performs a systematic investigation of the effects of different combinations of values for the Random Forest hyper-parameters on the approximation of well-known multi-objective combinatorial benchmark problems. The results show that the number of samples to consider when building each tree and the minimum number of samples to be at the leaf node are the two most important hyper-parameters in this context.