A New Algorithm for Adaptive Online Selection of Auxiliary Objectives

Abstract

Consider optimization problems, where a target objective should be optimized. Some auxiliary objectives can be used to obtain the optimum of the target objective in less number of objective evaluations. We call such auxiliary objective a supporting one. Usually there is no prior knowledge about properties of auxiliary objectives, some objectives can be obstructive as well. What is more, an auxiliary objective can be both supporting and obstructive at different stages of the target objective optimization. Thus, an adaptive online method of objective selection is needed. Earlier, we proposed a method for doing that, which is based on reinforcement learning. In this paper, a new algorithm for adaptive online selection of optimization objectives is proposed. The algorithm meets the interface of a reinforcement learning agent, so it can be fit into the previously proposed framework. The new algorithm is applied for solving some benchmark problems with single-objective evolutionary algorithms. Specifically, Leading Ones with OneMax auxiliary objective is considered, as well as the MH-IFF problem. Experimental results are presented. The proposed algorithm outperforms Q-learning and random objective selection on the considered problems.