Improved Selection of Auxiliary Objectives Using Reinforcement Learning in Non-stationary Environment

Abstract

Efficiency of evolutionary algorithms can be increased by using auxiliary objectives. The method which is called EA+RL is considered. In this method a reinforcement learning (RL) algorithm is used to select objectives in evolutionary algorithms (EA) during optimization. In earlier studies, reinforcement learning algorithms for stationary environments were used in the EA+RL method. However, if behavior of auxiliary objectives change during the optimization process, it can be better to use reinforcement learning algorithms which are specially developed for non-stationary environments. In our previous work we proposed a new reinforcement learning algorithm to be used in the EA+RL method. In this work we propose an improved version of that algorithm. The new algorithm is applied to a non-stationary problem and compared with the methods which were used in other studies. It is shown that the proposed method achieves optimal value more often and obtains higher values of the target objective than the other algorithms.