EMORL: Effective multi-objective reinforcement learning method for hyperparameter optimization

Abstract

Hyperparameter optimization is critical for the performance of machine learning algorithms. Significant efforts have been dedicated to improve the final accuracy of algorithm by hyperparameter tuning. However, some indicators (such as latency, cpu utilization) are also very important in the actual environment. In this paper, we propose a novel method EMORL (Effective Multi-Objective Reinforcement Learning) based on multi-objective reinforcement learning for hyperparameter optimization to solve the above limitations. Specifically, we extend hyperparameter optimization problem to the reinforcement learning framework and employ an agent to select hyperparameters sequentially, and design a scalarization function that combines accuracy and latency as a multi-objective reward to guide the policy update. To improve the efficiency of hyperparameter optimization, previously successful configuration is reused for reshaping the advantage function. In the experiment, we apply the proposed method to tune the hyperparameters of the eXtreme Gradient Boosting on 101 tasks and convolutional neural networks on 2 tasks. The experimental results demonstrate that the proposed method is better than other methods in most tasks, especially in terms of latency. In addition, we verify the various components of the proposed method through ablation experiments.