Reinforcement Learning for Model Selection and Hyperparameter Optimization

Abstract

With the development of machine learning technology, the number of machine learning algorithms grows rapidly and the models become more and more complex. That causes two major problems in practice: the selection of machine learning models and the hyperparameter optimization. In order to tackle these issues, this paper proposes a new method based on deep reinforcement learning. Long short-term memory (LSTM) network is used to build an agent which automatically selects the machine learning model and optimizes hyperparameters for a given dataset. The agent aims to maximize the accuracy of the selected machine learning model on the validation dataset. At each iteration, it utilizes the accuracy of the selected model on the validation dataset as a reward signal to improve its decision for the next time. The reinforcement learning algorithm is used to guide the learning process for the agent. To verify the idea, the proposed method is compared with two widely optimization methods, tree-structured Parzen estimator and random search on UCI datasets. The results show that the proposed method outperforms other methods in terms of stability, time efficiency and accuracy.