An ensemble learning framework for click-through rate prediction based on a reinforcement learning algorithm with parameterized actions

Abstract

Click-through rate (CTR) prediction is essential for targeted advertising systems. Although there have been many studies on CTR prediction and forming some representative models, building a CTR prediction model that performs well under various scenarios remains a formidable challenge. Ensemble learning is a class of methods that improve the final performance by aggregating multiple base models. This paper proposes an ensemble learning framework that can predict the click probability for each ad impression by combining the results of one or more CTR prediction models. The framework utilizes an improved Reinforcement Learning (RL) algorithm that supports parametric operations to dynamically determine the models and their weights that participated in predicting CTR for each ad impression. Specifically, for each ad impression, the agent generates a discrete action and a continuous action vector based on predicted click-through rates by all base models. The discrete action determines the number of base models involved in prediction, while each continuous action decides the weight of each base model. We validate the ensemble learning framework on three benchmark datasets, including iPinYou, Criteo, and Avazu. The experimental results demonstrate that, in most cases, the ensemble learning framework outperforms the base models in terms of AUC and LogLoss metrics. Finally, we give two simplified variants of the ensemble learning framework and discuss their applicability.