DistilledCTR: Accurate and scalable CTR prediction model through model distillation

Abstract

Accuracy and scalability are critical to the efficiency and effectiveness of real-time recommender systems. Recent deep learning-based click-through rate prediction models are improving in accuracy but at the expense of computational complexity. The purpose of this study is to propose an accurate and scalable click-through rate (CTR) prediction model for real-time recommendations. This study investigates the complexity, accuracy, and scalability aspects of various CTR models. This work ensembles top CTR models using a gated network and distill into a deep neural network (DNN) using a knowledge distillation framework. Distilled DNN model is more accurate and 20x scalable than any of the individual CTR models. The low latency of distilled model makes it scalable and fit for deployment in real-time recommender systems. The proposed distillation framework is extensible to integrate any CTR models to the ensemble and can be distilled to any neural architecture.