Explicit Sparse Self-Attentive Network for CTR Prediction

Abstract

Abstract The click-through rate (CTR) prediction is an essential task in the recommendation system. Available raw category features and feature combinations are important for CTR prediction, but their sparsity and high-dimension lead to huge computation. In recent years, methods based on deep learning, factorization machines and attention mechanism are widely used to model feature interactions among features for effective sparse predictive analysis. However, these previous algorithms are affected by irrelevant information. In this paper, we propose a novel model ESAtInt to model high-dimensional and sparse features, while being able to explicitly select meaningful higher-order feature interactions and eliminate the impact of irrelevant information. Specifically, we further conduct automatic feature interaction learning through a concentrated attention mechanism with explicit selection. Our model can explicitly select the most relevant parts to emphasize the attention to global features. Experimental results on the real-world dataset show that our proposed method is superior to state-of-the-art approaches for CTR prediction.