Hierarchical Attention Factorization Machine for CTR Prediction

Abstract

Click-through rate (CTR) prediction is a crucial task in recommender systems and online advertising. The most critical step in this task is to perform feature interaction. Factorization machines are proposed to complete the second-order interaction of features to improve the prediction accuracy, but they are not competent for high-order feature interactions. In recent years, many state-of-the-art models employ shallow neural networks to capture high-order feature interactions to improve prediction accuracy. However, some studies have proven that the addictive feature interactions using feedforward neural networks are inefficient in capturing common high-order feature interactions. To solve this problem, we propose a new way, a hierarchical attention network, to capture high-order feature interactions. Through the hierarchical attention network, we can refine the feature representation of the previous layer at each layer, making it become a new feature representation containing the feature interaction information of the previous layer, and this representation will be refined again through the next layer. We also use shallow neural networks to perform higher-order non-linear interactions on feature interaction terms to further improve prediction accuracy. The experiment results on four real-world datasets demonstrate that our proposed HFM model outperforms state-of-the-art models.