Multi-interest Network Based on Double Attention for Click-Through Rate Prediction

Abstract

Whether in the field of personalized advertising or recommender systems, click through rate (CTR) prediction is a very important task. In recent years, Alibaba Group has done a lot of advanced research on the prediction of click through rate, and proposed a technical route includes kinds of deep learning models. For example, in the deep interest network (DIN) proposed by the Alibaba Group, the sequence of users’ browsing behaviors is used to express their interest features, and this sequence is made up of items clicked by users. Usually, the item is mapped into a static vector, but the fixed length embedded vector is difficult to express the user’s dynamic interest features. In order to solve this problem, Alibaba Group introduced the attention mechanism in the field of natural language processing (NLP) into deep interest network (DIN), and designed a unique activation unit to extract the important informations in the user’s historical behavior sequence, and use these important informations to express user’s dynamic interest features. In this paper, we propose a novel deep learning model: Multi-Interest Network Based on Double Attention for Click-Through Rate Prediction (DAMIN), which based on Deep Interest Network (DIN) and combined with multi-head attention mechanism. In the deep interest network (DIN), the attention weight between the candidate item vector and the item vector of the user’s historical behavior sequence is learned by fully connected neural network. Different from deep interest network (DIN), we design a new method, which uses the reciprocal of Euclidean Distance to represent the attention weight between two item vectors. Then, the item vectors in user’s historical behavior sequence are weighted by the attention weights and meanwhile the candidate item vectors are also weighted by the attention weights. In the next, we can obtain new item vectors by add the weighted item vectors of user’s historical behavior sequence and weighted candidate item vectors, and those new item vectors are used to represent user’s dynamic interest feature vectors. In the end, the user’s dynamic interest features are send into the three multi-head attention layers, which can extract users’ various interest features. We have conducted a lot of experiments on three real-world datasets of Amazon and the results show that the model proposed by this paper acquires a better performance than some classical models. Compared with DIN, the model proposed in this paper improves the average of AUC by 4%–5%, which proves that the model proposed in this paper is effective. In addition, a large number of ablation experiments have been carried out to prove that each module of the proposed model is effective.