Click-through Rate Prediction Research Articles

Click-through rate prediction model based on graph networks and feature squeeze-and-excitation mechanism

PurposePredicting a user’s click-through rate on an advertisement or item often uses deep learning methods to mine hidden information in data features, which can provide users with more accurate personalized recommendations. However, existing works usually ignore the problem that the drift of user interests may lead to the generation of new features when they compute feature interactions. Based on this, this paper aims to design a model to address this issue.Design/methodology/approachFirst, the authors use graph neural networks to model users’ interest relationships, using the existing user features as the node features of the graph neural networks. Second, through the squeeze-and-excitation network mechanism, the user features and item features are subjected to squeeze operation and excitation operation, respectively, and the importance of the features is adaptively adjusted by learning the channel weights of the features. Finally, the feature space is divided into multiple subspaces to allocate features to different models, which can improve the performance of the model.FindingsThe authors conduct experiments on two real-world data sets, and the results show that the model can effectively improve the prediction accuracy of advertisement or item click events.Originality/valueIn the study, the authors propose graph network and feature squeeze-and-excitation model for click-through rate prediction, which is used to dynamically learn the importance of features. The results indicate the effectiveness of the model.

A Lambda Layer-Based Convolutional Sequence Embedding Model for Click-Through Rate Prediction

In the era of intelligent economy, the click-through rate (CTR) prediction system can evaluate massive service information based on user historical information, and screen out the products that are most likely to be favored by users, thus realizing customized push of information and achieve the ultimate goal of improving economic benefits. Sequence modeling is one of the main research directions of CTR prediction models based on deep learning. The user's general interest hidden in the entire click history and the short-term interest hidden in the recent click behaviors have different influences on the CTR prediction results, which are highly important. In terms of capturing the user's general interest, existing models paid more attention to the relationships between item embedding vectors (point-level), while ignoring the relationships between elements in item embedding vectors (union-level). The Lambda layer-based Convolutional Sequence Embedding (LCSE) model proposed in this paper uses the Lambda layer to capture features from click history through weight distribution, and uses horizontal and vertical filters on this basis to learn the user's general preferences from union-level and point-level. In addition, we also incorporate the user's short-term preferences captured by the embedding-based convolutional model to further improve the prediction results. The AUC (Area Under Curve) values of the LCSE model on the datasets Electronic, Movie & TV and MovieLens are 0.870 7, 0.903 6 and 0.946 7, improving 0.45%, 0.36% and 0.07% over the Caser model, proving the effectiveness of our proposed model.

Feature-Interaction-Enhanced Sequential Transformer for Click-Through Rate Prediction

Click-through rate (CTR) prediction plays a crucial role in online services and applications, such as online shopping and advertising. The performance of CTR prediction can have a direct impact on user experience and the revenue of the online platforms. For CTR prediction models, self-attention-based methods have been widely applied to this field. Recent works generally adopted the Transformer architecture, where the self-attention mechanism can capture the global dependencies of the user’s historical interactions and predict the next item. Despite the effectiveness of self-attention methods in modeling sequential user behaviors, most sequential recommenders hardly exploit feature interaction techniques to extract high-order feature combinations. In this paper, we propose a Feature-Interaction-Enhanced Sequence Model (FESeq), which integrates feature interaction and the sequential recommendation model in a cascading structure. Specifically, the interacting layer in FESeq is an automatic feature engineering step for the Transformer model. Then, we add a linear time interval embedding layer and a positional embedding layer to the Transformer in the sequence-refiner layer to learn both the time intervals and the position information in the user’s sequence behaviors. We also design an attention-based sequence pooling layer that can model the relevance of the user’s historical behaviors and the target item representation through scaled bilinear attention. Our experiments show that the proposed method beats all the baselines on both public and industrial datasets.

Generalize for Future: Slow and Fast Trajectory Learning for CTR Prediction

Deep neural networks (DNNs) have achieved significant advancements in click-through rate (CTR) prediction by demonstrating strong generalization on training data. However, in real-world scenarios, the assumption of independent and identically distributed (i.i.d.) conditions, which is fundamental to this problem, is often violated due to temporal distribution shifts. This violation can lead to suboptimal model performance when optimizing empirical risk without access to future data, resulting in overfitting on the training data and convergence to a single sharp minimum. To address this challenge, we propose a novel model updating framework called Slow and Fast Trajectory Learning (SFTL) network. SFTL aims to mitigate the discrepancy between past and future domains while quickly adapting to recent changes in small temporal drifts. This mechanism entails two interactions among three complementary learners: (i) the Working Learner, which updates model parameters using modern optimizers (e.g., Adam, Adagrad) and serves as the primary learner in the recommendation system, (ii) the Slow Learner, which is updated in each temporal domain by directly assigning the model weights of the working learner, and (iii) the Fast Learner, which is updated in each iteration by assigning exponentially moving average weights of the working learner. Additionally, we propose a novel rank-based trajectory loss to facilitate interaction between the working learner and trajectory learner, aiming to adapt to temporal drift and enhance performance in the current domain compared to the past. We provide theoretical understanding and conduct extensive experiments on real-world CTR prediction datasets to validate the effectiveness and efficiency of SFTL in terms of both convergence speed and model performance. The results demonstrate the superiority of SFTL over existing approaches.

AT4CTR: Auxiliary Match Tasks for Enhancing Click-Through Rate Prediction

Click-through rate (CTR) prediction is a vital task in industrial recommendation systems. Most existing methods focus on the network architecture design of the CTR model for better accuracy and suffer from the data sparsity problem. Especially in industrial recommendation systems, the widely applied negative sample down-sampling technique due to resource limitation worsens the problem, resulting in a decline in performance. In this paper, we propose Auxiliary Match Tasks for enhancing Click-Through Rate (AT4CTR) prediction accuracy by alleviating the data sparsity problem. Specifically, we design two match tasks inspired by collaborative filtering to enhance the relevance modeling between user and item. As the "click" action is a strong signal which indicates the user's preference towards the item directly, we make the first match task aim at pulling closer the representation between the user and the item regarding the positive samples. Since the user's past click behaviors can also be treated as the user him/herself, we apply the next item prediction as the second match task. For both the match tasks, we choose the InfoNCE as their loss function. The two match tasks can provide meaningful training signals to speed up the model's convergence and alleviate the data sparsity. We conduct extensive experiments on one public dataset and one large-scale industrial recommendation dataset. The result demonstrates the effectiveness of the proposed auxiliary match tasks. AT4CTR has been deployed in the real industrial advertising system and has gained remarkable revenue.

Multi-Domain Deep Learning from a Multi-View Perspective for Cross-Border E-commerce Search

Building click-through rate (CTR) and conversion rate (CVR) prediction models for cross-border e-commerce search requires modeling the correlations among multi-domains. Existing multi-domain methods would suffer severely from poor scalability and low efficiency when number of domains increases. To this end, we propose a Domain-Aware Multi-view mOdel (DAMO), which is domain-number-invariant, to effectively leverage cross-domain relations from a multi-view perspective. Specifically, instead of working in the original feature space defined by different domains, DAMO maps everything to a new low-rank multi-view space. To achieve this, DAMO firstly extracts multi-domain features in an explicit feature-interactive manner. These features are parsed to a multi-view extractor to obtain view-invariant and view-specific features. Then a multi-view predictor inputs these two sets of features and outputs view-based predictions. To enforce view-awareness in the predictor, we further propose a lightweight view-attention estimator to dynamically learn the optimal view-specific weights w.r.t. a view-guided loss. Extensive experiments on public and industrial datasets show that compared with state-of-the-art models, our DAMO achieves better performance with lower storage and computational costs. In addition, deploying DAMO to a large-scale cross-border e-commence platform leads to 1.21%, 1.76%, and 1.66% improvements over the existing CGC-based model in the online AB-testing experiment in terms of CTR, CVR, and Gross Merchandises Value, respectively.

Fusion of GBDT and neural network for click-through rate estimation

Aiming at the issue that the current click-through rate prediction methods ignore the varying impacts of different input features on prediction accuracy and exhibit low accuracy when dealing with large-scale data, a click-through rate prediction method (GBIFM) which combines Gradient Boosting Decision Tree (GBDT) and Input-aware Factorization Machine (IFM) is proposed in this paper. The proposed GBIFM method employs GBDT for data processing, which can flexibly handle various types of data without the need for one-hot encoding of discrete features. An Input-aware strategy is introduced to refine the weight vector and embedding vector of each feature for different instances, adaptively learning the impact of each input vector on feature representation. Furthermore, a fully connected network is incorporated to capture high-order features in a non-linear manner, enhancing the method’s ability to express and generalize complex structured data. A comprehensive experiment is conducted on the Criteo and Avazu datasets, the results show that compared to typical methods such as DeepFM, AFM, and IFM, the proposed method GBIFM can increase the AUC value by 10% –12% and decrease the Logloss value by 6% –20%, effectively improving the accuracy of click-through rate prediction.

TMH: Two-Tower Multi-Head Attention neural network for CTR prediction.

Click-through rate (CTR) prediction is a term used to predict the probability of a user clicking on an ad or item and has become a popular research area in advertising. As the volume of Internet data increases, the labor costs of traditional feature engineering continue to rise. To reduce the dependence on feature interactions, this paper proposes a fusion model that combines explicit and implicit feature interactions, called the Two-Tower Multi-Head Attention Neural Network (TMH) approach. The model integrates multiple components such as multi-head attention, residual network, and deep neural networks into an end-to-end model that automatically obtains vector-level combinations of explicit and implicit features to predict click-through rates through higher-order explicit and implicit interactions. We evaluated the effectiveness of TMH in CTR prediction through numerous experiments using three real datasets. The results demonstrate that our proposed method not only outperforms existing prediction methods but also offers good interpretability.

Causal Inference in Recommender Systems: A Survey and Future Directions

Recommender systems have become crucial in information filtering nowadays. Existing recommender systems extract user preferences based on the correlation in data, such as behavioral correlation in collaborative filtering, feature-feature, or feature-behavior correlation in click-through rate prediction. However, unfortunately, the real world is driven by causality , not just correlation, and correlation does not imply causation. For instance, recommender systems might recommend a battery charger to a user after buying a phone, where the latter can serve as the cause of the former; such a causal relation cannot be reversed. Recently, to address this, researchers in recommender systems have begun utilizing causal inference to extract causality, thereby enhancing the recommender system. In this survey, we offer a comprehensive review of the literature on causal inference-based recommendation. Initially, we introduce the fundamental concepts of both recommender system and causal inference as the foundation for subsequent content. We then highlight the typical issues faced by non-causality recommender system. Following that, we thoroughly review the existing work on causal inference-based recommender systems, based on a taxonomy of three-aspect challenges that causal inference can address. Finally, we discuss the open problems in this critical research area and suggest important potential future works.

Click-through rate prediction based on feature interaction and behavioral sequence

Click-through rate prediction based on feature interaction and behavioral sequence

It’s Not Always about Wide and Deep Models: Click-Through Rate Prediction with a Customer Behavior-Embedding Representation

Alongside natural language processing and computer vision, large learning models have found their way into e-commerce. Especially, for recommender systems and click-through rate prediction, these models have shown great predictive power. In this work, we aim to predict the probability that a customer will click on a given recommendation, given only its current session. Therefore, we propose a two-stage approach consisting of a customer behavior-embedding representation and a recurrent neural network. In the first stage, we train a self-supervised skip-gram embedding on customer activity data. The resulting embedding representation is used in the second stage to encode the customer sequences which are then used as input to the learning model. Our proposed approach diverges from the prevailing trend of utilizing extensive end-to-end models for click-through rate prediction. The experiments, which incorporate a real-world industrial use case and a widely used as well as openly available benchmark dataset, demonstrate that our approach outperforms the current state-of-the-art models. Our approach predicts customers’ click intention with an average F1 accuracy of 94% for the industrial use case which is one percentage point higher than the state-of-the-art baseline and an average F1 accuracy of 79% for the benchmark dataset, which outperforms the best tested state-of-the-art baseline by more than seven percentage points. The results show that, contrary to current trends in that field, large end-to-end models are not always needed. The analysis of our experiments suggests that the reason for the performance of our approach is the self-supervised pre-trained embedding of customer behavior that we use as the customer representation.

PeNet: A feature excitation learning approach to advertisement click-through rate prediction

Since the physical meaning of the fields of the dataset is unknown, we have to use the feature interaction method to select the correlated features and exclude uncorrelated features. The current state-of-the-art methods employ various methods based on feature interaction to predict advertisement Click-Through Rate (CTR); however, the feature interaction based on potential new feature mining is rarely considered, which can provide effective assistance for feature interaction. This motivates us to investigate methods that combine potential new features and feature interactions. Thus, we propose a potential feature excitation learning network (PeNet), which is a neural network model based on feature combination and feature interaction. In PeNet, we treat the row compression and column compression of the original feature matrix as potential new features, and proposed the excitation learning mechanism that is a weighted mechanism based on residual principle. Through this excitation learning mechanism, the original embedded features and potential new features are subjected to weighted interaction based on the residual principle. Moreover, a deep neural network is exploited to iteratively learn and iteratively combine features. The excitation learning structure of PeNet neural network is well demonstrated in this paper, that is, the control flow of embedding, compression, excitation and output, which further strengthens the correlated features and weakens the uncorrelated features by compressing and expanding the features. Experimental results on multiple benchmark datasets indicate the PeNet as a general-purpose plug-in has more superior performance and better efficiency than previous state-of-the-art methods.

Disentangled self-attention neural network based on information sharing for click-through rate prediction.

With the exponential growth of network resources, recommendation systems have become successful at combating information overload. In intelligent recommendation systems, the prediction of click-through rates (CTR) plays a crucial role. Most CTR models employ a parallel network architecture to successfully capture explicit and implicit feature interactions. However, the existing models ignore two aspects. One limitation observed in most models is that they focus only on the interaction of paired term features, with no emphasis on modeling unary terms. The second issue is that most models input characteristics indiscriminately into parallel networks, resulting in network input oversharing. We propose a disentangled self-attention neural network based on information sharing (DSAN) for CTR prediction to simulate complex feature interactions. Firstly, an embedding layer transforms high-dimensional sparse features into low-dimensional dense matrices. Then, the disentangled multi-head self-attention learns the relationship between different features and is fed into a parallel network architecture. Finally, we set up a shared interaction layer to solve the problem of insufficient information sharing in parallel networks. Results from experiments conducted on two real-world datasets demonstrate that our proposed method surpasses existing methods in predictive accuracy.

SIIR: Symmetrical Information Interaction Modeling for News Recommendation.

Accurate matching between user and candidate news plays a fundamental role in news recommendation. Most existing studies capture fine-grained user interests through effective user modeling. Nevertheless, user interest representations are often extracted from multiple history news items, while candidate news representations are learned from specific news items. The asymmetry of information density causes invalid matching of user interests and candidate news, which severely affects the click-through rate prediction for specific candidate news. To resolve the problems mentioned above, we propose a symmetrical information interaction modeling for news recommendation (SIIR) in this article. We first design a light interactive attention network for user (LIAU) modeling to extract user interests related to the candidate news and reduce interference of noise effectively. LIAU overcomes the shortcomings of complex structure and high training costs of conventional interaction-based models and makes full use of domain-specific interest tendencies of users. We then propose a novel heterogeneous graph neural network (HGNN) to enhance candidate news representation through the potential relations among news. HGNN builds a candidate news enhancement scheme without user interaction to further facilitate accurate matching with user interests, which mitigates the cold-start problem effectively. Experiments on two realistic news datasets, i.e., MIND and Adressa, demonstrate that SIIR outperforms the state-of-the-art (SOTA) single-model methods by a large margin.

PREDICTIVE MODELING OF CLICK-THROUGH RATES: A REGRESSION ANALYSIS APPROACH

This research uses advanced regression techniques to develop a robust predictive model for Click-Through Rates (CTR) in online advertising. The study leverages a diverse dataset encompassing various advertising campaigns and user interactions to uncover patterns and relationships influencing click-through behavior. The goal is to provide advertisers with a tool for accurate CTR prediction, enabling them to optimize campaigns and allocate resources effectively.

DMMP: A distillation-based multi-task multi-tower learning model for personalized recommendation

Recommendation systems involve the matching and ranking stages. The click-through rate (CTR) and the conversion rate (CVR) predictions are two fundamental modules in recommendation systems. Most candidate generation matching models leverage a two-tower architecture to model the CTR prediction task. However, items with low-quality but attractive titles, i.e., click baits, may be recommended to the user, which worsens the user’s experience. Therefore, both click and conversion tasks should be modeled during the matching stage to improve user engagement. An intuitive way is to model these two tasks with a three-tower matching model. However, its efficiency and effectiveness are limited. By inheriting the merits of knowledge distillation and multi-task learning, we propose a distillation-based multi-task multi-tower model (DMMP) for personalized recommendation. Specifically, the MTL-based teacher network builds the task-shared and task-specific expert networks, and employs a customized multi-gate control network to merge these expert networks adaptively within the task embedding layer. Two auxiliary pCTR and pCTCVR tasks are incorporated to model CVR directly across the whole space. By sharing the feature representation parameters with the CTR modeling, the CVR modeling can be trained with richer samples. Three-tower-based candidate generation student network incorporates a user preference gating network to learn the task scores so that personalized candidates can be produced. Furthermore, we design an adaptive weighting strategy for total loss to eventually adjust the importance and relevance of different networks. Extensive experiments on public and industrial datasets validate the effectiveness of DMMP.

An ensemble learning framework for click-through rate prediction based on a reinforcement learning algorithm with parameterized actions

Click-through rate (CTR) prediction is essential for targeted advertising systems. Although there have been many studies on CTR prediction and forming some representative models, building a CTR prediction model that performs well under various scenarios remains a formidable challenge. Ensemble learning is a class of methods that improve the final performance by aggregating multiple base models. This paper proposes an ensemble learning framework that can predict the click probability for each ad impression by combining the results of one or more CTR prediction models. The framework utilizes an improved Reinforcement Learning (RL) algorithm that supports parametric operations to dynamically determine the models and their weights that participated in predicting CTR for each ad impression. Specifically, for each ad impression, the agent generates a discrete action and a continuous action vector based on predicted click-through rates by all base models. The discrete action determines the number of base models involved in prediction, while each continuous action decides the weight of each base model. We validate the ensemble learning framework on three benchmark datasets, including iPinYou, Criteo, and Avazu. The experimental results demonstrate that, in most cases, the ensemble learning framework outperforms the base models in terms of AUC and LogLoss metrics. Finally, we give two simplified variants of the ensemble learning framework and discuss their applicability.

A novel graph-based feature interaction model for click-through rate prediction

A novel graph-based feature interaction model for click-through rate prediction

Click-through conversion rate prediction model of book e-commerce platform based on feature combination and representation

The prediction of click-through conversion rates has always been an integral focus within e-commerce platforms. This paper proposes a novel prediction model that leverages feature combination and representation learning techniques, centering around three crucial elements of the Buyer-Book-Activity (BBA) e-commerce platform. Specifically, our approach considers the object characteristics of book e-commerce platforms. Firstly, we create a comprehensive three-part BBA diagram encompassing user-book-marketing activities and thoroughly describe the book e-commerce platform’s information space. To construct a comprehensive feature space, we employ a factor decomposition method, enabling us to combine and mine large-scale information from the BBA. In particular, we introduce book2tag text features to capture textual information’s hidden richness and decompose it. Secondly, to represent the overall information space of the book e-commerce platform, we propose the BBA2vec representation learning method. This method captures the complexity and diversity of the BBA’s effective data feature space. This is crucial as feature combinations alone may not fully capture the diversity of e-commerce features. Finally, a deep neural network is used to predict click-through conversion rates by combining feature combination and feature representation. The experimental results show that the model accurately predicts the bookstore e-commerce platform’s click-purchase conversion rate.

CFF: combining interactive features and user interest features for click-through rate prediction

CFF: combining interactive features and user interest features for click-through rate prediction