Click-through Rate Prediction Research Articles

GraphHINGE: Learning Interaction Models of Structured Neighborhood on Heterogeneous Information Network

Heterogeneous information network (HIN) has been widely used to characterize entities of various types and their complex relations. Recent attempts either rely on explicit path reachability to leverage path-based semantic relatedness or graph neighborhood to learn heterogeneous network representations before predictions. These weakly coupled manners overlook the rich interactions among neighbor nodes, which introduces an early summarization issue. In this article, we propose GraphHINGE ( H eterogeneous IN teract and aggre G at E ), which captures and aggregates the interactive patterns between each pair of nodes through their structured neighborhoods. Specifically, we first introduce Neighborhood-based Interaction (NI) module to model the interactive patterns under the same metapaths, and then extend it to Cross Neighborhood-based Interaction (CNI) module to deal with different metapaths. Next, in order to address the complexity issue on large-scale networks, we formulate the interaction modules via a convolutional framework and learn the parameters efficiently with fast Fourier transform. Furthermore, we design a novel neighborhood-based selection (NS) mechanism, a sampling strategy, to filter high-order neighborhood information based on their low-order performance. The extensive experiments on six different types of heterogeneous graphs demonstrate the performance gains by comparing with state-of-the-arts in both click-through rate prediction and top-N recommendation tasks.

Upper confidence bound integrated genetic algorithm‐optimized long short‐term memory network for click‐through rate prediction

AbstractData on online advertising is rising rapidly due to the fast development of science and technology. Click‐through rate (CTR) prediction has become a critical task regarding the digital advertising industry and a key element in increasing advertising profits and user experience. Therefore, this article describes the problem of CTR prediction as a function of sequence classification tasks. Then, we proposed a novel optimization strategy to solve the high‐dimensional problem and find a subset of relevant variables to ensure high performance of our model and maximize the number of clicks. Here, we introduced a feature selection and hyper‐parameter optimization approach using genetic algorithms (GA) and the upper confidence bound (UCB) model to optimize micro‐targeting technology, along with the long short‐term memory (LSTM) network‐based CTR prediction model. The efficiency of the proposed UCB‐LSTM‐GA model and two hybrid models, namely LSTM‐GA and LSTM‐PSO, is evaluated by comparing them to each other and to other machine‐learning‐based classification methods, including LSTM using a UCB algorithm (UCB‐LSTM), High‐order Attentive Factorization Machine (HoAFM), genetic algorithm‐artificial neural network (GA‐ANN), and a feature interaction graph neural network model (Fi‐GNN). Our solution achieved as high as 87%, 89%, and 92% for respectively accuracy, precision, and recall, using the popular python tools with real Avazu datasets.

CAN: Effective cross features by global attention mechanism and neural network for ad click prediction

Online advertising click-through rate (CTR) prediction is aimed at predicting the probability of a user clicking an ad, and it has undergone considerable development in recent years. One of the hot topics in this area is the construction of feature interactions to facilitate accurate prediction. Factorization machine provides second-order feature interactions by linearly multiplying hidden feature factors. However, real-world data present a complex and nonlinear structure. Hence, second-order feature interactions are unable to represent cross information adequately. This drawback has been addressed using deep neural networks (DNNs), which enable high-order nonlinear feature interactions. However, DNN-based feature interactions cannot easily optimize deep structures because of the absence of cross information in the original features. In this study, we propose an effective CTR prediction algorithm called CAN, which explicitly exploits the benefits of attention mechanisms and DNN models. The attention mechanism is used to provide rich and expressive low-order feature interactions and facilitate the optimization of DNN-based predictors that implicitly incorporate high-order nonlinear feature interactions. The experiments using two real datasets demonstrate that our proposed CAN model performs better than other cross feature- and DNN-based predictors.

Usability and Security Testing of Online Links: A Framework for Click-Through Rate Prediction Using Deep Learning

The user, usage, and usability (3U’s) are three principal constituents for cyber security. The effective analysis of the 3U data using artificial intelligence (AI) techniques allows to deduce valuable observations, which allow domain experts to design practical strategies to alleviate cyberattacks and ensure decision support. Many internet applications, such as internet advertising and recommendation systems, rely on click-through rate (CTR) prediction to anticipate the possibility that a user would click on an ad or product, which is key for understanding human online behaviour. However, online systems are prone to click on fraud attacks. We propose a Human-Centric Cyber Security (HCCS) model that additionally includes AI techniques targeted at the key elements of user, usage, and usability. As a case study, we analyse a CTR prediction task, using deep learning methods (factorization machines) to predict online fraud through clickbait. The results of experiments on a real-world benchmark Avazu dataset show that the proposed approach outpaces (AUC is 0.8062) other CTR forecasting approaches, demonstrating the viability of the proposed framework.

Multi-scale and multi-channel neural network for click-through rate prediction

Click-through rate (CTR) prediction aims to predict the probability of a user clicking on items and ads. Feature engineering plays an important role in improving the accuracy of CTR predict ion because features are normally multi-field. Recently, existing neural network based approaches mostly focus on the feature interactions in both low- and high-order. However, this methodology ignores salient features in both local and global contexts and diverse semantic features in various aspects. In this paper, we propose a novel framework called Multi-Scale and Multi-Channel neural network (MSMC) to learn the feature importance and feature semantics for enhancing CTR prediction. MSMC consists of two parallel modules: a salient feature encoder (SFE) and a diverse semantic feature encoder (DFE). The SFE employs an attentive global–local contexts module to extract the salient features. The DFE uses an attentive semantic module to aggregate the diverse semantic features. Then, a fusion function is adopted to adaptively combine features from the SFE and the DFE so as to obtain high-order feature interactions. Experimental results on two public real datasets illustrate the effectiveness of MSMC compared to the state-of-the-art CTR prediction methods. Our extensive analysis of MSMC shows how the salient features, diverse semantic features, and fusion function positively impact the performance of CTR prediction.

Session interest model for CTR prediction based on self-attention mechanism

Click-through rate prediction, which aims to predict the probability of the user clicking on an item, is critical to online advertising. How to capture the user evolving interests from the user behavior sequence is an important issue in CTR prediction. However, most existing models ignore the factor that the sequence is composed of sessions, and user behavior can be divided into different sessions according to the occurring time. The user behaviors are highly correlated in each session and are not relevant across sessions. We propose an effective model for CTR prediction, named Session Interest Model via Self-Attention (SISA). First, we divide the user sequential behavior into session layer. A self-attention mechanism with bias coding is used to model each session. Since different session interest may be related to each other or follow a sequential pattern, next, we utilize gated recurrent unit (GRU) to capture the interaction and evolution of user different historical session interests in session interest extractor module. Then, we use the local activation and GRU to aggregate their target ad to form the final representation of the behavior sequence in session interest interacting module. Experimental results show that the SISA model performs better than other models.

Click-through rate prediction in online advertising: A literature review

Predicting the probability that a user will click on a specific advertisement has been a prevalent issue in online advertising, attracting much research attention in the past decades. As a hot research frontier driven by industrial needs, recent years have witnessed more and more novel learning models employed to improve advertising CTR prediction. Although extant research provides necessary details on algorithmic design for addressing a variety of specific problems in advertising CTR prediction, the methodological evolution and connections between modeling frameworks are precluded. However, to the best of our knowledge, there are few comprehensive surveys on this topic. We make a systematic literature review on state-of-the-art and latest CTR prediction research, with a special focus on modeling frameworks. Specifically, we give a classification of state-of-the-art CTR prediction models in the extant literature, within which basic modeling frameworks and their extensions, advantages and disadvantages, and performance assessment for CTR prediction are presented. Moreover, we summarize CTR prediction models with respect to the complexity and the order of feature interactions, and performance comparisons on various datasets. Furthermore, we identify current research trends, main challenges and potential future directions worthy of further explorations. This review is expected to provide fundamental knowledge and efficient entry points for IS and marketing scholars who want to engage in this area.

DistilledCTR: Accurate and scalable CTR prediction model through model distillation

Accuracy and scalability are critical to the efficiency and effectiveness of real-time recommender systems. Recent deep learning-based click-through rate prediction models are improving in accuracy but at the expense of computational complexity. The purpose of this study is to propose an accurate and scalable click-through rate (CTR) prediction model for real-time recommendations. This study investigates the complexity, accuracy, and scalability aspects of various CTR models. This work ensembles top CTR models using a gated network and distill into a deep neural network (DNN) using a knowledge distillation framework. Distilled DNN model is more accurate and 20x scalable than any of the individual CTR models. The low latency of distilled model makes it scalable and fit for deployment in real-time recommender systems. The proposed distillation framework is extensible to integrate any CTR models to the ensemble and can be distilled to any neural architecture.

PeerRank: Robust Learning to Rank With Peer Loss Over Noisy Labels

User-generated data are extensively utilized in learning to rank as they are easy to collect and up-to-date. However, the data inevitably contain noisy labels attributed to users’ annotation mistakes, lack of domain knowledge, system failure, etc., making building a robust model challenging. On account of the remarkable nature of deep neural networks in fitting datasets, the noisy labels significantly degrade the performance of learning-to-rank algorithms. To cope with this problem, previous studies have put forward several methods for label de-noising. However, they are either susceptible to the noise distribution on datasets, raising the demand for clean data or incurring more computational costs. Moreover, most of them are tough to extend to different scenarios. This paper proposes a simple yet effective framework named PeerRank that can be applied in broad applications such as click-through rate prediction and commercial web search in learning-to-rank tasks. PeerRank is a robust, effective, and adaptable framework that can couple with numerous models with theoretical guarantees. Extensive experiments on three public real-world datasets with thirteen point-wise base models and four semi-synthetic generation datasets with four pair-wise base models show the consistent improvement of PeerRank. The results comparing PeerRank with seven classic and state-of-the-art de-noising methods validate the advantages of PeerRank framework for learning to rank over noisy labels.

Research on Improvement of the Click-Through Rate Prediction Model Based on Differential Privacy

Click-through rate prediction is crucial in network applications such as recommendation systems and online networks. Designing feature extraction schemes to obtain features and modeling users’ click behavior are used to estimate the probability of users clicking on recommended items. The AutoInt model is a recent and effective research finding. It constructs combined features by referencing the multi-head attention mechanism but does not fully mine meaningful high-order cross-features and ignores user privacy protection. To address this problem, this study proposes the differential privacy bidirectional long short-term memory network (DP-Bi-LSTM-AutoInt) model, which is an improved AutoInt model. A bidirectional long short-term memory network is added after the embedding layer to deeply mine the nonlinear relationship between user click behaviors and construct high-order features. Further, differential privacy technology is adopted for user privacy protection, and the Gaussian mechanism is used to randomly perturb the gradient descent algorithm of the model. Using the Criteo dataset to conduct experiments, the experimental results show that the accuracy of the Bi-LSTM-AutoInt model proposed herein is improved by 0.65 % compared to the original AutoInt model. When the privacy budget is greater than 3.0, the accuracies of the DP-Bi-LSTM-AutoInt and Bi-LSTM-AutoInt models are nearly equivalent. However, the DP-Bi-LSTM-AutoInt model algorithm is more secure and reliable than the AutoInt model.

Click-Through Rate Prediction in Online Advertising: A Literature Review

Click-Through Rate Prediction in Online Advertising: A Literature Review

Research on prediction of user advertising recommendation system based on NFM model

Click-through rate (CTR) predictions can impact business revenue and improve user experience. In large-scale enterprise-level advertising systems and recommendation systems, CTR prediction is a very important link, and it has always been one of the key issues of academic research. In the case of extremely unbalanced positive and negative samples, this paper uses as few positive samples as possible to predict the user's advertising click behavior. To solve the above problems, on the basis of data processing, this paper proposes a CTR prediction model that combines feature engineering and machine learning. In terms of feature engineering, this model selects the relevant features of users and videos from the original data based on the feature selection method, selects important features based on random forest (RF), and forms the interaction features of users and videos by factoring interaction parameters. In terms of machine learning, this model is based on the factorization machine (FM) and the integrated learning model of the deep neural network (DNN), the neural factorization machine (NFM) to realize the click rate prediction model. Based on the NFM model, this paper generates a suitable CTR prediction model through the training of the train data set. After continuous optimization and improvement, the parameters with the best prediction effect are selected. The NFM model established in this paper has a high improvement in performance. Compared with FM, DeepFM, FNN, IFM and DCN models, the AUC value of NFM model is improved by about 0.03 on average. The NFM model can better predict the click behavior of users on advertisements, and can provide decision-making reference for advertisement placement, and can be applied to content recommendation of different advertising systems.

Secure Your Ride: Real-time Matching Success Rate Prediction for Passenger-Driver Pairs

In recent years, online ride-hailing platforms, such as Uber and Didi, have become an indispensable part of urban transportation and make our lives more convenient. After a passenger is matched up with a driver by the platform, both the passenger and the driver have the freedom to simply accept or cancel a ride with one click. Hence, accurately predicting whether a passenger-driver pair is a good match, i.e., its matching success rate (MSR), turns out to be crucial for ride-hailing platforms to devise instant strategies such as order assignment. However, since the users of ride-hailing platforms consist of two parties, decision-making needs to simultaneously account for the dynamics from both the driver and the passenger sides. This makes it more challenging than traditional online advertising tasks that predict a user's response towards an object, e.g., click-through rate prediction for advertisements. Moreover, the amount of available data is severely imbalanced across different cities, creating difficulties for training an accurate model for smaller cities with scarce data. Though a sophisticated neural network architecture can help improve the prediction accuracy under data scarcity, the overly complex design will impede the model's capacity of delivering timely predictions in a production environment. In the paper, to accurately predict the MSR of passenger-driver, we propose the <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>M</b></u> ulti- <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>V</b></u> iew model ( <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MV</b> ) which comprehensively learns the interactions among the dynamic features of the passenger, driver, trip order, as well as the context. Regarding the data imbalance problem, we further design the <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>K</b></u> nowledge <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><b>D</b></u> istillation framework ( <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">KD</b> ) to supplement the model's predictive power for smaller cities using the knowledge from cities with denser data, and also generate a simple model to support efficient deployment. Finally, we conduct extensive experiments on real-world datasets from several different cities, which demonstrates the superiority of our solution.

A CTR Prediction Model With Double Matrix-Level Cross-Features

CTR prediction is an important task in recommender systems, which is used to estimate the likelihood of a user clicking on an advertisement.In the past, the CTR prediction model based on the deep neural network mainly obtains the implicit feature combination of the model at the bit-wise level, and the interpretability and generalization of the model are poor. At the same time, the prediction accuracy of the model is poor. For the above problems, We propose a click-through rate prediction model (DTM) with double matrix-level cross-features. The model integrates various components such as multi-head self-attention, residual network and interaction network into an end-to-end model, and automatically obtains explicit feature combinations at the vector-wise level and bit-wise level, which not only has better interpretability, generalization and memory, and reduce the inherent flaws and engineering complexity of multi-modules. The experimental results show that on the datasets Criteo and Avazu, compared with other state-of-the-art CTR prediction models, the AUC values of the DTM model are increased by 4% and 3% on average, and the loss values are decreased by 3.5% and 2.8% on average, respectively.

Deep context interaction network based on attention mechanism for click-through rate prediction

Click-through rate (CTR) prediction, which aims to predict the probability of a user clicking on an ad, is a critical task in online advertising systems. The problem is very challenging since(1) an effective prediction relies on high-order combinatorial features, and(2)the relationship to auxiliary ads that may impact the CTR. In this paper, we propose Deep Context Interaction Network on Attention Mechanism(DCIN-Attention) to process feature interaction and context at the same time. The context includes other ads in the current search page, historically clicked and unclicked ads of the user. Specifically, we use the attention mechanism to learn the interactions between the target ad and each type of auxiliary ad. The residual network is used to model the feature interactions in the low-dimensional space, and with the multi-head self-attention neural network, high-order feature interactions can be modeled. Experimental results on Avito dataset show that DCIN outperform several existing methods for CTR prediction.

HSR: Hyperbolic Social Recommender

With the prevalence of online social media, users’ social connections have been widely studied and utilized to enhance the performance of recommender systems. In this paper, we explore the use of hyperbolic geometry for social recommendation. We present the Hyperbolic Social Recommender (HSR), a novel social recommendation framework that utilizes hyperbolic geometry to boost the performance. With the help of hyperbolic space, HSR can learn high-quality user and item representations to better model user-item interaction and user-user social relations. Through extensive experiments on four real-world datasets, we show that our proposed HSR outperforms its Euclidean counterpart and state-of-the-art social recommenders in click-through rate prediction and top-K recommendation, demonstrating the effectiveness of social recommendation in the hyperbolic space.

Hierarchical attention and feature projection for click-through rate prediction

Hierarchical attention and feature projection for click-through rate prediction

SELNN: Effective Crossover Features by Attention Mechanism and Logarithmic Conversion for CTR

With the continuous increase of web resources, information overload is becoming more and more serious. Getting the information that meets the needs of a large amount of data has become an urgent problem. One of the popular studies in click-through rate prediction is to construct feature interactions to improve prediction accuracy. Traditional models with low-order interactions cannot adequately represent the intersection information, and deep models with undifferentiated feature intersections lack relevance. In this paper, we propose a model called SELNN, which improves the accuracy of click-through rate prediction through attention mechanism and logarithmic conversion. On the one hand, improving attention mechanisms learns the importance of each feature itself. On the other hand, combining logarithmic conversion structure with feedforward neural networks to learn different orders of feature interactions. The experimental results indicated that the AUC values of SELNN on the Criteo and Movielens-1M datasets were 81.46% and 87.36%, respectively. SELNN effectively improves the prediction accuracy and reduces the number of parameters and computational effort.

Practical differentially private online advertising

Powered by machine learning technology, online advertising achieves accurate advertisement delivery to potential customers according to online user profiles. However, it raises serious privacy concerns since the learning process may reveal sensitive information in the profiles. It is highly desirable to provide high-quality advertisement recommendations while respecting an individual’s privacy.To address the privacy issues, we propose a practical privacy-preserving system that predicts advertisement Click Through Rate (CTR) accurately without revealing any sensitive information of individuals. Our system combines both offline and online training to achieve the goal. In the offline training phase, we develop two differentially private algorithms built upon Gradient Boosting Decision Tree (GBDT) and Field-aware Factorization Machine (FFM) algorithms, respectively, to transform features and train a privacy-preserving offline model. In the online training phase, we propose a privacy-preserving online learning algorithm to compensate for the decline of offline performance when online feature changes occur.We perform extensive experiments to evaluate the performance of our system based on three real-world datasets. The results demonstrate that our system can protect user privacy without compromising the accuracy of CTR prediction, with reasonable computation overhead.

An adaptive hybrid XdeepFM based deep Interest network model for click-through rate prediction system.

Recent advances in communication enable individuals to use phones and computers to access information on the web. E-commerce has seen rapid development, e.g., Alibaba has nearly 12 hundred million customers in China. Click-Through Rate (CTR) forecasting is a primary task in the e-commerce advertisement system. From the traditional Logistic Regression algorithm to the latest popular deep neural network methods that follow a similar embedding and MLP, several algorithms are used to predict CTR. This research proposes a hybrid model combining the Deep Interest Network (DIN) and eXtreme Deep Factorization Machine (xDeepFM) to perform CTR prediction robustly. The cores of DIN and xDeepFM are attention and feature cross, respectively. DIN follows an adaptive local activation unit that incorporates the attention mechanism to adaptively learn user interest from historical behaviors related to specific advertisements. xDeepFM further includes a critical part, a Compressed Interactions Network (CIN), aiming to generate feature interactions at a vectorwise level implicitly. Furthermore, a CIN, plain DNN, and a linear part are combined into one unified model to form xDeepFM. The proposed end-to-end hybrid model is a parallel ensemble of models via multilayer perceptron. CIN and xDeepFM are trained in parallel, and their output is fed into a multilayer perceptron. We used the e-commerce Alibaba dataset with the focal loss as the loss function for experimental evaluation through online complex example mining (OHEM) in the training process. The experimental result indicates that the proposed hybrid model has better performance than other models.