Click-through Rate Estimation Research Articles

Improving CTR Prediction in Advertising with XGBoost

Click Through Rate (CTR) prediction is crucial in digital advertising for optimizing marketing strategies. This paper presents a review of significant contributions in this field, highlighting methodologies and findings from various studies. Pioneering research laid foundational groundwork for CTR estimation methods, while subsequent analyses explored the impact of ad types and design effects on user engagement. Utilization of data mining techniques and the proposal of advanced prediction models further enhanced CTR prediction accuracy. Additionally, this paper introduces our method utilizing XGBoost, a powerful ensemble learning algorithm, to address existing challenges and enhance CTR prediction accuracy. This review offers valuable insights for marketers aiming to optimize their advertising campaigns in the dynamic landscape of advertising.

Fusion of GBDT and neural network for click-through rate estimation

Aiming at the issue that the current click-through rate prediction methods ignore the varying impacts of different input features on prediction accuracy and exhibit low accuracy when dealing with large-scale data, a click-through rate prediction method (GBIFM) which combines Gradient Boosting Decision Tree (GBDT) and Input-aware Factorization Machine (IFM) is proposed in this paper. The proposed GBIFM method employs GBDT for data processing, which can flexibly handle various types of data without the need for one-hot encoding of discrete features. An Input-aware strategy is introduced to refine the weight vector and embedding vector of each feature for different instances, adaptively learning the impact of each input vector on feature representation. Furthermore, a fully connected network is incorporated to capture high-order features in a non-linear manner, enhancing the method’s ability to express and generalize complex structured data. A comprehensive experiment is conducted on the Criteo and Avazu datasets, the results show that compared to typical methods such as DeepFM, AFM, and IFM, the proposed method GBIFM can increase the AUC value by 10% –12% and decrease the Logloss value by 6% –20%, effectively improving the accuracy of click-through rate prediction.

Learning to Retrieve User Behaviors for Click-through Rate Estimation

Click-through rate (CTR) estimation plays a crucial role in modern online personalization services. It is essential to capture users’ drifting interests by modeling sequential user behaviors to build an accurate CTR estimation model. However, as the users accumulate a large amount of behavioral data on the online platforms, the current CTR models have to truncate user behavior sequences and utilize the most recent behaviors, which leads to a problem that sequential patterns such as periodicity or long-term dependency are not contained in the recent behaviors but in far back history. However, it is non-trivial to model the entire user sequence by directly using it for two reasons. Firstly, the very long input sequences will make online inference time and system load infeasible. Secondly, the very long sequences contain much noise, thus making it difficult for CTR models to capture useful patterns effectively. To tackle this issue, we consider it from the input data perspective instead of designing more sophisticated yet complex models. As the entire user behavior sequence contains much noise, it is unnecessary to input the entire sequence. Instead, we could just retrieve only a small part of it as the input to the CTR model. In this article, we propose the U ser B ehavior R etrieval (UBR) framework which aims at learning to retrieve the most informative user behaviors according to each CTR estimation request. Retrieving only a small set of behaviors could alleviate the two problems of utilizing very long sequences (i.e., inference efficiency and noisy input). The distinguishing property of UBR is that it supports arbitrary and learnable retrieval functions instead of utilizing a fixed pre-defined function, which is different from the current retrieval-based methods. Offline evaluations on three large-scale real-world datasets demonstrate the superiority and efficacy of the UBR framework. We further deploy UBR at the Huawei App Store, where it achieves 6.6% of eCPM gain in the online A/B test and now serves the main traffic in the Huawei App Store advertising scenario.

Click-through rate prediction model integrating user interest and multi-head attention mechanism

The purpose of click-through rate (CTR) prediction is to anticipate how likely a person is to click on an advertisement or item. It's required for a lot of internet applications, such online advertising and recommendation systems. The previous click-through rate estimation approach suffered from the following two flaws. On the one hand, input characteristics (such as user id, user age, user age, item id, item category) are usually sparse and multidimensional, making them effective. High-level combination characteristics are used for prediction. Obtaining it manually by domain experts takes a long time and is difficult to finish; also, customer interests are not all the same. The accuracy of the model findings will significantly increase if this immediately recognized component is incorporated in the prediction model. As a consequence, this study creates an IARM (interactive attention rate estimation model) that incorporates user interest as well as a multi-head self-attention mechanism. The deep learning network is used in the model to determine the user's interest expression based on user attributes. The multi-head self-attention mechanism with residual network is then employed to get feature interaction, which enhances the degree of effect of significant characteristics on the estimation result as well as its accuracy. The IARM model outperforms other recent prediction models in the assessment metrics AUC and LOSS, and it has superior accuracy, according to the results from the public experimental data set.

Bid-Aware Active Learning in Real-Time Bidding for Display Advertising

In Real-time Bidding (RTB) based display advertising, demand side platforms (DSPs) estimate the click-through rate (CTR) of each advertisement impression, and then decide whether and how much to bid based on the information of the user and the advertiser. Typically, when a new campaign is launched, the CTR estimation module of the DSP needs to collect data to train an accurate estimator. The advertiser is charged for each ad impression in display advertising, therefore there is some cost for obtaining each training instance. Thus one crucial task is to actively train an accurate CTR estimator within the constraint of the budget. Traditional active learning algorithms fail to deal with such scenario because (i) acquiring training instances is implemented via performing real-time bidding for the corresponding auctions; (ii) RTB requires the bidding agent to make real-time decisions for sequentially coming bid requests; (iii) cost for each ad impression will be unveiled only after giving the bid price and winning the auction; (iv) training data gathered in post-bid stage has a strong bias towards the won impressions. In this paper, we propose a Bid-aware Active Real-time Bidding (BARB) algorithm to actively choose training instances by setting different bid prices for each ad auction, in order to efficiently train an accurate CTR estimation model within the budget constraint. The empirical study on different campaigns of three real-world datasets with three budget constraints shows the effectiveness of our proposed algorithm.

Predicting ad click-through rates via feature-based fully coupled interaction tensor factorization

In the real-time bidding (RTB) display advertising ecosystem, when receiving a bid request, the demand-side platform (DSP) needs to predict the click-through rate (CTR) for ads and calculate the bid price according to the CTR estimated. In addition to challenges similar to those encountered in sponsored search advertising, such as data sparsity and cold start problems, more complicated feature interactions involving multi-aspects, such as the user, publisher and advertiser, make CTR estimation in RTB more difficult. We consider CTR estimation in RTB as a tensor complement problem and propose a fully coupled interactions tensor factorization (FCTF) model based on Tucker decomposition (TD) to model three pairwise interactions between the user, publisher and advertiser and ultimately complete the tensor complement task. FCTF is a special case of the Tucker decomposition model; however, it is linear in runtime for both learning and prediction. Different from pairwise interaction tensor factorization (PITF), which is another special case of TD, FCTF is independent from the Bayesian personalized ranking optimization algorithm and is applicable to generic third-order tensor decomposition with popular simple optimizations, such as the least square method or mean square error. In addition, we also incorporate all explicit information obtained from different aspects into the FCTF model to alleviate the impact of cold start and sparse data on the final performance. We compare the performance and runtime complexity of our method with Tucker decomposition, canonical decomposition and other popular methods for CTR prediction over real-world advertising datasets. Our experimental results demonstrate that the improved model not only achieves better prediction quality than the others due to considering fully coupled interactions between three entities, user, publisher and advertiser but also can accomplish training and prediction with linear runtime.

Online Advertisement, Optimization and Stochastic Networks

In this paper, we propose a stochastic model to describe how search service providers charge client companies based on users' queries for the keywords related to these companies' ads by using certain advertisement assignment strategies. We formulate an optimization problem to maximize the long-term average revenue for the service provider under each client's long-term average budget constraint and design an online algorithm which captures the stochastic properties of users' queries and click-through behaviors. We solve the optimization problem by making connections to scheduling problems in wireless networks, queueing theory and stochastic networks. Unlike prior models, we do not assume that the number of query arrivals is known. Due to the stochastic nature of the arrival process considered here, either temporary “free” service, i.e., service above the specified budget (which we call “overdraft”) or under-utilization of the budget (which we call “underdraft”) is unavoidable. We prove that our online algorithm can achieve a revenue that is within <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (ε) of the optimal revenue while ensuring that the overdraft or underdraft is <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (1/ε) , where ε can be arbitrarily small. With a view towards practice, we can show that one can always operate strictly under the budget. In addition, we extend our results to a click-through rate maximization model and also show how our algorithm can be modified to handle non-stationary query arrival processes and clients with short-term contracts. Our algorithm also allows us to quantify the effect of errors in click-through rate estimation on the achieved revenue. We show that we lose at most Δ/1+Δ fraction of the revenue if Δ is the relative error in click-through rate estimation. We also show that in the long run, an expected overdraft level of Ω(log(1/ε)) is unavoidable (a universal lower bound) under any stationary ad assignment algorithm which achieves a long-term average revenue within <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (ε) of the offline optimum.