Real-time Bidding Research Articles

Budget-Constrained Real-Time Bidding Optimization

In this article, we pursue a better solution for the promising problem, i.e., the bidding strategy design, in the real-time bidding (RTB) advertising (AD) environment. Under the budget constraint, the design of an optimal strategy for bidding on each incoming impression opportunity targets at acquiring as many clicks as possible during an AD campaign. State-of-the-art bidding algorithms rely on a single predictor, the clickthrough rate predictor, to calculate the bidding value for each impression. This provides reasonable performance if the predictor has appropriate accuracy in predicting the probability of user clicking. However, the classical methods usually fail to capture optimal results since the predictor accuracy is limited. We improve the situation by accomplishing an additional winning price predictor in the bidding process. In this article, an algorithm combining powers of multiple prediction models is developed. It emerges from an analogy to the online stochastic knapsack problem, and the efficiency of the algorithm is also theoretically analyzed. Experiments conducted on real world RTB datasets show that the proposed solution performs better with regard to both number of clicks achieved and effective cost per click in many different settings of budget constraints.

‘Trust Me, I’m Fair’: Analysing Google’s Latest Practices in Ad Tech From the Perspective of EU Competition Law

In a first paper released in January 2019, we explained the mechanics of online display advertising and real-time bidding with a focus on the tools and technologies comprising the ad tech market. We identified Google as the leading, and most likely dominant player across the ad tech value chain and expressed the concern that it engages in prima facie anticompetitive conduct, in that it uses its leading ad server to favor its ad intermediation business and exclude competitors. We also explained how the lack of competition across the ad tech value chain enables Google to exploit advertisers and publishers by charging hidden fees for ad intermediation on top of its disclosed commissions. In March 2019, Google announced that it would switch to a first-price unified auction by the end of 2019, arguing that its move would help create a fair and transparent market for everyone. Meanwhile, online advertising has attracted significant regulatory interest in the EU, the USA and Australia. In this new paper, we analyse whether Google’s switch to a unified auction has addressed the concerns we expressed in our first paper. We conclude that this is not the case. Google’s latest switch does nothing to increase auction transparency. Worse, it seems to strengthen Google’s ability to extract hidden margins from its customers, while undermining the competitive pressure exercised by header bidding. If substantiated, Google’s conduct could result in significant welfare losses for publishers, advertisers and ultimately consumers. We conclude our paper by suggesting various structural and behavioral remedies that could help restore competition in ad tech should our concerns be substantiated.

“Trust me, I’m fair”: analysing Google’s latest practices in ad tech from the perspective of EU competition law

ABSTRACTIn a first paper published in February 2019, we explained the mechanics of online display advertising and real-time bidding and noted that the system is extremely opaque. We identified Google as the leading, and most likely dominant player across the ad tech value chain and expressed the concern that it engages in prima facie anticompetitive conduct, in that it uses its leading ad server to favour its ad intermediation business. We also explained how lack of competition across the ad tech chain enables Google to exploit advertisers and publishers by charging hidden fees for ad intermediation on top of its disclosed commission. In March 2019 Google announced that it would be switching to a first-price unified auction by the end of 2019, arguing that its move will help create a fair and transparent market for everyone. Meanwhile, online advertising has attracted significant regulatory interest in the EU, the USA and Australia. In this new paper, we analyse whether Google’s switch to a unified auction has addressed the concerns we expressed in our first paper. Unfortunately, we conclude that Google’s latest switch does nothing to increase auction transparency. Worse, it seems to strengthen Google’s ability to extract hidden margins from its customers, while undermining the competitive pressure exercised by header bidding. If substantiated, Google’s conduct could result in significant welfare losses for publishers, advertisers and ultimately consumers. We conclude our paper by suggesting various structural and behavioural remedies that could help restore competition in ad tech should our concerns be substantiated.

Bid-Aware Active Learning in Real-Time Bidding for Display Advertising

In Real-time Bidding (RTB) based display advertising, demand side platforms (DSPs) estimate the click-through rate (CTR) of each advertisement impression, and then decide whether and how much to bid based on the information of the user and the advertiser. Typically, when a new campaign is launched, the CTR estimation module of the DSP needs to collect data to train an accurate estimator. The advertiser is charged for each ad impression in display advertising, therefore there is some cost for obtaining each training instance. Thus one crucial task is to actively train an accurate CTR estimator within the constraint of the budget. Traditional active learning algorithms fail to deal with such scenario because (i) acquiring training instances is implemented via performing real-time bidding for the corresponding auctions; (ii) RTB requires the bidding agent to make real-time decisions for sequentially coming bid requests; (iii) cost for each ad impression will be unveiled only after giving the bid price and winning the auction; (iv) training data gathered in post-bid stage has a strong bias towards the won impressions. In this paper, we propose a Bid-aware Active Real-time Bidding (BARB) algorithm to actively choose training instances by setting different bid prices for each ad auction, in order to efficiently train an accurate CTR estimation model within the budget constraint. The empirical study on different campaigns of three real-world datasets with three budget constraints shows the effectiveness of our proposed algorithm.

A Dynamic Bidding Strategy Based on Model-Free Reinforcement Learning in Display Advertising

Real-time bidding (RTB) is one of the most striking advances in online advertising, where the websites can sell each ad impression through a public auction, and the advertisers can participate in bidding the impression based on its estimated value. In RTB, the bidding strategy is an essential component for advertisers to maximize their revenues (e.g., clicks and conversions). However, most existing bidding strategies may not work well when the RTB environment changes dramatically between the historical and the new ad delivery periods since they regard the bidding decision as $\boldsymbol {a}$ static optimization problem and derive the bidding function only based on historical data. Thus, the latest research suggests using the reinforcement learning (RL) framework to learn the optimal bidding strategy suitable for the highly dynamic RTB environment. In this paper, we focus on using model-free reinforcement learning to optimize the bidding strategy. Specifically, we divide an ad delivery period into several time slots. The bidding agent decides each impression's bidding price depending on its estimated value and the bidding factor of its arriving time slot. Therefore, the bidding strategy is simplified to solve each time slot's optimal bidding factor, which can adapt dynamically to the RTB environment. We exploit the Twin Delayed Deep Deterministic policy gradient (TD3) algorithm to learn each time slot's optimal bidding factor. Finally, the empirical study on a public dataset demonstrates the superior performance and high efficiency of the proposed bidding strategy compared with other state-of-the-art baselines.

An Effective Budget Management Framework for Real-Time Bidding in Online Advertising

Real-time bidding (RTB) has achieved great success and significantly improved the efficiency and transparency of online advertising. It allows advertisers to purchase ad impressions via auctions. Advertisers who adopt RTB always seek an optimal strategy of budget spending to reach as a wider range of audiences with a more sustainable impact as possible. Traditional bidding strategies, such as fixed bid and performance-based bid, easily lead to being either too aggressive (budgets wiped out too fast) or conservative (budget surplus exists at the end with low clicks), due to lack of optimal budget management. In this paper, we study the optimization of budget efficiency under the smooth delivery constraint for display advertising. We model the problem as a multi-constrained budget allocation optimization and use a heuristic algorithm to solve an approximate optimal budget allocation. The key to the solution is to determine the bidding function for each time slot. Here, we propose a piecewise bidding strategy to filter out the low-quality impressions, where each time slot has its own predicted click-through rate (pCTR) threshold - only when the pCTR of an impression is not lower than the threshold, will the campaign participate in the bidding. However, determining the pCTR threshold is challenging due to market uncertainty, we tackle this problem by modeling the distributions of pCTRs and market prices in each time slot. On this basis, we derive an optimal bidding function for each time slot to make the bid price more adaptable to its available budget. We conduct our experiments on a public real-world dataset and the results show that our proposed method performs the best in terms of various standard metrics (e.g., the number of clicks, cost-per-click) under a given budget comparing to the state-of-the-art baselines.

An Expected Win Rate-Based Real Time Bidding Strategy for Branding Campaign by the Model-Free Reinforcement Learning Model

An Expected Win Rate-Based Real Time Bidding Strategy for Branding Campaign by the Model-Free Reinforcement Learning Model

The Reserve Price Optimization for Publishers on Real-Time Bidding on-Line Marketplaces with Time-Series Forecasting

Abstract Today's Internet marketing ecosystems are very complex, with many competing players, transactions concluded within milliseconds, and hundreds of different parameters to be analyzed in the decision-making process. In addition, both sellers and buyers operate under uncertainty, without full information about auction results, purchasing preferences, and strategies of their competitors or suppliers. As a result, most market participants strive to optimize their trading strategies using advanced machine learning algorithms. In this publication, we propose a new approach to determining reserve-price strategies for publishers, focusing not only on the profits from individual ad impressions, but also on maximum coverage of advertising space. This strategy combines the heuristics developed by experienced RTB consultants with machine learning forecasting algorithms like ARIMA, SARIMA, Exponential Smoothing, and Facebook Prophet. The paper analyses the effectiveness of these algorithms, recommends the best one, and presents its implementation in real environment. As such, its results may form a basis for a competitive advantage for publishers on very demanding online advertising markets.

Website Fingerprinting with Website Oracles

Abstract Website Fingerprinting (WF) attacks are a subset of traffic analysis attacks where a local passive attacker attempts to infer which websites a target victim is visiting over an encrypted tunnel, such as the anonymity network Tor. We introduce the security notion of a Website Oracle (WO) that gives a WF attacker the capability to determine whether a particular monitored website was among the websites visited by Tor clients at the time of a victim’s trace. Our simulations show that combining a WO with a WF attack—which we refer to as a WF+WO attack—significantly reduces false positives for about half of all website visits and for the vast majority of websites visited over Tor. The measured false positive rate is on the order one false positive per million classified website trace for websites around Alexa rank 10,000. Less popular monitored websites show orders of magnitude lower false positive rates. We argue that WOs are inherent to the setting of anonymity networks and should be an assumed capability of attackers when assessing WF attacks and defenses. Sources of WOs are abundant and available to a wide range of realistic attackers, e.g., due to the use of DNS, OCSP, and real-time bidding for online advertisement on the Internet, as well as the abundance of middleboxes and access logs. Access to a WO indicates that the evaluation of WF defenses in the open world should focus on the highest possible recall an attacker can achieve. Our simulations show that augmenting the Deep Fingerprinting WF attack by Sirinam et al. [60] with access to a WO significantly improves the attack against five state-of-the-art WF defenses, rendering some of them largely ineffective in this new WF+WO setting.

A Novel Zero-Sum Polymatrix Game Theory Bidding Strategy for Power Supply Market

The competitive power system market involves very high financial risk due to the essential requirements of real-time bidding decision making. Decisions once taken cannot be altered easily because multiple generators participate in bidding process while simultaneously dispatching to meet the load demand most economically. In order to avoid such risks it becomes pertinent to re-structure the bidding strategies from time to time to meet upcoming techno-economical challenges. In this paper, three generating units are studied using Matrix Laboratory software with a novel approach for deciding the best strategy from the most economical strategy viewpoint. A scenario of different formulations is created for muti-player game, which then is solved with the help of zero-sum polymatrix game theory. A systematic tabular layout of revenues pertaining to each formulation in terms of mixed strategies is developed. The minimax and maximin revenues, identified using Game theoretic approach, gave the most economical strategy. Thus exact and self-enforcing generalized method for best bidding strategies of all three generators are logically derived for the most optimal solution.

Pri-RTB: Privacy-preserving real-time bidding for securing mobile advertisement in ubiquitous computing

Real-time bidding (RTB), one of the major trading mechanisms used for online advertising, allows the advertiser to make an impression-level bid decision. However, the security and privacy concerns of RTB are gaining increasing attention with the recent enforcement of the European Union General Data Protection Regulation (GDPR). In this study, we present a novel privacy-preserving RTB (Pri-RTB) protocol, which aims at preserving both data privacy and utility by performing additively homomorphic encryption on the user profile. To our knowledge, this is the first work to address the privacy issue of the RTB paradigm. We present a formal proof for the security of Pri-RTB under the assumption of the decisional Diffie–Hellman (DDH) problem. We analyze and elaborate the superiority of Pri-RTB over other related works. We also developed a prototype for Pri-RTB and conducted several experiments to evaluate its feasibility and efficiency under different parameters. Our experiments demonstrate that Pri-RTB can work highly efficiently in a practical setting (for example, a scenario with approximately 200 advertisers per auction).

An extensible approach for real-time bidding with model-free reinforcement learning

In this paper, we propose an extensible framework for model-free reinforcement learning (RL) for real-time bidding (RTB) in display advertising. This framework can be applied into both simple environments and extend to the comprehensive environment that the DSP bids for multiple advertisers at the same time. To process new information that is collected via real-time interaction with the environment, an extensible model is first introduced, which is based on the distribution of the recharging probability. Substantial effort is expended to alleviate the problem of the sparsity of the click signal with the reward function. The proposed scheme has high feasibility and can address dynamic environments in contrast to prior works, which assumed that the distribution of the feature vectors and the dealing price were already known. Furthermore, a fund-recharging mechanism is introduced for transforming the RTB model into an endless task, which allows the policy to be optimized in a farsighted rather than a myopic manner. Illustrative experiments on both the small- and large-scale real datasets demonstrate the state-of-the-art performance of the proposed framework for the issue of interest.

Automating the audience commodity: The unacknowledged ancestry of programmatic advertising

Programmatic advertising describes techniques for automating and optimizing transactions in the audience marketplace. Facilitating real-time bidding for audience impressions and personalized targeting, programmatic technologies are at the leading edge of digital, data-driven advertising. But almost no research considers programmatic advertising within a general history of information technology in commercial media industries. The computerization of advertising and media buying remains curiously unexamined. Using archival sources, this study situates programmatic advertising within a longer trajectory, focusing on the incorporation of electronic data processing into the spot television business, starting in the 1950s. The article makes three contributions: it illustrates that (1) demands for information, data processing, and rapid communications have long been central to advertising and media buying; (2) automation “ad tech” developed gradually through efforts to coordinate and accelerate transactions; and (3) the use of computers to increase efficiency and approach mathematical optimization reformatted calculative resources for media and marketing decisions.

On the regulation of personal data distribution in online advertising platforms

Online tracking is the key enabling technology of modern online advertising. In the recently established model of real-time bidding (RTB), the web pages tracked by ad platforms are shared with advertising agencies (also called DSPs), which, in an auction-based system, may bid for user ad impressions. Since tracking data are no longer confined to ad platforms, RTB poses serious risks to privacy, especially with regard to user profiling, a practice that can be conducted at a very low cost by any DSP or related agency, as we reveal here. In this work, we illustrate these privacy risks by examining a data set with the real ad-auctions of a DSP, and show that for at least 55% of the users tracked by this agency, it paid nothing for their browsing data. To mitigate this abuse, we propose a system that regulates the distribution of bid requests (containing user tracking data) to potentially interested bidders, depending on their previous behavior. In our approach, an ad platform restricts the sharing of tracking data by limiting the number of DSPs participating in each auction, thereby leaving unchanged the current RTB architecture and protocols. However, doing so may have an evident impact on the ad platform’s revenue. The proposed system is designed accordingly, to ensure the revenue is maximized while the abuse by DSPs is prevented to a large degree. Experimental results seem to suggest that our system is able to correct misbehaving DSPs, and consequently enhance user privacy.

A New Approach to Real-Time Bidding in Online Advertisements: Auto Pricing Strategy

Real-time bidding (RTB) for digital advertising is becoming the norm for improving advertisers’ campaigns. Unlike traditional advertising practices, in the process of RTB, the advertisement slots of a mobile application or a website are mapped to a particular advertiser through a real-time auction. The auction is triggered and is held for a few milliseconds after an application is launched. As one of the key components of the RTB ecosystem, the demand-side platform gives the advertisers a full pledge window to bid for available impressions. Because of the fast-growing market of mobile applications and websites, the selection of the most pertinent target audience for a particular advertiser is not a simple human-mediated process. The real-time programmatic approach has become popular instead. To address the complexity and dynamic nature of the RTB process, we propose an auto pricing strategy (APS) approach to determine the applications to bid for and their respective bid prices from the advertising agencies’ perspective. We apply the APS to actual RTB data and demonstrate how it outperforms the existing RTB approaches with a higher conversion rate for a lower target spend. A video abstract is available at https://doi.org/10.1287/ijoc.2018.0812 .

An expected win rate-based real-time bidding strategy for branding campaigns on display advertising

For branding campaigns, the demand-side platforms (DSPs) in real-time bidding (RTB) usually need to win as many impressions as possible to inform most audiences about the product messages under constraints on budgets, campaign lifetimes and budget spending plans. In this paper, we propose a novel bidding strategy by introducing the concept of expected win rate. With the proposed expected win rate-based bidding strategy, the DSP can dynamically adjust the expected win rate for each incoming bid request based on factors such as the predicted number of bid requests in the near future, the remaining budget and the remaining lifetime of the campaign. The experimental results show that the proposed bidding strategy has a lower cost per thousand impressions and cost per clicks than existing pacing model-based bidding strategies for branding campaigns with the same budgets and budget spending plans.

Reinforcement Learning Under Drift

Motivated by applications in inventory control and real-time bidding, we consider un-discounted reinforcement learning (RL) in Markov decision processes (MDPs) under temporal drifts. In this setting, both the reward and state transition distributions are allowed to evolve over time, as long as their respective total variations, quantified by suitable metrics, do not exceed certain \emph{variation budgets}. We first develop the Sliding Window Upper-Confidence bound for Reinforcement Learning with Confidence Widening (\texttt{SWUCRL2-CW}) algorithm, and establish its \emph{dynamic regret} bound when the variation budgets are known. In addition, we propose the Bandit-over-Reinforcement Learning (\texttt{BORL}) algorithm to adaptively tune the \sw~to achieve the same dynamic regret bound, but in a \emph{parameter-free} manner, \ie, without knowing the variation budgets. Finally, we conduct numerical experiments to show that our proposed algorithms achieve superior empirical performance compared with existing algorithms. Notably, under non-stationarity, historical data samples may falsely indicate that state transition rarely happens. This thus presents a significant challenge when one tries to apply the conventional Optimism in the Face of Uncertainty (OFU) principle to achieve low dynamic regret bound our problem. We overcome this challenge by proposing a novel confidence widening technique that incorporates additional optimism into our learning algorithms. To extend our theoretical findings, we demonstrate, in the context of single item inventory control with lost-sales, fixed cost, and zero-lead time, how one can leverage special structures on the state transition distributions to bypass the difficulty of exploring time-varying demand environments.

A Revenue-Maximizing Bidding Strategy for Demand-Side Platforms

In real-time bidding (RTB) systems for display advertising, a demand-side platform (DSP) serves as an agent for advertisers and plays an important role in competing for online advertising spaces by placing proper bidding prices. A critical function of the DSP is formulating proper bidding strategies to maximize key performance indicators, such as the number of clicks and conversions. However, many small and medium-sized advertisers' main goal is to maximize revenue with an acceptable return on investment (ROI), rather than simply increase clicks or conversions. Most existing approaches are inapplicable of satisfying the revenue-maximizing goals directly. To solve this problem, we first theoretically analyze the relationships among the conversion rate, ROI, and ad cost, and how they affect revenue. By doing so, we reveal that it is a challenge to increase revenue by relying solely on improving ROI without considering the impact of the ad cost. Based on this insight, the maximal revenue (MR) bidding strategy is proposed to maximize revenue by maximizing the ad cost with a desirable ROI constraint. Unlike previous studies, the proposed MR first distinguishes bid prices from ad costs explicitly, which makes it more applicable to the real second-price auction (GSP) auction mechanism in RTB systems. Then, the winning function is empirically defined in the form of tanh that provides a promising solution for estimating ad costs by jointly considering ad costs with the winning function. The experimental results based on two real-world public datasets demonstrate that the MR significantly outperforms five state-of-the-art models in terms of both revenue and ROI.

Response prediction via integration of heterogeneous information

In recent years, the tensor factorization model has been used to model complicated feature interactions involving multiple aspects, such as the user, publisher, and advertiser, for response prediction in real-time bidding. Among numerous challenges, data sparsity and cold start problems always bother researchers, particularly for ad conversion rate prediction. Such problems in prediction become difficult if only one or several types of information are considered. All types of heterogeneous information must be simultaneously integrated to address these problems. This paper proposes an availability solution for integrating heterogeneous information in the tensor factorization model to efficiently alleviate data sparsity and cold start problems. It proposes different integration strategies and implementation methods for various types of information depending on their property, category, structure, form, and function. This solution efficiently alleviates data sparsity and cold start problems, and enhances the prediction reliability and precision for the tensor factorization model in real-time bidding systems. Finally, this solution achieves a significant improvement in response prediction compared to baselines methods on the selection datasets.

Measuring Display Advertising Response Using Observational Data: The Impact of Selection Biases

This paper studies the potential to better understand display advertising response using observational data from real-time bidding (RTB) exchanges. While non-random exposure to display ads creates endogeneity concerns, unique aspects of the RTB ecosystem allow for common sources of selection, targeting and browsing intensity, to be directly observed and measured. Including these measures in a standard advertising response model mitigates the associated biases. In a two-month panel detailing interactions between 29 campaigns and nearly one million users, we estimate that display advertising increased conversion probability by 127% and was positive and significant in 90% of campaigns. When selection controls are omitted, advertising response was overstated by more than five-fold. An advantage of the proposed approach is the ability to readily explore factors moderating response, and the paper concludes by examining two such features: conversion event and device type. By recovering accurate response estimates from available data, the approach allows advertisers to continuously monitor and improve display advertising effectiveness. Further, it provides an incentive aligning structure to mitigate pervasive agency problems between advertisers and ad-serving platforms.