Large Content Providers Research Articles

Game-theoretic analysis of Net Neutrality effects

Net Neutrality imposes restrictions on the work of Internet Service Providers (ISPs), which can significantly affect social welfare. This article analyzes the following rules established by the Net Neutrality: zero price rule and the prohibition of exclusive deals between ISPs and Content Providers (CPs). The aim of the study is to determine the impact of Net Neutrality on the welfare of various groups of economic agents. To study these implications, the authors create a game-theoretic model of the ISPs market, the unique feature of which is the following heterogeneity of CPs: one large CP creates a large cross-side network effect for consumers and is able to strike exclusive deals with ISPs without Net Neutrality; many small CPs create a small network effect and are unable to influence prices. The following conclusions were obtained: there are ranges of model parameters in which Net Neutrality increases profits of ISPs and reduces profits of a large CP; increases total social welfare if a large CP joins both ISPs without Net Neutrality. The impact of Net Neutrality on consumer surplus and profits of small CPs at these parameter ranges depends on the exclusivity of a large CP in absence of Net Neutrality. Consequently, certain effects of Net Neutrality may differ dramatically depending on how widespread the exclusivity of CPs will be in its absence. These findings may help regulators better assess the implications of Net Neutrality and make a justified decision on its implementation.

BGP-Multipath Routing in the Internet

BGP-Multipath (BGP-M) is a multipath routing technique for load balancing. Distinct from other techniques deployed at a router inside an Autonomous System (AS), BGP-M is deployed at a border router that has installed multiple inter-domain border links to a neighbor AS. It uses the equal-cost multi-path (ECMP) function of a border router to share traffic to a destination prefix on different border links. Despite recent research interests in multipath routing, there is little study on BGP-M. Here we provide the first measurement and a comprehensive analysis of BGP-M routing in the Internet. We extracted information on BGP-M from query data collected from Looking Glass (LG) servers. We revealed that BGP-M has already been extensively deployed and used in the Internet. A particular example is Hurricane Electric (AS6939), a Tier-1 network operator, which has implemented >1,000 cases of BGP-M at 69 of its border routers to prefixes in 611 of its neighbor ASes, including many hyper-giant ASes and large content providers, on both IPv4 and IPv6 Internet. We examined the distribution and operation of BGP-M. We also ran traceroute using RIPE Atlas to infer the routing paths, the schemes of traffic allocation, and the delay on border links. This study provided the state-of-the-art knowledge on BGP-M with novel insights into the unique features and the distinct advantages of BGP-M as an effective and readily available technique for load balancing.

A two-sided model of paid peering

Internet users have suffered collateral damage in tussles over paid peering between large ISPs and large content providers. Paid peering is a relationship where two networks exchange traffic with payment, which provides direct access to each other’s customers without having to pay a third party to carry that traffic for them. The issue will arise again when the United States Federal Communications Commission (FCC) considers a new net neutrality order.We first consider the effect of paid peering on broadband prices. We adopt a two-sided market model in which an ISP maximizes profit by setting broadband prices and a paid peering price. We analytically derive the profit-maximizing prices, and show that they satisfy a generalization of the well-known Lerner rule. Our result shows that paid peering fees reduce the premium plan price, increase the video streaming price and the total price for premium tier customers who subscribe to video streaming services; however, the ISP passes on to its customers only a portion of the revenue from paid peering. ISP profit increases but video streaming profit decreases as an ISP moves from settlement-free peering to paid peering price.We next consider the effect of paid peering on consumer surplus. We find that consumer surplus is a uni-modal function of the paid peering fee. The paid peering fee that maximizes consumer surplus depends on elasticities of demand for broadband and for video streaming. However, consumer surplus is maximized when paid peering fees are significantly lower than those that maximize ISP profit. However, it does not follow that settlement-free peering is always the policy that maximizes consumer surplus. The peering price depends critically on the incremental ISP cost per video streaming subscriber; at different costs, it can be negative, zero, or positive.

The Importance of National Data Infrastructure for Low and Middle-Income Countries

Businesses and individuals are consuming and producing more content over data-rich applications. At the core of this technological shift is the capability to deliver and control the distribution of increasing amounts of data. Countries’ digital ecosystems can be enhanced by data infrastructure building blocks: optimized internet traffic routes through the deployment of effective Internet Exchange Points (IXPs), proximity to the end user through content delivery networks (CDNs), and vast computing capabilities made available by data centers and cloud services. However, research on the distribution and effects of data infrastructure in Low- and Middle-income countries (LMICs) is sparse. This paper comparatively studies the effects of national data infrastructure – Internet Exchange Points and their diversity in terms of including peers and content providers, and Colocation Data Centers – in low- and middle-income countries. By curating a first-of-its-kind dataset, the paper proposes a conceptual framework that posits the different building blocks of a country’s data infrastructure as stages of a ladder, focusing on the openness of data infrastructure and not just its mere availability within a country. Our work looks at the entirety of the data infrastructure value chain for transit and storage, developing a framework that looks at IXPs’ presence as well as participant diversity (not just number but types of networks), and the presence of large content provider peers, co-location data centers, and cloud providers in a country. Relevant descriptive statistics are provided on the availability and distribution of data infrastructure globally by analyzing data collected from several secondary sources and validated with individual country counterparts. Second, low- and middle-income countries are clustered based on characteristics that affect a country’s data ecosystem, namely their income level, mobile broadband penetration, access to submarine cables and mobile market concentration. For a subsample of countries, public measurements taken by RIPE ATLAS probes are used to analyze the percentage of links that are transmitted outside the country and stay local, as well as the distribution of round-trip times for countries over time. Where data availability of probes and subsequent measurements are not available, the gap is filled through interviews with stakeholders and relying where possible on secondary sources. The paper empirically shows how countries on higher steps of the data infrastructure ladder see improvements in latency and reduction in prices. The case studies also highlight the importance of governance and policy to a well-functioning data infrastructure, with the research specifically focused on a diversity of countries across regions and levels of data infrastructure development.

On Migratory Behavior in Video Consumption

Today’s video streaming market is crowded with various content providers (CPs). For individual CPs, understanding user behavior, in particular how users migrate among different CPs, is crucial for improving users’ on-site experience and the CP’s chance of success. In this article, we take a data-driven approach to analyze and model user migration behavior in video streaming, i.e., users switching content provider during active sessions. Based on a large ISP dataset over two months (6 major content providers, 3.8 million users, and 315 million video requests), we study common migration patterns and reasons of migration. We find that migratory behavior is prevalent: 66% of users switch CPs with an average switching frequency of 13%. In addition, migration behaviors are highly diverse: regardless large or small CPs, they all have dedicated groups of users who like to switch to them for certain types of videos. Regarding reasons of migration, we find CP service quality rarely causes migration, while a few popular videos play a bigger role. Nearly 60% of cross-site migrations are landed to 0.14% top videos. Finally, we validate our findings by building an accurate regression model to predict user migration frequency as well as user survey, and discuss the implications of our results to CPs.

PrefCache: Edge Cache Admission With User Preference Learning for Video Content Distribution

With the deployment of video streaming in 4G/5G mobile network, Content Delivery Networks (CDN) are extending to the network edge to provide end-users better Quality of Experience (QoE). However, small cache size and irregular request patterns make it a great challenge for edge caching in video content distribution. Most of the existing cache policies are item-wise, they admit each video object separately, which performs poorly on the network edge due to irregular request patterns. We observe that compared with single video objects, users’ preferences for video topics are much more constant, thus are easier to be predicted. So we propose PrefCache, a novel cache admission policy based on preference learning, for video content edge caching. PrefCache enables an edge cache to learn users’ preferences for videos in real-time. Once receiving a video object, PrefCache decides whether to admit it to the cache by whether it is under users’ preference. We make three contributions in this work. (1) First, we design an information collector, which can proactively collect the preference-related information without any modification of clients and video providers. (2) Second, we propose a tree-structure model to learn and compress users’ preferences. (3) Third, to decide which videos should be admitted to the cache in real-time, an explore-and-exploit method is applied. We carried out extensive experiments with 24 hours of trace data from a large commercial video content provider. The experimental results demonstrate that PrefCache can improve hit ratio up to 12%, and save 92% memory / 98% CPU overhead, compared to the state-of-the-art cache policies.

RL-OPRA: Reinforcement Learning for Online and Proactive Resource Allocation of crowdsourced live videos

With the advancement of rich media generating devices, the proliferation of live Content Providers (CP), and the availability of convenient internet access, crowdsourced live streaming services have witnessed unexpected growth. To ensure a better Quality of Experience (QoE), higher availability, and lower costs, large live streaming CPs are migrating their services to geo-distributed cloud infrastructure. However, because of the dynamics of live broadcasting and the wide geo-distribution of viewers and broadcasters, it is still challenging to satisfy all requests with reasonable resources. To overcome this challenge, we introduce in this paper a prediction driven approach that estimates the potential number of viewers near different cloud sites at the instant of broadcasting. This online and instant prediction of distributed popularity distinguishes our work from previous efforts that provision constant resources or alter their allocation as the popularity of the content changes. Based on the derived predictions, we formulate an Integer-Linear Program (ILP) to proactively and dynamically choose the right data center to allocate exact resources and serve potential viewers, while minimizing the perceived delays. As the optimization is not adequate for online serving, we propose a real-time approach based on Reinforcement Learning (RL), namely RL-OPRA, which adaptively learns to optimize the allocation and serving decisions by interacting with the network environment. Extensive simulation and comparison with the ILP have shown that our RL-based approach is able to present optimal results compared to heuristic-based approaches.

Studying the evolution of content providers in IPv4 and IPv6 internet cores

There is recent evidence that the core of the Internet, which was formerly dominated by large transit providers, has been reshaped after the transition to a multimedia-oriented network, first by general-purpose CDNs and now by private CDNs. In this work we use k-cores, an element of graph theory, to define which ASes compose the core of the Internet and to track the evolution of the core since 1999. Specifically, we investigate whether large players in the Internet content and CDN ecosystem belong to the core and, if so, since when. In addition, we examine differences between the IPv4 and IPv6 cores. We further investigate regional differences in the evolution of large content providers. Finally, we show that the core of the Internet has incorporated an increasing number of content ASes in recent years. To enable reproducibility of this work, we provide a website to allow interactive analysis of our datasets to detect, for example, “up and coming” ASes using customized queries.

Paul Baran, Network Theory, and the Past, Present, and Future of Internet

Paul Baran’s seminal 1964 article “On Distributed Communications Networks” that first proposed packet switching also advanced an underappreciated vision of network architecture: a lattice-like, distributed network, in which each node of the Internet would be homogeneous and equal in status to all other nodes. Scholars who have subsequently embraced the concept of a lattice-like network approach have largely overlooked the extent to which it is both inconsistent with network theory (associated with the work of Duncan Watts and Albert-Laszlo Barabasi), which emphasizes the importance of short cuts and hubs in enabling networks to scale, and the actual way, the Internet initially deployed, which relied on a three-tiered, hierarchical architecture that was actually what Baran called a decentralized network. However, empirical studies reveal that the Internet’s architecture is changing: it is in the process of becoming flatter and less hierarchical, as large content providers build extensive wide area networks and undersea cables to connect directly to last-mile networks. This change is making the network more centralized rather than becoming more distributed. As a result, this article suggests that the standard reference model that places backbones at the center of the architecture should be replaced with a radically different vision: a stack of centralized star networks, each centered on one of the leading content providers.

Deliberate, Principled, Self-Interested Law Breaking: The Ethics of Digital ‘Piracy’

Is digital piracy—understood as illegally accessing or using copyrighted works, such as through a file-sharing platform—morally wrong? Such piracy typically falls into the intriguing category of self-interested law breaking, performed deliberately and in the context of a principled disagreement with the law. Existing treatments of the ethics of piracy fail to consider the full sweep of moral considerations implicated by such law breaking, collapsing the question into deceptively narrow enquiries. I argue that there are many reasons, some stemming from quite surprising sources, for respecting copyright law, even for those who think the law is unjust, are sceptical of the law’s democratic legitimacy, and are frustrated at the immoral behaviour of large corporate content providers.

Standing on the Shoulders of Web Giants: The Economic Effects of Personal Data Brokers

In several jurisdictions, like the European Union and California, consumers now have a legal right to port the data that a company has collected about them to another service. This has spurred the emergence of personal data brokers (PDBs), which offer consumers financial rewards for access to their personal data collected by large online content and service providers (CSPs). Using a game-theoretic model, we investigate how PDBs affect market outcomes in the data economy and under which conditions users will indeed benefit from them. Across several model variants, we find that in markets with a PDB two main equilibrium outcomes exist. First, a minimum income equilibrium can arise, where the PDB only pays a minimal reward, and the CSP lowers its service quality in the long run, leaving consumers worse off than without a PDB. Second, a positive income equilibrium can arise, where the PDB pays significant rewards to consumers, and the CSP’s quality may be higher or lower depending on the PDB's efficiency on the data market. Whether consumers are better off in this equilibrium depends on the relative strengths of three effects that we characterize formally and denote as competition effect, displacement effect, and appropriation effect. Consumers will only benefit from a PDB if it can sell their data efficiently; but in this case, the CSP will also demand a higher (non-zero) price for using its service. Thus, our results bear important managerial and policy implications for data-driven business models and the regulation of data access and portability.

Modeling Dynamics of Online Video Popularity

Large Internet video delivery systems serve millions of videos to tens of millions of users on daily basis, via Video-on-Demand and live streaming. Video popularity evolves over time. It represents the workload, as welll as business value, of the video to the overall system. The ability to predict video popularity is very helpful for improving service quality and operating efficiency. Previous studies adopted simple models for video popularity, or directly adopted patterns from measurement studies. In this paper, we develop a stochastic fluid model that tries to capture two hidden processes that give rise to different patterns of a given video's popularity evolution: the information spreading process, and the user reaction process. Specifically, these processes model how the video is recommended to the user, the videos inherent attractiveness, and users reaction rate, and yield specific popularity evolution patterns. We then validate our model by matching the predictions of the model with observed patterns from our collaborator, a large content provider in China. This model thus gives us the insight to explain the common and different video popularity evolution patterns and why.

Net neutrality and innovation at the core and at the edge

How would abandoning Internet net neutrality affect content providers that have different sizes? We model an Internet broadband provider that can offer a different quality of service (priority) to heterogeneous content providers. Internet users can potentially access all content, although they browse and click ads with different probabilities. Net neutrality regulation effectively protects innovation done at the edge by small content providers. Prioritization, instead, increases both infrastructure core investment and welfare only if it sufficiently stimulates innovation from the large content provider.

Routing-policy aware peering for large content providers

Large content providers, such as Google, Yahoo, and Microsoft, aim to directly connect with consumer networks and place the content closer to end users. Exchanging traffic directly between end users and content providers can reduce the cost of transit services. However, direct connection to all end users is simply not feasible. Content providers by-and-large still rely on transit services to reach the majority of end users. We argue that routing policies are an important factor in considering the selection of ISPs for content providers. Therefore, determining which ISP to peer or use as a transit becomes a key question for content providers. In this paper, we formulate the policy-aware peering problem, in which we determine not only which ISP to connect with, but also the kind of peering agreement to establish. We prove that such a policy-aware peering problem is NP-complete, and propose a heuristic algorithm to solve the problem. Further, we perform a large-scale measurement study of the peering characteristics of five large content providers, and evaluate the existing peering connections deployed by the content providers. Our measurement results show that changing the existing peering agreements or adding as little as 3–5 new peering connections can enhance the connection between content providers and end users significantly.

Adding Enhanced Services to the Internet: Lessons from History

We revisit the last 35 years of history related to the design and specification of Quality of Service (QoS) on the Internet, in hopes of offering some clarity to the current debates around service differentiation. We describe the continual failure to get QoS capabilities deployed on the public Internet, including the technical challenges of the 1980s and 1990s, the market-oriented (business) challenges of the 1990s and 2000s, and recent regulatory challenges. In short, the engineering community successfully developed protocols and mechanisms to implement enhanced services (QoS), and a few individual service providers have deployed them internally or in trusted two-party scenarios. The long-standing failure has been to deploy this capability across the public Internet. We reflect on lessons learned from the history of this failure, the resulting tensions and risks, and their implications for the future of Internet infrastructure regulation. First, the continued failure of QoS over the last three decades derives from political and economic (business) obstacles as well as technical obstacles. The competitive nature of the industry, and a long history of anti-trust regulation (at least in the U.S.) conflicts with the need for competing providers to agree on protocols that require sharing operational data with each other to parameterize and verify committed service qualities. Second, QoS technology can yield benefits as well as harms, so policymaking should focus on harms rather than mechanisms. To assure the benefit to consumers, regulators may need to require transparency about the state of congestion and provisioning on networks using such mechanisms. Third, using QoE as the basis for any regulation will require research, tools and capabilities to measure, quantify, and characterize QoE, and developing metrics of service quality that better reflect our understanding of QoS and QoE for a range of applications. Finally, profound shifts in interconnection arrangements suggest a reshaping of the debate over QoS on the public Internet. Some access networks are interconnecting their private IP-based network platforms to support enhanced services, and using this interconnected platform to vertically integrate infrastructure and applications. Access networks are also connecting directly to large content providers to minimize the risk of performance impairments. These changes trigger new regulatory concerns over the fate of the public Internet, including capital investment incentives and gaps across different bodies of law. Barriers to the deployment of scalable interprovider QoS may be unsurmountable, but since any Internet of the future will face them, it is worth developing a systematic understanding to the challenge of enhanced services, and documenting successes and failures over the history of the Internet as carefully as possible.

Quo vadis Open-IX?

The recently launched initiative by the Open-IX Association (OIX) to establish the European-style Internet eXchange Point (IXP)model in the US suggests an intriguing strategy to tackle a problem that some Internet stakeholders in the US consider to be detrimental to their business; i.e., a lack of diversity in available peering opportunities. We examine in this paper the cast of Internet stakeholders that are bound to play a critical role in determining the fate of this Open-IX effort. These include the large content and cloud providers, CDNs, Tier-1 ISPs, the well-established and some of the newer commercial datacenter and colocation companies, and the largest IXPs in Europe. In particular, we comment on these different parties' current attitudes with respect to public and private peering and discuss some of the economic arguments that will ultimately determine whether or not the currently pursued strategy by OIX will succeed in achieving the main OIX-articulated goal -- a more level playing field for private and public peering in the US such that the actual demand and supply for the different peering opportunities will be reflected in the cost structure.

Optimal Management of Digital Content on Tiered Infrastructure Platforms

Media firms are increasingly using tiered infrastructure to cost-effectively manage heterogeneous resources by strategically allocating digital content across multiple tiers to avoid overcapacitating high-performance, expensive infrastructure tiers. Complex migrations in tiered environments are currently possible in a seamless, nondisruptive manner. We model digital content as a network capturing the inter-item impacts and use this network structure to develop optimal migration policies that partition media content into tiers. Addressing the context of large content providers such as video-on-demand providers that employ infrastructure platforms for content storage and delivery, we develop a bilevel programming model to maximize the profits of a price-setting platform and a tiered allocation-setting content provider. We model two fundamental effects with digital content: a revenue effect emanating from the tiered architecture and a traffic generating effect among media objects. Using a detailed longitudinal simulation study, we demonstrate the effectiveness of the proposed provisioning policy and pricing strategy and illustrate the existence and impact of these effects in media markets. Finally, we show that repeated execution of the model can help providers respond effectively to a changing environment and thus better manage the risk from demand fluctuations.

Towards a Developmental Framework for Net Neutrality: The Rise of Sponsored Data Plans in Developing Countries

Academic research has been paying little attention to the net neutrality debate in developing countries, where large content providers such as Facebook, Google and Twitter have long been executing agreements with local mobile ISPs for prioritized or subsidized data delivery for their customers. Thus, the goal of this paper is to assess the potential consequences of these business arrangements for the IT industry in these countries. Our hypothesis is that these agreements may actually lead to unintended effects for the expansion of the mobile sector in developing countries, as they may empower market concentration, restrict local innovation and reduce user choices. In order to overcome these consequences, a net neutrality rule could play an instrumental role in fostering the innovation sector in developing countries, without resorting to protectionist policies.

The Master Switch and the Hyper Giant: Google's Infrastructure and Network Neutrality Strategy in the 2000s

During the first decade of the 2000s, Google Inc., a global information company historically unprecedented in scope and influence, was a strong proponent of the principle of network neutrality, the notion that all Internet traffic should be treated more or less equally. In his 2010 book The Master Switch, Tim Wu argued that network neutrality was critical to Google’s viability as a business; without it, any Internet service provider might act as a “master switch” and limit the search engine’s access to both content and customers. Yet in 2010, the company’s strong support for what it had called the “open Internet” appeared to diminish, first through a series of joint policy statements with telecommunications giant Verizon, and then by a seeming withdrawal from the network neutrality policy debate. During this same period, Google poured enormous resources into the creation of one of the world’s largest technological infrastructures, a global system of over one million servers connected by both the public Internet and Google’s own global network.This working paper examines the extent to which Google’s massive infrastructure has allowed the company to mitigate the danger of an ISP master switch. Using data drawn from documentary evidence and technical analysis of Google’s network topology we present a high-level view of the company’s infrastructure and the affordances it provides. We suggest that Google’s infrastructure drastically reduced the company’s reliance on the public Internet, while allowing Google to build strong interdependencies with Internet service providers. Drawing on Labovitz’s notion of the hyper giant, we argue that Google and other large Internet content and service providers transcended the traditional “content versus carrier” dichotomy that was a key component of the network neutrality debate of the 2000s.

Assessing Access Network Market Power

Interconnection agreements in the telecommunications industry have always been constrained by regulation, but Internet interconnection has not received the same level of scrutiny. Recent debates regarding proposed mergers, network neutrality, Internet peering, and last mile competition have generated much discussion about whether Internet interconnection regulation is warranted.Concerns about network market power are not limited to those network providers at the top of the Internet hierarchy. In recent years, the concerns have actually shifted from the backbone providers at the core of the Internet to the access networks at the edge. Limited access network competition can not only impact consumer pricing or increase the likelihood of harmful discrimination, it can also lead to unfair interconnection practices. Recent interconnection disputes have generated questions about whether the concentration in access networks impacts the ability of other network providers to deliver traffic to those end users. This concern is not restricted to interconnection between access networks and upstream networks. Due to the emergence of “hyper giants” who are responsible for a large percentage of Internet traffic, it extends to the relationships between access networks and pure play CDNs as well as access networks and large content providers. In order to determine whether regulation is necessary, policymakers need appropriate metrics to help gauge a network provider’s market power. The metrics used to assess backbone network market power may not be as relevant for access networks. Ideally, policymakers would have access to Internet interconnection agreements as these reflect a network’s bargaining power. Unfortunately these are not typically publicly available. Policymakers must instead rely on proxy metrics and inferred interconnection relationships. This paper proposes a new metric, access variance, for assessing access network market power. This metric is defined as the diversity of feasible paths into an access network and can provide a useful assessment of access network’s ability to control access to its end users.Content delivery networks (CDNs) may provide some visibility into an access network's access variance. A CDN will typically distribute requests from an access network's end users across multiple deployments depending on factors such as performance and cost. While some of these deployments may be located within the access network, other deployments may be in upstream networks that have interconnection agreements with the access network. This access variance can help ascertain the diversity of access methods other than establishing an interconnection agreement directly with the access network. The paper provides an empirical example analyzing whether a CDN has alternative options for reaching an access network’s end users. The example focuses on France as the French ISPs have publicly stated their intention to limit CDN access if they are not compensated according to their terms. By analyzing the CDN traffic flows in France, we can determine the extent of the French ISPs’ ability to restrict access. If a CDN is either unable to deliver content to an access network’s end users or is only able to deliver content to those users from its deployments within the access network, this may suggest that the access network is able to control access to its subscribers. The analysis involves identifying the DNS nameservers associated with the French ISPs and issuing requests for CDN content from those nameservers. The resulting server IP addresses are recorded and analyzed to map the CDN deployments responding to those requests. In this analysis, the CDN deployments within the access networks served less than 50% of the requests originating from the access networks’ on-network nameservers. This suggests that there are viable alternatives to establishing interconnection agreements directly with the access networks.