Large-scale Video-on-demand Systems Research Articles

User Behavior Analysis and Video Popularity Prediction on a Large-Scale VoD System

Understanding streaming user behavior is crucial to the design of large-scale Video-on-Demand (VoD) systems. In this article, we begin with the measurement of individual viewing behavior from two aspects: the temporal characteristics and user interest. We observe that active users spend more hours on each active day, and their daily request time distribution is more scattered than that of the less active users, while the inter-view time distribution differs negligibly between two groups. The common interest in popular videos and the latest uploaded videos is observed in both groups. We then investigate the predictability of video popularity as a collective user behavior through early views. In the light of the limitations of classical approaches, the Autoregressive-Moving-Average (ARMA) model is employed to forecast the popularity dynamics of individual videos at fine-grained time scales, thus achieving much higher prediction accuracy. When applied to video caching, the ARMA-assisted Least Frequently Used (LFU) algorithm can outperform the Least Recently Used (LRU) by 11--16%, the well-tuned LFU by 6--13%, and the LFU is only 2--4% inferior to the offline LFU in terms of hit ratio.

Migrating big video data to cloud: a peer-assisted approach for VoD

Since user demand for a Video-on-demand (VoD) service varies with time in one-day period, provisioning self-owned servers for the peak load it must sustain afew hours per day leads to bandwidth under-utilization at other times. Content clouds, e.g. Amazon CloudFront and Azure CDN, let VoD providers pay by bytes for bandwidth resources, potentially leading to cost savings even if the unit rate to rent a machine from a cloud provider is higher than the rate to own one. In addition, recent studies have presented fog computing as a new paradigm to extend the cloud-based platform for a cost-effective and highly scalable service. In this paper, based on long-term traces from two large-scale VoD systems and temporal development model of content clouds, we tackle challenges, design and potential benefits in migrating both Clients/Server-based and peer-assisted VoD services into the hybrid cloud and edge peers in fog computing environment. Our measurements show that the popularity of the most popular videos decays so quickly, for example, by 11% after one hour that it poses large challenges on updating videos in the cloud. However, the trace-driven evaluations show that our proposed migration strategies (active, reactive and smart strategies), although simply based on the current information, can make the hybrid cloud-assisted VoD deployment save up to 30% bandwidth expense compared with the Clients/Server mode. Moreover, they can also handle the flash crowd traffic with little cost. Leveraging the edge peers in fog computing, we propose a cloud-friendly peer replication strategy, which further reduces the migration cost by a factor of 4. Our simulation also shows that the cloud price and server bandwidth chosen play the most important roles in saving cost, while the cloud storage size and cloud content update strategy play the key roles in the user experience improvement.

Capacity of P2P On-Demand Streaming With Simple, Robust, and Decentralized Control

The performance of large-scale peer-to-peer (P2P) video-on-demand (VoD) streaming systems can be very challenging to analyze due to sparse connectivity and complex, random dynamics. Specifically, in practical P2P VoD systems, each peer only interacts with a small number of other peers/neighbors. Furthermore, its upload capacity, downloading position, and content availability change dynamically and randomly. In this paper, we rigorously study large-scale P2P VoD systems with sparse connectivity among peers and investigate simple and decentralized P2P control strategies that can provably achieve close-to-optimal streaming capacity. We first focus on a single streaming channel. Using a simple algorithm that assigns each peer a random set of $\Theta (\log N)$ neighbors and allocates upload capacity uniformly, we show that a close-to-optimal streaming rate can be asymptotically achieved for all peers with high probability as the number of peers $N$ increases. Furthermore, the tracker does not need to obtain detailed knowledge of which chunks each peer caches, and hence incurs low overhead. We then study multiple streaming channels where peers watching one channel may help peers in another channel with insufficient upload bandwidth. We propose a simple random cache-placement strategy and show that a close-to-optimal streaming capacity region for all channels can be attained with high probability, again with only $\Theta (\log N)$ per-peer neighbors. These results provide important insights into the dynamics of large-scale P2P VoD systems, which will be useful for guiding the design of improved P2P control protocols.

Video Delivery Performance of a Large-Scale VoD System and the Implications on Content Delivery

Video delivery performance is the main factor that affects Internet video quality. Characterizing the video delivery performance, especially the delivery throughput, can help content providers as well as Internet service providers (ISPs) in system optimization and network planning. Based on a unique dataset consisting of 20 million video download speed measurements , this paper comprehensively studies the video delivery throughput of a large-scale commercial video-on- demand (VoD) system. We observe that user speed exhibits a large variation over time of day as well as across provincial locations. In particular, the worst performance of day is 30% lower than the peak performance . The analysis also reveals that video download speed has a notable impact on Internet video quality, which in turn influences user engagement . The impact, however, becomes limited when the speed increases beyond a certain threshold, which is mostly dependent on the video encoded bitrates. We further examine the interaction between Internet infrastructure and video delivery throughput using the linear regression model and find that crossing the ISP or regional network border yields 15–20% speed loss. Based on these observations , we finally evaluate the potential of edge caching and hybrid CDN-P2P in the improvement of video download performance and video quality.

Implementation and Analysis of a Peer-to-Peer Retransmissions System for Live Video Services

As video-on-demand (VoD) access is progressively becoming consum ers' preferred way to access video content, there are many chal lenges in designing large-scale VoD systems that are capable of high standards in scalability and reliability. Video-streaming reliability has always been synonymous with reduced scalabil ity and higher cost in terms of bandwidth and complexity. This article addresses this perfor mance trade-off. Our system offers VoD services while reconciling this trade-off in scalability and reliability. Our VoD system supports a significantly higher number of VoD sessions than other systems, while still providing an improved reliability through a distributed error-control mechanism.

Video-on-Demand Systems With Cooperative Clients in Multicast Environment

Peer-to-peer (P2P) and multicast approaches are two common transmission strategies to provide scalable and cost efficient solutions for video-on-demand (VoD) services. The former approach requires small server resources and provides a negligible delay to users, but the bandwidth requirement inside the network will be rapidly increased when more customers join the video session. On the other hand, if the system simply uses a multicast scheme to deliver a video, customers will experience a noticeable delay before watching the video whereas the overall bandwidth requirement will not be significantly increased. In this paper, a new transmission policy denoted peer-to-peer batching (PPB) policy is proposed to efficiently deliver video data in a large scale VoD system by exploiting the multicast capability of the network and P2P paradigm. To avoid the disruption of services, the fault tolerance and recovery mechanism is also developed for PPB. In addition, in order to further improve the scalability of the system, a distributed PPB (DPPP) is also proposed. We then develop a mathematical model to evaluate the performance of each of our proposed policies analytically. Based on this model, system designers are able to study various design options as well as perform system dimensioning. Both analytical and simulation results show that the proposed scheme achieves better system performance compared with existing schemes.

Design, performance analysis, and implementation of a super-scalar video-on-demand system

Despite the availability of video-on-demand (VoD) services in a number of cities around the world, large-scale deployment of VoD services in a metropolitan area is still economically impractical. This study presents a novel super-scalar architecture for building very large-scale and efficient VoD systems. The proposed architecture combines the use of static multicast, dynamic multicast, and intelligent client-side caching to vastly reduce server and network resource requirement. Moreover, in sharp contrast to conventional VoD systems where the system cost increases at least linearly with the system scale, the proposed architecture becomes more efficient as the system scales up and can ultimately be scaled up to serve any number of users while still keeping the startup latency short. This paper presents this new architecture, proposes methods to support interactive playback controls without the need for additional server or client resources, and derives an approximate performance model to relate the startup latency with other system parameters. The performance model is validated using simulation and the architecture is evaluated under various system settings. Lastly, a system implementation is presented and benchmarking results obtained to further verify the architecture, the performance model, and the simulation results.

A multi-server video-on-demand system with arbitrary-rate playback support

In this paper, we propose a multi-server design for video-on-demand (VoD) systems that can provide different quality of service (QoS) guarantees by arbitrary-rate video playback. This design chooses a symmetric architecture that consists of a set of identically configured servers and supports arbitrary rates of video playback required by different clients in the following way. The playback requirements requested from all clients are first sent to the master server. If the master server decides to accept a new client, it distributes the task of the new client to all the servers, including itself. Through the pre-processing of the video streams, we can map the data of each frame of every video to a frame map. This frame map indicates whether a frame should be sent to the client at a given frame rate. In our multi-server design for a VoD system, using these frame maps and a suitable transmission schedule, a client can demand to play any video, at any playback rate and at any time. Furthermore, the VoD system provides a high degree of fault tolerance. This design not only can tolerate the failure of one or more servers, but also requires relatively small bandwidth capability or reservation for fault tolerance. In addition, this multi-server design can effectively overcome the hot spot problem of popular videos and keep the load balanced at the same time. Finally, we discuss how to generalize the proposed method for a large-scale VoD system with heterogeneous architecture.

Threshold-Based Dynamic Replication in Large-ScaleVideo-on-Demand Systems

Recent advances in high speed networking technologies and video compression techniques have made Video-on-Demand (VOD) services feasible. A large-scale VOD system imposes a large demand on I/O bandwidth and storage resources, and therefore, parallel disks are typically used for providing VOD service. Although striping of movie data across a large number of disks can balance the utilization among these disks, such a striping technique can exhibit additional complexity, for instance, in data management, such as synchronization among disks during data delivery, as well as in supporting fault tolerant behavior. Therefore, it is more practical to limit the extent of data striping, for example, by arranging the disks in groups (or nodes) and then allowing intra-group (or intra-node) data striping only. With multiple striping groups, however, we may need to assign a movie to multiple nodes so as to satisfy the total demand of requests for that movie. Such an approach gives rise to several design issues, including: (1) what is the right number of copies of each movie we need so as to satisfy the demand and at the same time not waste storage capacity, (2) how to assign these movies to different nodes in the system, and (3) what are efficient approaches to altering the number of copies of each movie (and their placement) when the need for that arises. In this paper, we study an approach to dynamically reconfiguring the VOD system so as to alter the number of copies of each movie maintained on the server as the access demand for these movies fluctuates. We propose various approaches to addressing the above stated issues, which result in a VOD design that is adaptive to the changes in data access patterns. Performance evaluation is carried out to quantify the costs and the performance gains of these techniques.

Performance measurements of a small-scale vod server based on the unix

Recent progress in network technologies has made it possible to integrate digital video and audio into distributed computing environments. Among the various distributed multimedia applications, video-on-demand (VOD) services are the most typical application area. However, interest in large-scale VOD systems is gradually transferring to small and low-cost VOD systems because of the excess investment required in the transmission media. When we make a small-scale VOD system, one of the most important factors is the efficient resource management of the VOD server in order to achieve cost-effectiveness. Thus, multimedia has generated widespread interest in real-time and high-performance I/O support within general-purpose operating systems. This paper presents the performance of a small-scale, UNIX-based VOD server equipped with a real-time scheduler and a continuous media file system (CMFS). The real-time scheduler is based on the theory of non-preemptive real-time systems to test schedulability within polynomial time. To reduce interference of the interrupt processing for the real-time processes, the processes are classified into two groups in order of importance. The CMFS has been designed and implemented to acquire the performance of the raw disk. According to the analysis of the measured data, UNIX can efficiently transfer video data from disk to network and it can also guarantee most deadlines.