Adaptive Bitrate Algorithms Research Articles

Beyond QoE: Diversity Adaptation in Video Streaming at the Edge

Adaptive bitrate (ABR) algorithms are critical techniques for high quality-of-experience (QoE) Internet video delivery. Early ABR algorithms conducting the overall QoE function of fixed parameters are limited by the fact that the QoE of end-users are diverse such that the video bitrate is often chosen in a misleading way. State-of-the-art ABR algorithms like MPC and Pensieve utilize offline modeling techniques and result in performance degradation for online QoE diversity adaptation. To address this issue, we propose Elephanta, an online ABR algorithm for edge users, which incorporates user QoE perception interface and adaptation algorithm with flexible parameters. In order to avoid overhead from updating parameters online, we model video streaming as a renewal system and formulate the specific QoE function into flexible formats by setting constraints on corresponding QoE metrics. To validate parameter settings, we emulate Elephanta under 1500 throughput traces, including FCC broadband, $3G$ HSDPA data set from the Internet, as well as the $4G$ /LTE data set we collect. Evaluation results show that Elephanta achieves QoE improvement of 7% over MPC and 3% over Pensieve under QoE diversity in part because of its superior adaptability to QoE diversity. We implemented Elephanta in dash.js at the client side for subjective experiments. We observed the diverse QoE preferences across users and 19/21 users (strongly) agree that Elephanta is responsive to parameter changes while watching videos.

Quality-Aware Neural Adaptive Video Streaming With Lifelong Imitation Learning

Existing Adaptive Bitrate (ABR) algorithms pick future video chunks' bitrates via fixed rules or offline trained models to ensure good quality of experience (QoE) for Internet video. Nevertheless, data analysis demonstrates that a good ABR algorithm is required to continually and fast update for adapting itself to time-varying network conditions. Therefore, we propose Comyco, a video quality-aware learning-based ABR approach that enormously improves recent schemes by i) picking the chunk with higher perceptual video qualities rather than video bitrates; ii) training the policy via imitating expert trajectories given by the expert strategy; iii) employing the lifelong learning method to continually train the model w.r.t the fresh trace collected by the users. To achieve this, we develop a complete quality-aware lifelong imitation learning-based ABR system, construct quality-based neural network architecture, collect a quality-driven video dataset, and estimate QoE metrics with video quality features. Using trace-driven and real-world experiments, we demonstrate Comyco reaches 1700-fold improvements in the number of samples required and 16-fold speedup in the training time compared with the prior work. Meanwhile, Comyco outperforms existing methods, with the improvements on average QoE of 7.5%-16.79%. Moreover, experimental results on continual training also illustrate that lifelong learning helps Comyco further improve the average QoE of 1.07%-9.81% in comparison to the offline trained model.

Traffic-Aware Rate Adaptation for Improving Time-Varying QoE Factors in Mobile Video Streaming

Mobile video has become one of the most valuable services in next-generation heterogeneous networks, and users’ quality of experience (QoE) is recognized as its important performance metric. In this paper, we propose an adaptive bitrate (ABR) algorithm to achieve improvement of the timevarying QoE determinants during a mobile video playback. Since watching videos will bring mobile data charges to the users, the traffic consumed by video downloads should be customized to a user-specified amount. To this end, we first analyze the real-world traces, and find the key factors to formulate a continuous QoE model. Then, we introduce a traffic-aware rate adaptation strategy (TARA). Given that users are aware of their real-time capabilities when watching a mobile video, TARA enables robust ABR process which can satisfy users’ requirements with traffic constraints. Finally, a centralized reinforcement learning (RL) approach is proposed for the joint optimization of TARA, which aims to maximize the designed QoE metric, and deliver the expected viewing experience to users. Results of experiments driven by both the simulated, and real-world network traces reveal the efficiency of the proposed TARA strategy, and demonstrate its signicant performance improvement as compared to the state-of-the-art ABR algorithms.

Towards Influence of Chunk Size Variation on Video Streaming in Wireless Networks

In recent years, the growth in popularity of mobile video streaming services is unbroken. There are tremendous demands for video streaming over wireless networks. Currently, most video streaming is over HTTP. Up to now, HTTP-based adaptive video streaming is standardized as DASH, where a client-side video player can dynamically pick the bitrate level according to the perceived network conditions. Actually, not only the available bandwidth drastically varies due to wireless network properties, but also the chunk sizes in the same bitrate level significantly fluctuate, which also influences the bitrate adaptation. However, existing bitrate adaptation algorithms mostly focus on available bandwidth but do not involve chunk size variation, leading to performance losses. In this paper, we theoretically analyze the influence of chunk size variation on bitrate adaptation performance in wireless networks. Based on DASH system features, we build a general model describing playback buffer evolution. Applying stochastic theories, we respectively analyze the influence of the chunk size variation on rebuffering probability, average bitrate, and bitrate switching interval. Furthermore, based on theoretical insights, we provide several suggestions for algorithm designing and rate encoding, and also design a simple bitrate adaptation algorithm. Extensive simulations verify our insights, suggestions, and designed algorithm effectiveness.

Adaptive Bitrate Video Streaming in Non-Orthogonal Multiple Access Networks

Facing the ever-increasing demand for high quality video streaming and the dramatic variations of wireless network environment, adaptive bitrate video streaming (ABS) emerges as a prominent video quality adaptation technique in improving users’ quality of experience (QoE). Reinforcement learning (RL)-based ABS and network-assisted ABS can improve users’ QoE through resource utilization efficiency enhancement. However, limited network capacity is still the bottleneck of QoE improvement. Considering the potential capacity gain of non-orthogonal multiple access (NOMA), we study network-assisted ABS in NOMA networks. Based on a synchronous video streaming model and the knowledge of channel state information for next segment, a joint optimization problem of resource allocation and bitrate adaptation is formulated to maximize users’ QoE and minimize energy consumption. The original problem is NP-hard and intractable due to the interference coordination problem introduced by NOMA and the multi-slot sum rate constraint. We decompose the original problem into the subproblem of joint subchannel assignment and power allocation (JSAPA), and the subproblem of bitrate adaptation (BA) with Lagrange relaxation method. A joint resource allocation and bitrate adaptation (JRABA) algorithm is further proposed to solve the JSAPA and BA subproblems iteratively. Specifically, the JSAPA subproblem is solved by an extended Gale-Shapley matching algorithm and differ of convex (DC) programming method iteratively. For the BA subproblem, appropriate video bitrate is explored with subgradient method. Numerical results validate the effectiveness of the proposed algorithm and reveal that network-assisted ABS in NOMA networks benefits users in achieving higher QoE with less energy consumption, especially when radio resources are insufficient or with large segment size.

DQ-DASH

The significant popularity of HTTP adaptive video streaming (HAS), such as Dynamic Adaptive Streaming over HTTP (DASH), over the Internet has led to a stark increase in user expectations in terms of video quality and delivery robustness. This situation creates new challenges for content providers who must satisfy the Quality-of-Experience (QoE) requirements and demands of their customers over a best-effort network infrastructure. Unlike traditional single server DASH, we developed a D istributed Q ueuing theory bitrate adaptation algorithm for DASH (DQ-DASH) that leverages the availability of multiple servers by downloading segments in parallel. DQ-DASH uses a M x /D/1/K queuing theory based bitrate selection in conjunction with the request scheduler to download subsequent segments of the same quality through parallel requests to reduce quality fluctuations. DQ-DASH facilitates the aggregation of bandwidth from different servers and increases fault-tolerance and robustness through path diversity. The resulting resilience prevents clients from suffering QoE degradations when some of the servers become congested. DQ-DASH also helps to fully utilize the aggregate bandwidth from the servers and download the imminently required segment from the server with the highest throughput. We have also analyzed the effect of buffer capacity and segment duration for multi-source video streaming.

Exploiting the layer correlation to improve DASH scheduling with scalable video coding

DASH is a promising video streaming solution that gracefully deals with network throughput fluctuations as well as terminal heterogeneity. In DASH, an MPD file describes the available video qualities and their average bitrates. However, the coarse-grained average bitrate is insufficient for an accurate client-side scheduler. The bitrate overestimate or underestimate of bandwidth requirements for segments to be fetched could result in suboptimal scheduling decisions and possibly drain the playback buffer. In this paper, we first made the observation that with SVC, video segments across different layers exhibit high size correlation. We then showed that enhancement layer segment size can be predicted by the corresponding base layer segment with relatively high accuracy. Based on this observation, we enhanced existing rate-based and buffered-based adaptive bitrate (ABR) algorithms with a size predictor, which is orthogonal to existing adaptation logics. Results show that augmented by the size predictor, the average playback bitrate can be improved by up to 20% and playback stalls can also be significantly reduced. Our results demonstrated that similar to the bandwidth predictor, a size predictor should also be added to SVC-based ABR algorithms to increase the performance gain.

Vabis: Video Adaptation Bitrate System for Time-Critical Live Streaming

With the rise of time-critical and interactive scenarios, ultra-low latency has become the most urgent requirement. Adaptive bitrate (ABR) schemes have been widely used in reducing latency for live streaming services. However, the traditional solutions suffer from a key limitation: they only utilize coarse-grained chunk to solve the I-frame misalignment problem in different bitrate switching process at the cost of increasing latency. As a result, existing schemes are difficult to guarantee the timeliness and granularity of control in essence. In this paper, we use a frame-based approach to solve the I-frame misalignment problem and propose a video adaptation bitrate system (Vabis) in units of the frame for time-critical live streaming to obtain the optimal quality of experience (QoE). On the server-side, a Few-Wait ABR algorithm based on Reinforcement Learning (RL) is designed to adaptively select the bitrate of future frames by state information that can be observed, which can subtly solve the problem of I-frame misalignment. A rule-based ABR algorithm is designed to optimize the Vabis system for the weak network. On the client-side, three delay control mechanisms are designed to achieve frame-based fine-grained control. We construct a trace-driven simulator and the real live platform to evaluate the comprehensive live streaming performance. The results show that Vabis is significantly better than the existing methods with decreases in an average delay of 32%-77% and improvements in average QoE of 28-67%.

Adaptive Chunklets and AQM for Higher-Performance Content Streaming

Commercial streaming services such as Netflix and YouTube use proprietary HTTP-based adaptive streaming (HAS) techniques to deliver content to consumers worldwide. MPEG recently developed Dynamic Adaptive Streaming over HTTP (DASH) as a unifying standard for HAS-based streaming. In DASH systems, streaming clients employ adaptive bitrate (ABR) algorithms to maximise user Quality of Experience (QoE) under variable network conditions. In a typical Internet-enabled home, video streams have to compete with diverse application flows for the last-mile Internet Service Provider (ISP) bottleneck capacity. Under such circumstances, ABR algorithms will only act upon the fraction of the network capacity that is available, leading to possible QoE degradation. We have previously explored chunklets as an approach orthogonal to ABR algorithms, which uses parallel connections for intra-video chunk retrieval. Chunklets effectively make more bandwidth available for ABR algorithms in the presence of cross-traffic, especially in environments where Active Queue Management (AQM) schemes such as Proportional Integral controller Enhanced (PIE) and FlowQueue-Controlled Delay (FQ-CoDel) are deployed. However, chunklets consume valuable server/middlebox resources which typically handle hundreds of thousands of requests/connections per second. In this article, we propose ‘adaptive chunklets’ -- a novel chunklet enhancement that dynamically tunes the number of concurrent connections. We demonstrate that the combination of adaptive chunklets and FQ-CoDel is the most effective strategy. Our experiments show that adaptive chunklets can reduce the number of connections by almost 30% and consume almost 8% less bandwidth than fixed chunklets while providing the same QoE.

NEWCAST: Joint Resource Management and QoE-Driven Optimization for Mobile Video Streaming

Predicting future throughput in mobile networks becomes more and more possible today thanks to the rich contextual information provided by mobile applications and smartphone sensors. It is even likely that such contextual information, which may include traffic, mobility and radio conditions will lead to a novel agile resource management not yet thought of. In this paper, we propose a framework (called NEWCAST) that anticipates the throughput variations to deliver video streaming content. NEWCAST takes advantage of the capacity prediction in order to better distribute the resources allocated by the scheduler among users over the prediction horizon. This has the advantage of leading towards better user engagement for video streaming users without harming other traffics present in the system. We develop an optimization problem that realizes a fundamental trade-off among critical metrics that impact the user’s perceptual quality of experience (QoE) and the cost of system utilization. Both simulated and real-world throughput traces −1.5mm]Per IEEE style, reference citations are not permitted in the abstract. Please cite in text or confirm okay to leave as uncited. were carried out to evaluate the performance of NEWCAST. It is shown from our numerical results that NEWCAST provides the efficiency that the new 5G architectures require in terms of computational complexity and robustness. We also implement a prototype system of NEWCAST and evaluate it in a real environment with a real player to show its efficiency and scalability in a multi-users scenario compared to baseline adaptive bitrate algorithms.

Sensor-Augmented Neural Adaptive Bitrate Video Streaming on UAVs

Recent advances in unmanned aerial vehicle (UAV) technology have revolutionized a broad class of civil and military applications. However, the designs of wireless technologies that enable real-time streaming of high-definition video between UAVs and ground clients present a conundrum. Most existing adaptive bitrate (ABR) algorithms are not optimized for the air-to-ground links, which usually fluctuate dramatically due to the dynamic flight states of the UAV. In this paper, we present SA-ABR, a new sensor-augmented system that generates ABR video streaming algorithms with the assistance of various kinds of inherent sensor data that are used to pilot UAVs. By incorporating the inherent sensor data with network observations, SA-ABR trains a deep reinforcement learning (DRL) model to extract salient features from the flight state information and automatically learn an ABR algorithm to adapt to the varying UAV channel capacity through the training process. SA-ABR does not rely on any assumptions or models about UAV's flight states or the environment, but instead, it makes decisions by exploiting temporal properties of past throughput through the long short-term memory (LSTM) to adapt itself to a wide range of highly dynamic environments. We have implemented SA-ABR in a commercial UAV and evaluated it in the wild. We compare SA-ABR with a variety of existing state-of-the-art ABR algorithms, and the results show that our system outperforms the best known existing ABR algorithm by 21.4% in terms of the average quality of experience (QoE) reward.

Dynamic Adaptive Streaming based on Deep Reinforcement Learning

Dynamic adaptive streaming over HTTP (DASH) is the dominant technology of multimedia delivery over the Internet. In DASH system, adaptive bitrate (ABR) algorithms running on client-side video player are the key to improve user quality of experience (QoE). However, most existing ABR algorithms employ fixed control rules to make bitrate decisions based on throughput, playback buffer size, or a combination of the two. As a result, their performance in the complicated and fluctuant network environment is incompetent. In this paper, we propose QRL, a bitrate adaptation approach based on deep reinforcement learning. QRL uses double Q-Learning, an enhanced Q-Learning method. After training the neural network model, the algorithm can select proper bitrates for future video segments based on all the information collected by client during the video playback process. Simulation results show that QRL achieves better performance than other algorithms.

From Theory to Practice

Modern video streaming uses adaptive bitrate (ABR) algorithms that run inside video players and continually adjust the quality (i.e., bitrate) of the video segments that are downloaded and rendered to the user. To maximize the quality-of-experience (QoE) of the user, ABR algorithms must stream at a high bitrate with low rebuffering and low bitrate oscillations. Further, a good ABR algorithm is responsive to user and network events and can be used in demanding scenarios such as low-latency live streaming. Recent research papers provide an abundance of ABR algorithms but fall short on many of the above real-world requirements. We develop Sabre, an open-source publicly available simulation tool that enables fast and accurate simulation of adaptive streaming environments. We empirically validated Sabre to show that it accurately simulates real-world environments. We used Sabre to design and evaluate BOLA-E and DYNAMIC, two novel ABR algorithms. We also developed a FAST SWITCHING algorithm that can replace segments that have already been downloaded with higher-bitrate (thus, higher-quality) segments. The new algorithms provide higher QoE to the user in terms of higher bitrate, fewer rebuffers, and lesser bitrate oscillations. In addition, these algorithms react faster to user events such as startup and seek, and they respond more quickly to network events such as improvements in throughput. Further, they perform very well for live streams that require low latency, a challenging scenario for ABR algorithms. Overall, our algorithms offer superior video QoE and responsiveness for real-life adaptive video streaming, in comparison to the state-of-the-art. Importantly, all three algorithms presented in this article are now part of the official DASH reference player dash.js and are being used by video providers in production environments. While our evaluation and implementation are focused on the DASH environment, our algorithms are equally applicable to other adaptive streaming formats such as Apple HLS.

DGBA: Delay-Guaranteed Bitrate Adaptation for Mobile Broadcaster in Interactive Live Video Streaming

Improving the broadcaster-side performance is crucial for ensuring the video quality experience (QoE) for mobile interactive live streaming. But it is challenging in the presence of mobile network’s bandwidth fluctuation, as we need not only to ensure a high video bitrate but also to meet the broadcaster-side constraint. This paper presents Delay-Guaranteed Bitrate Adaptation (DGBA), an algorithm based on the idea of controllable bitrate adaptation used to optimize the broadcaster-side performance. We compare the performance of DGBA with a commercial bitrate adaptation algorithm by implementing these two algorithms in a prototype system. The experiment results show that DGBA can ensure 95.2% broadcaster-side delay constraint satisfaction compared to commercial algorithm’s 84.7%, while being more effective in bitrate adaptation.

SATE: Providing Stable and Agile Adaptation in HTTP-Based Video Streaming

Online video streaming service, such as Youtube and Netflix, is emerging as a killer application that constitutes most IP traffic. Users want high-quality video streaming service; however, network bandwidth cannot keep up user's demand. In the HTTP-based video streaming technology, the video provider divides the video into multiple chunks, encodes it at various bitrates, and stores it in the media server. The client requests a video chunk using an adaptive bitrate (ABR) algorithm, considering the network conditions. While there are many works to maximize user satisfaction, the state-of-the-art ABR algorithms are still vulnerable to rebuffering due to inaccurate estimates and decisions. To address this problem, we first analyze the real-world traces and find several design inferences for an efficient ABR algorithm. Then, we introduce SATE, an ABR algorithm that provides a stable and agile adaptation even in bandwidth constraint and dramatically changing network conditions. We evaluate the performance of SATE in various network settings and demonstrate its efficacy in practice. SATE provides a stable adaptation and dramatically reduced rebuffering while delivering a similar average bitrate compared to the state-of-the-art ABR algorithms.

A Survey on Bitrate Adaptation Schemes for Streaming Media Over HTTP

In this survey, we present state-of-the-art bitrate adaptation algorithms for HTTP adaptive streaming (HAS). As a key distinction from other streaming approaches, the bitrate adaptation algorithms in HAS are chiefly executed at each client, i.e. , in a distributed manner. The objective of these algorithms is to ensure a high quality of experience (QoE) for viewers in the presence of bandwidth fluctuations due to factors like signal strength, network congestion, network reconvergence events, etc. While such fluctuations are common in public Internet, they can also occur in home networksor even managed networks where there is often admission control and QoS tools. Bitrate adaptation algorithms may take factors like bandwidth estimations, playback buffer fullness, device features, viewer preferences, and content features into account, albeit with different weights. Since the viewer’s QoE needs to be determined in real-time during playback, objective metrics are generally used including number of buffer stalls, duration of startup delay, frequency and amount of quality oscillations, and video instability. By design, the standards for HAS do not mandate any particular adaptation algorithm, leaving it to system builders to innovate and implement their own method. This survey provides an overview of the different methods proposed over the last several years.

Opens source column

Sabre is an open source framework for simulating adaptive bitrate (ABR) video-player algorithms. ABR is used when delivering online video to different devices with different connectivity characteristics. The delivery bitrate is adapted dynamically to match the network connection, with the design of the ABR algorithms being a very active research area. Sabre is a useful tool for researchers to simulate performance of their ABR algorithms and to fairly compare performance of different algorithms.

SABA: segment and buffer aware rate adaptation algorithm for streaming over HTTP

Adaptive streaming allows for dynamic adaptation of the bitrate to varying network conditions, to guarantee the best user experience. Adaptive bitrate algorithms face a significant challenge in correctly estimating the throughput, as the throughput varies widely over time. The current throughput estimation methods cannot distinguish between throughput fluctuations of different amplitude and frequency. In this paper, we propose a throughput estimation method that accurately estimates the throughput based on previous throughput samples. It is robust to short term and small fluctuations, and sensitive to large fluctuations in throughput. Furthermore, we propose a rate adaptive algorithm for video on demand (VoD) services that selects the quality of the video based on the estimated throughput and playback buffer occupancy. The objective of the rate adaptive algorithms is to guarantee high video quality to improve user experience. The proposed algorithm dynamically adjusts the quality level of the video stream. The proposed method selects high quality video segments, while minimizing the risk of playback interruption. Furthermore, the proposed method minimizes the frequency of video rate changes. We show that the algorithm smoothly switches the video rate to improve user experience. Furthermore, we determine that it efficiently utilizes network resources to achieve a high video rate; competing HTTP clients achieve equitable video rates. We also confirm that variations in the playback buffer size and segment duration do not affect the performance of the proposed algorithm.

Proposed an Adaptive Bitrate Algorithm based on Measuring Bandwidth and Video Buffer Occupancy for Providing Smoothly Video Streaming

Dynamic adaptive streaming via HTTP (DASH) has been popular disseminated over the Internet especially under the circumstances of the time varying network, which it is currently the most challenging for providing smoothly video streaming via high quality. In DASH system, after completing the download of one segment, the player estimates the available network bandwidth by calculating the downloading throughput and then adapting the video bitrate level based on its estimations. However, the estimated bandwidth in the application layer is not accurate due to off-intervals appearance during the downloading process. To avoid the unfairness of bandwidth estimation by the clients, this work proposes a logarithmic approach for received network bandwidth, which includes increasing or decreasing this bandwidth logarithmically to converge the fair share bandwidth (estimated bandwidth). After obtaining the measured bandwidth, an adaptive bitrate algorithm is proposed by considering this measured bandwidth in addition to video buffer occupancy. The video buffer model is associated with three thresholds (i.e. one for initial startup and two for operating thresholds). When the video buffer’s level stays between the two operating thresholds, the video bitrate will keep unchanged. Otherwise, when the buffer occupancy is too high or too low, an appropriate video bitrate is chosen to avoid buffer overflow/underflow. Simulation results show that the proposed scheme is able to converge the measured bandwidth to the fair share bandwidth very quickly. Also the proposed scheme is compared with conventional scheme, we found that our proposed scheme outperforms in achieving the best performance in terms of efficiency, stability and fairness.

D-DASH: A Deep Q-Learning Framework for DASH Video Streaming

The ever-increasing demand for seamless high-definition video streaming, along with the widespread adoption of the dynamic adaptive streaming over HTTP (DASH) standard, has been a major driver of the large amount of research on bitrate adaptation algorithms. The complexity and variability of the video content and of the mobile wireless channel make this an ideal application for learning approaches. Here, we present D-DASH, a framework that combines deep learning and reinforcement learning techniques to optimize the quality of experience (QoE) of DASH. Different learning architectures are proposed and assessed, combining feed-forward and recurrent deep neural networks with advanced strategies. D-DASH designs are thoroughly evaluated against prominent algorithms from the state-of-the-art, both heuristic and learning-based, evaluating performance indicators such as image quality across video segments and freezing/rebuffering events. Our numerical results are obtained on real and simulated channel traces and show the superiority of D-DASH in nearly all the considered quality metrics. Besides yielding a considerably higher QoE, the D-DASH framework exhibits faster convergence to the rate-selection strategy than the other learning algorithms considered in the study. This makes it possible to shorten the training phase, making D-DASH a good candidate for client-side runtime learning.