Buffer Fullness Research Articles

Adaptive Streaming of Content-Aware-Encoded Videos in dash.js

In Hypertext Transfer Protocol (HTTP) adaptive streaming, the client makes rate adaptation decisions based on the measured network bandwidth and buffer fullness. This simplifies the adaptation logic; however, it often produces noticeable quality fluctuations during the streaming session. With content-aware encoding (CAE), one can improve the visual quality without increasing the total number of bits spent by carefully choosing where the bits are spent based on human perception. However, an adaptation logic that is unaware of the resulting variable-size segments may cause more stalls, defeating the purpose of improving viewer experience through CAE. This article explains the design steps of a size-aware rate adaptation (SARA) logic for one of the most popular Dynamic Adaptive Streaming over HTTP (DASH) clients, namely dash.js, and shows the improvements in rebuffering behavior and fetching top-resolution segments as a result of applying this logic.

A Survey on Bitrate Adaptation Schemes for Streaming Media Over HTTP

In this survey, we present state-of-the-art bitrate adaptation algorithms for HTTP adaptive streaming (HAS). As a key distinction from other streaming approaches, the bitrate adaptation algorithms in HAS are chiefly executed at each client, i.e. , in a distributed manner. The objective of these algorithms is to ensure a high quality of experience (QoE) for viewers in the presence of bandwidth fluctuations due to factors like signal strength, network congestion, network reconvergence events, etc. While such fluctuations are common in public Internet, they can also occur in home networksor even managed networks where there is often admission control and QoS tools. Bitrate adaptation algorithms may take factors like bandwidth estimations, playback buffer fullness, device features, viewer preferences, and content features into account, albeit with different weights. Since the viewer’s QoE needs to be determined in real-time during playback, objective metrics are generally used including number of buffer stalls, duration of startup delay, frequency and amount of quality oscillations, and video instability. By design, the standards for HAS do not mandate any particular adaptation algorithm, leaving it to system builders to innovate and implement their own method. This survey provides an overview of the different methods proposed over the last several years.

Optimize adaptive media playout using dynamic fuzzy logic control for video streaming

Adaptive Media Playout (AMP) controls adapt playout rate to prevent buffer outage and to reduce delay in playout. Most AMP techniques use buffer fullness or its variation as indicator to adapt to playout rate. Nonetheless, selecting a convenient buffer threshold of particular design is a great challenge. Based on this principle, Dynamic Fuzzy Logic (DFL) controls principle fused with AMP technique to propose a new technique, known as DFLAMP. The technique is implemented to reduce playout delay and prevent buffer outage. Moreover, DFLAMP is also utilized to adjust playout rate based on arrival rate variation and adapt playout rate to keep the buffer fullness around the desired level. Simulation results have indicated that DFLAMP technique efficiently outperforms other AMP approaches. DFLAMP shows 14.5% less buffer underflow, 25% less buffer overflow, and 26.4% less distortion of variance as compared to APTA. Moreover, it depicts 7.5% decreased total playout delay as compared to DPTA-APTA.

Real-Time QoE Monitoring System for Video Streaming Services with Adaptive Media Playout

Quality of Experience (QoE) of video streaming services has been attracting more and more attention recently. Therefore, in this work we designed and implemented a real-time QoE monitoring system for streaming services with Adaptive Media Playout (AMP), which was implemented into the VideoLAN Client (VLC) media player to dynamically adjust the playout rate of videos according to the buffer fullness of the client buffer. The QoE monitoring system reports the QoE of streaming services in real time so that network/content providers can monitor the qualities of their services and resolve troubles immediately whenever their subscribers encounter them. Several experiments including wired and wireless streaming were conducted to show the effectiveness of the implemented AMP and QoE monitoring system. Experimental results demonstrate that AMP significantly improves the QoE of streaming services according to the Mean Opinion Score (MOS) estimated by our developed program. Additionally, some challenging issues in wireless streaming have been easily identified using the developed QoE monitoring system.

Adaptive video streaming over HTTP through 4G wireless networks based on buffer analysis

One of the main challenges in video transmission is understanding and adapting to the varying network bandwidth. The traditional approach of bandwidth estimation is not accurate as there are many factors like congestion that can delay the arrival rate of the ping packet which may lead to a misconception that the bandwidth was low. Thus, the better approach to this problem will be to estimate the link conditions based on the buffer fullness. In this paper, a new system to support streaming of live and stored video through wireless network is proposed which is based on adaptive playback buffer management on the top of HTTP at the client. The buffer fullness is treated as a direct state variable that reflects the fluctuation of the network bandwidth. The buffer fullness estimation predicts the buffer status at a point in the future based on observations of the buffer over a stipulated period of time. The proposed algorithm uses non-linear exponential non-parametric regression for computing the decision parameter. A feedback message is then sent to the server in order to change the quality of the video stream for smoother video play at the client side. The synchronized update and feedback between the server and clients is achieved using HTTP protocol. During the experimentation with live video streaming, the proposed algorithm shows an improvement of 24.48% in average peak signal-to-noise ratio and 6.63% in average structural similarity index against the buffer underflow probability algorithm.

How to perform AMP? Cubic adjustments for improving the QoE

Adaptive Media Playout (AMP) consists of smoothly and dynamically adjusting the media playout rate to recover from undesired (e.g., buffer overflow/underflow or out-of-sync) situations. The existing AMP solutions are mainly characterized by two main aspects. The first one is their goal (e.g., keeping the buffers’ occupancy into safe ranges or enabling media synchronization). The second one is the criteria that determine the need for triggering the playout adjustments (e.g., buffer fullness or asynchrony levels). This paper instead focuses on a third key aspect, which has not been sufficiently investigated yet: the specific adjustment strategy to be performed. In particular, we propose a novel AMP strategy, called Cubic AMP, which is based on employing a cubic interpolation method to adjust a deviated playout point to a given reference. On the one hand, mathematical analysis and graphical examples show that our proposal provides superior performance than other existing linear and quadratic AMP strategies in terms of the smoothness of the playout curve, while significantly outperforming the quadratic AMP strategy regarding the duration of the adjustment period and without increasing the computational complexity. It has also been proved and discussed that higher-order polynomial interpolation methods are less convenient than cubic ones. On the other hand, the results of subjective tests confirm that our proposal provides better Quality of Experience (QoE) than the other existing AMP strategies.

Adaptive Layer Switching Algorithm Based on Buffer Underflow Probability for Scalable Video Streaming Over Wireless Networks

Scalable Video Coding (SVC) has been raised as a promising technique to enable flexible video transmission for mobile users with heterogeneous terminals and varying channel capacities. In this paper, we design an adaptive layer switching algorithm for on-demand scalable video service based on receiver’s buffer underflow probability (BUP). Since the low quality of channel may lead to a low buffer fullness, the buffer fullness is an indicator for reflecting the channel condition and we define BUP for characterizing the mismatch between the video bitrate and the channel throughput. Accordingly, the adaptive SVC transmission problem is formulated as the adaptive adjustment of video layers based on BUP. This allows us to optimize the attainable video quality, while keeping BUP below a desired level. To estimate BUP, we derive an analytical model based on the large deviation principles. Then, an online layer switching algorithm is proposed using this estimation model, which is capable of accommodating different channel qualities without any prior knowledge of the channel variations and of the video characteristics. We further introduce a perturbation-based layer switching approach for reducing the quality fluctuating issue caused by frequent layer switches, thus improving the viewer’s quality of experience. A system prototype is implemented to evaluate the success of the proposed method. We also conduct simulations in multiuser scenarios with real video traces and the results demonstrate that the proposed algorithm is capable of improving the playback experience, while keeping a low playback interruption rate and quality variation.

Low-Delay Rate Control for Consistent Quality Using Distortion-Based Lagrange Multiplier.

Video quality fluctuation plays a significant role in human visual perception, and hence, many rate control approaches have been widely developed to maintain consistent quality for video communication. This paper presents a novel rate control framework based on the Lagrange multiplier in high-efficiency video coding. With the assumption of constant quality control, a new relationship between the distortion and the Lagrange multiplier is established. Based on the proposed distortion model and buffer status, we obtain a computationally feasible solution to the problem of minimizing the distortion variation across video frames at the coding tree unit level. Extensive simulation results show that our method outperforms the rate control used in HEVC Test Model (HM) by providing a more accurate rate regulation, lower video quality fluctuation, and stabler buffer fullness. The average peak signal-to-noise ratio (PSNR) and PSNR deviation improvements are about 0.37 dB and 57.14% in the low-delay (P and B) video communication, where the complexity overhead is ∼4.44% .

Performance of Buffer-Aided Adaptive Modulation in Multihop Communications

In multihop diversity (MHD) aided multihop links, the nodes are assumed to have buffers for temporarily storing their received packets for further transmission at instances of good channel quality. Since adaptive modulation is employed, the number of packets in each time slot (TS) is affected both by the channel quality and the buffer fullness. During each time-slot (TS), the criterion used for activating a specific hop is that of transmitting the highest number of packets. When more than one hop is capable of transmitting the same number of packets, the particular hop having the highest channel quality (reliability) is activated. Hence, we refer to this regime as the maximum throughput adaptive rate transmission (MTART) scheme. The bit error ratio (BER), the outage probability, the throughput as well as the bandwidth-efficiency of the MTART scheme is analyzed. Our results demonstrate that our MTART regime has the potential of significantly outperforming conventional adaptive modulation. Naturally, the BER is improved by the buffering scheme advocated at the cost of an increased delay. Hence, the distribution of the end-to-end packet delay will also be characterized.

Radio Resource Allocation on Complex 4G Wireless Cellular Networks

In this article we consider the heuristic algorithm which improves step by step wireless data delivery over LTE cellular networks by using the total transmit power with the constraint on users’ data rates, and the total throughput with the constraints on the total transmit power as well as users’ data rates, which are jointly integrated into a hybrid-layer design framework to perform radio resource allocation for multiple users, and to effectively decide the optimal system parameter such as modulation and coding scheme (MCS) in order to adapt to the varying channel quality. We propose new heuristic algorithm which balances the accessible data rate, the initial data rates of each user allocated by LTE scheduler, the priority indicator which signals delay- throughput- packet loss awareness of the user, and the buffer fullness by achieving maximization of radio resource allocation for multiple users. It is noted that the overall performance is improved with the increase in the number of users, due to multiuser diversity. Experimental results illustrate and validate the accuracy of the proposed methodology.

An efficient delay-constrained ARQ scheme for MMT packet-based real-time video streaming over IP networks

The MPEG has recently Querydeveloped a new standard, MPEG media transport (MMT), for the next-generation hybrid media delivery service over IP networks considering the emerging convergence of digital broadcast and broadband services. On account of the heterogeneous characteristics of broadcast and broadband networks, MMT provides an efficient delivery timing model to enable inter-network synchronization, measure various kinds of transmission delays and jitters caused by the transmission delay, and re-adjust the timing relationship between the MMT packets to ensure synchronized playback. By exploiting the delivery timing model, it is possible to accurately estimate the round-trip time (RTT) experienced during MMT packet transmission. Based on the measured RTT, we propose an efficient delay-constrained automatic repeat request (ARQ) scheme, which is applicable to MMT packet-based real-time video streaming service over IP networks. In the proposed ARQ scheme, the receiver buffer fullness at the time of packet loss detection is used to compute the arrival deadline, which is the maximum allowed time for completing the requesting and retransmitting of the lost MMT packet. Simulation results demonstrate that the proposed delay-constrained ARQ scheme can not only provide reliable error recovery, but it also achieves significant bandwidth savings by reducing the number of wastefully retransmitted packets that arrive at the receiver side and exceed the allowed arrival deadline.

Optimal Rate Control in H.264 Video Coding Based on Video Quality Metric

The aim of this research is to find a method for providing better visual quality across the complete video sequence in H.264 video coding standard. H.264 video coding standard with its significantly improved coding efficiency finds important applications in various digital video streaming, storage and broadcast. To achieve comparable quality across the complete video sequence with the constrains on bandwidth availability and buffer fullness, it is important to allocate more bits to frames with high complexity or a scene change and fewer bits to other less complex frames. A frame layer bit allocation scheme is proposed based on the perceptual quality metric as indicator of the frame complexity. The proposed model computes the Quality Index ratio (QIr) of the predicted quality index of the current frame to the average quality index of all the previous frames in the group of pictures which is used for bit allocation to the current frame along with bits computed based on buffer availability. The standard deviation of the perceptual quality indicator MOS computed for the proposed model is significantly less which means the quality of the video sequence is identical throughout the full video sequence. Thus the experiment results shows that the proposed model effectively handles the scene changes and scenes with high motion for better visual quality.

A novel total variation based frame layer rate control algorithm for H.264/AVC

Rate control (RC) plays a crucial role in controlling compression bitrates and encoding qualities for networked video applications. In this research, we propose a new total variation (TV) based frame layer rate control algorithm for H.264/AVC. One of its novelties is that a total variation measure, used in image processing field, is proposed to describe encoding distortion in video compression. For intraframes, we present a TV distortion–quantization (DTV–Qstep) model to obtain accurate QP step size (Qstep). Using TV measure to represent frame complexity, we also present an analytic model to calculate Qstep for the initial frame, and develop an effective scene change detection method. In addition, an incomplete derivative proportional integral derivative (IDPID) buffer controller is proposed to reduce the deviation between the current buffer fullness and the target buffer fullness, and minimizes the buffer overflow or underflow. Extensive experimental results show that, compared with JVT-W042, the proposed algorithm successfully achieves more accurate target bit rates, reduces frame skipping, decreases quality fluctuation and improves the overall coding quality.

A novel rate control algorithm for video coding based on fuzzy-PID controller

Rate control algorithms (RCAs) aim to achieve the best visual quality under the minimum bit rate and the limited buffer size. A self-parameter-tuning fuzzy-PID controller is proposed to reduce the deviation between the target buffer level and the current buffer fullness. Fuzzy logic is used to tune each parameter of the proportional-integral-derivative controller by selecting appropriate fuzzy rules through simulation in H.264/advanced video coding (AVC). To control the quality fluctuation between consecutive frames, a quality controller is adopted. The proposed RCA has been implemented in an H.264/AVC video codec, and our experimental results show that the proposed algorithm achieves smooth target bits while enabling better buffer control and visual quality.

Adaptive Rate Control Algorithm for H.264/AVC Considering Scene Change

Scene change in H.264 video sequences has significant impact on the video communication quality. This paper presents a novel adaptive rate control algorithm with little additional calculation for H.264/AVC based on the scene change expression. According to the frame complexity quotiety, we define a scene change factor. It is used to allocate bits for each frame adaptively. Experimental results show that it can handle the scene change effectively. Our algorithm, in comparison to the JVT-G012 algorithm, reduces rate error and improves average peak signal-noise ratio with smaller deviation. It cannot only control bit rate accurately, but also get better video quality with the lower encoder buffer fullness to improve the quality of service.

Buffer Occupancy of Double-Buffer Traffic Shaper in Real-Time Multimedia Applications across Slow-Speed Links

In this paper, a double-buffer traffic shaper was investigated to adjust video frame rate inflow into the TCP sender-buffer of a multimedia application source across a slow-speed link. In order to guarantee QoS across a slow-speed link (i.e. 1 MBPS), the double-buffer traffic shaper was developed. In this paper, the buffer size dynamics of double-buffer was investigated. The arrival and departure of frames were modeled as a stochastic process. The transition matrix for the process was generated and the stationary probability computed. A simulation program was written in Matlab 7.0 to monitor the buffer fullness of the second buffer when a 3600 seconds H.263 encoder trace data was used as test data. In the second buffer, it was discovered that over 90% of the play-time, the buffer occupancy was upper bounded at 300 frames per second and utilization maintained below 30%.

An efficient intra-frame rate control algorithm for H.264/AVC video coding

Intra-frame plays a significant role in improving video quality for rate control of H.264/AVC. In this paper, an efficient intra-frame rate control algorithm is proposed for H.264/AVC video coding. The proposed scheme adjusts the quantization parameter (QP) of intra-frame adaptively according to intra-frame buffer fullness ratio, current buffer fullness ratio, and skipped frame ratio. Experimental results demonstrate that for a set of typical testing sequences, the proposed scheme can obtain accurate target bit rate, better peak signal-to-noise ratio (PSNR) performance and reduce the number of skipped frames significantly, when compared with existing schemes. More importantly, the proposed scheme is very flexible for different initial QPs and target bits.

Novel rate control scheme for intra frame video coding with exponential rate–distortion model on H.264/AVC

Rate control regulates the output bit rate of a video encoder in order to obtain optimum visual quality within the available network bandwidth and to maintain buffer fullness within a specified tolerance range. Due to the benefits of intra-only encoding, such as less computational cost and less latency, it has been more and more widely used. In this paper, we propose an accurate intra-only rate control scheme for H.264/AVC, which includes a novel complexity measurement and a new rate–distortion (R–D) model. We also propose a linear rate–complexity model which takes the intercept into consideration to reduce the estimation error. The proposed R–D model is integrated by the linear rate–complexity model and an exponential rate–quantization model. Based on theoretical analysis and experimental validation, the proposed scheme has high bits prediction precision, and it can also accurately handle buffer fullness. Compared with JVT-W042, our algorithm achieves higher average PSNR and improves the coding quality up to 0.35dB.

Arrival process-controlled adaptive media playout with multiple thresholds for video streaming

To improve the playout quality of video streaming services, an arrival process-controlled adaptive media playout (AMP) mechanism is designed in this study. The proposed AMP scheme sets three threshold values, denoted by P n , L and H, for the playout controller to start playback and dynamically adjust the playout rate based on the buffer fullness. In the preroll period, the playout can start only when the buffer fullness n is not less than the dynamic playback threshold P n , which is determined by the jitters of incoming video frames. In the playback period, if the buffer fullness is below L or over H, the playout rate will slow down or speed up in a quadratic manner. Otherwise, the playback speed depends on the instantaneous frame arrival rate, which is estimated by the proposed arrival process tracking algorithm. We employ computer simulations to demonstrate the performance of the proposed AMP scheme, and compare it with several conventional AMP mechanisms. Numerical results show that our AMP design can shorten the playout delay and reduce both buffer underflow and overflow probabilities. In addition, our proposed AMP also outperforms traditional AMP schemes in terms of the variance of distortion of playout and the playout curve. Hence, the proposed arrival process-controlled AMP is really an outstanding design.

Joint H.264/scalable video coding-multiple input multiple output rate control for wireless video applications

Integrating H.264 scalable video coding (SVC) technology with a multiple input multiple output (MIMO) wireless system can significantly enhance the overall performance of high-quality real-time wireless video transmissions. However, the state-of-the-art techniques in these two areas are largely developed independently. In this research, with the objective to deliver the optimal visual quality and accurate rate regulation for wireless video applications, the authors propose a novel joint H.264/SVC-MIMO rate control (RC) algorithm for video compression and transmission over MIMO systems. The authors first present a systematic architecture for H.264/SVC compression and transmission over MIMO systems. Then, based on MIMO channel properties, the authors use a packet-level two-state Markov model to estimate MIMO channel states and predict the number of retransmitted bits in the presence of automatic repeat request. Finally, an efficient joint rate controller is proposed to regulate the output bit rate of each layer according to the available channel throughput and buffer fullness. The authors' extensive simulation results demonstrate that their algorithm outperforms JVT-W043 RC algorithm, adopted in the H.264/SVC reference software, by providing more accurate output bit rate, reducing buffer overflow, lessen frame skipping, and finally, improves the overall coding quality.