Bit Rate Video Research Articles

A Video Bitrate Adaptation and Prediction Mechanism for HTTP Adaptive Streaming

The Hypertext Transfer Protocol (HTTP) Adaptive Streaming (HAS) has now become ubiquitous and accounts for a large amount of video delivery over the Internet. But since the Internet is prone to bandwidth variations, HAS's up and down switching between different video bitrates to keep up with bandwidth variations leads to a reduction in Quality of Experience (QoE). In this article, we propose a video bitrate adaptation and prediction mechanism based on Fuzzy logic for HAS players, which takes into consideration the estimate of available network bandwidth as well as the predicted buffer occupancy level in order to proactively and intelligently respond to current conditions. This leads to two contributions: First, it allows HAS players to take appropriate actions, sooner than existing methods, to prevent playback interruptions caused by buffer underrun, reducing the ON-OFF traffic phenomena associated with current approaches and increasing the QoE. Second, it facilitates fair sharing of bandwidth among competing players at the bottleneck link. We present the implementation of our proposed mechanism and provide both empirical/QoE analysis and performance comparison with existing work. Our results show that, compared to existing systems, our system has (1) better fairness among multiple competing players by almost 50% on average and as much as 80% as indicated by Jain's fairness index and (2) better perceived quality of video by almost 8% on average and as much as 17%, according to the estimate the Mean Opinion Score (eMOS) model.

An Efficient Framework for Compressed Domain Watermarking in P Frames of High-Efficiency Video Coding (HEVC)--Encoded Video

Digital watermarking has received much attention in recent years as a promising solution to copyright protection. Video watermarking in compressed domain has gained importance since videos are stored and transmitted in a compressed format. This decreases the overhead to fully decode and re-encode the video for embedding and extraction of the watermark. High Efficiency Video Coding (HEVC/H.265) is the latest and most efficient video compression standard and a successor to H.264 Advanced Video Coding. In this article, we propose a robust watermarking framework for HEVC-encoded video using informed detector. A readable watermark is embedded invisibly in P frames for better perceptual quality. Our framework imposes security and robustness by selecting appropriate blocks using a random key and the spatio-temporal characteristics of the compressed video. A detail analysis of the strengths of different compressed domain features is performed for implementing the watermarking framework. We experimentally demonstrate the utility of the proposed work. The results show that the proposed work effectively limits the increase in video bitrate and degradation in perceptual quality. The proposed framework is robust against re-encoding and image processing attacks.

SDP-Based Quality Adaptation and Performance Prediction in Adaptive Streaming of VBR Videos

Recently, various adaptation methods have been proposed to cope with throughput fluctuations in HTTP adaptive streaming (HAS). However, these methods have mostly focused on constant bitrate (CBR) videos. Moreover, most of them are qualitative in the sense that performance metrics could only be obtained after a streaming session. In this paper, we propose a new adaptation method for streaming variable bitrate (VBR) videos using stochastic dynamic programming (SDP). With this approach, the system should have a probabilistic characterization along with the definition of a cost function that is minimized by a control strategy. Our solution is based on a new statistical model where the future streaming performance is directly related to the past bandwidth statistics. We develop mathematical models to predict and develop simulation models to measure the average performance of the adaptation policy. The experimental results show that the prediction models can provide accurate performance prediction which is useful in planning adaptation policy and that our proposed adaptation method outperforms the existing ones in terms of average quality and average quality switch.

User-Oriented Video Streaming Service Based on Passive Aggressive Learning

The authors propose a method to dynamically determine appropriate quality of service (QoS) required by users for video streaming services. In the proposed method, the video bit rate as the QoS parameter is determined based on the passive aggressive learning, which is an online learning algorithm for a regression problem, according to the user requirements, computational/network resources and service provisioning environments. Moreover, the method makes it possible to provide appropriate QoS by using optimization solution. In this paper, the authors describe the design and implementation of the method, then confirm the feasibility of the proposed method through the experiments.

A Probabilistic Adaptation Method for HTTP Low-Delay Live Streaming over Mobile Networks

The study focuses on the adaptation problem for HTTP low-delay live streaming over mobile networks. In this context, the client's small buffer could be easily underflown due to throughput variations. To maintain seamless streaming, we present a probabilistic approach to adaptively decide the bitrate for each video segment by taking into account the instant buffer level. The experimental results show that the proposed method can significantly reduce buffer underflows while providing high video bitrates.

JAVRE: A Joint Asymmetric Video Rendering and Encoding Approach to Enable Optimized Cloud Mobile 3D Virtual Immersive User Experience

With the growing adoption of mobile devices as the primary device for information and media consumption, and recent growth in availability of mobile devices with auto-stereoscopic 3D displays, the time has come to enable rich 3D virtual immersive experiences on mobile devices, like 3D display gaming, 3D virtual classroom, 3D virtual art gallery, etc., which are currently enabled on PC platforms. Instead of developing such compute-intensive applications on mobile devices, a scalable and portable approach is to take advantage of the elastic cloud resources. In this paper, we explore the possibility of developing a cloud based mobile 3D virtual immersive application architecture, where the 3D video rendering and encoding are performed on cloud servers, with the resulting 3D video streamed to mobile 3D devices through wireless networks. However, with the significantly higher bit rate requirement for 3D videos, ensuring user experience is a challenge considering the inherent dynamic variability of mobile networks. In this paper, in order to address this challenge, we propose a joint asymmetric graphics rendering and video encoding approach called JAVRE, where both the encoding quality and rendering richness of the left view and the right view can be asymmetric. Specifically, we first develop two mathematical models (user experience model and video bit rate model) through subjective and objective tests. Then we propose an optimization algorithm that can automatically choose the video encoding settings and graphics rendering settings for the left view and the right view to ensure the best user experience given the network conditions. Experiments conducted using real 4G-LTE network profiles and a commercial cloud service demonstrate significant improvement in user experience when JAVRE is used.

Impact Analysis of Baseband Quantizer on Coding Efficiency for HDR Video

Digitally acquired high dynamic range (HDR) video baseband signal can take 10 to 12 bits per color channel. It is economically important to be able to reuse the legacy 8 or 10-bit video codecs to efficiently compress the HDR video. Linear or nonlinear mapping on the intensity can be applied to the baseband signal to reduce the dynamic range before the signal is sent to the codec, and we refer to this range reduction step as a baseband quantization. We show analytically and verify using test sequences that the use of the baseband quantizer lowers the coding efficiency. Experiments show that as the baseband quantizer is strengthened by 1.6 bits, the drop of PSNR at a high bitrate is up to 1.60dB. Our result suggests that in order to achieve high coding efficiency, information reduction of videos in terms of quantization error should be introduced in the video codec instead of on the baseband signal.

Enabling QoE-aware mobile cloud video recording over roadside vehicular networks

Most of previous video recording devices in mobile vehicles commonly store captured video contents locally. With the rapid development of 4G/WiFi networks, there emerges a new trend to equip video recording devices with wireless interfaces to enable video uploading to the cloud for video playback in a later time point. In this paper, we propose a QoE-aware mobile cloud video recording scheme in the roadside vehicular networks, which can adaptively select the proper wireless interface and video bitrate for video uploading to the cloud. To maximize the total utility, we need to design a control strategy to carefully balance the transmission cost and the achieved QoE for users. To this purpose, we investigate the tradeoff between cost incurred by uploading through cellular networks and the achieved QoE of users. We apply the optimization framework to solve the formulated problem and design an online scheduling algorithm. We also conduct extensive trace-driven simulations and our results show that our algorithm achieves a good balance between the transmission cost and user QoE.

HTTP 적응적 스트리밍을 위한 TCP 인지형 세그먼트 스케줄링 기법

HTTP Adaptive Streaming (HAS) is a technique that adapts its video quality to network conditions for providing Quality of Experience. In the HAS approach, a video content is encoded at multiple bitrates and the encoded video content is divided into several video segments. A HAS player estimates the network bandwidth and adjusts the video bitrate based on estimated bandwidth. However, the segment scheduler in the conventional HAS player requests video segments periodically without considering TCP. If the waiting duration for the next segment request is quite long, the TCP connection can be initialized and it restarts slow-start. Slow-start causes the reduction in TCP throughput and consequentially leads to low-quality video streaming. In this study, we propose a TCP-aware segment scheduling scheme to improve performance of HAS service. The proposed scheme adjusts request time for the next video request to prevent initialization of TCP connection and also considers the point of scheduling time. The simulation proves that our scheme improves the Quality of Service of the HAS service without buffer underflow issue.

Low-Complexity and Context-Aware Estimation of Spatial and Temporal Activity Parameters for Automotive Camera Rate Control

Rate control in video compression adjusts the encoding parameters to reach a certain target bitrate for the encoded video. State-of-the-art rate controllers for hybrid video coding typically employ content-dependent video bitrate models and video quality metrics (VQMs). To capture the content characteristics, temporal and spatial video activity measures are determined from the raw video using computationally complex algorithms that require access to the uncompressed source video. In automotive deployments, however, full access to the uncompressed source video and the internal functions of video encoders is typically not possible. As a remedy, in this paper, we present a low-complexity approach to estimate the temporal activity (TA) and spatial activity (SA) measures for videos that are captured by a front-facing camera of a vehicle, based on the context information of the vehicle. To this end, we exploit information about the dynamics of the vehicle and other vehicles in the field-of-view of the front-facing camera. We apply the estimated TA and SA values to a video bitrate model and an objective VQM and use these models to solve the rate control problem to determine the optimal encoding settings for given bitrate constraints. The proposed low-complexity solution offers a similar accuracy in achieving rate constraints and similar perceptual quality characteristics as a solution that uses the computed TA and SA values, with the advantage that no access to the uncompressed source video stream or the internal functions of the video encoder is required.

Adaptive Layer Switching Algorithm Based on Buffer Underflow Probability for Scalable Video Streaming Over Wireless Networks

Scalable Video Coding (SVC) has been raised as a promising technique to enable flexible video transmission for mobile users with heterogeneous terminals and varying channel capacities. In this paper, we design an adaptive layer switching algorithm for on-demand scalable video service based on receiver’s buffer underflow probability (BUP). Since the low quality of channel may lead to a low buffer fullness, the buffer fullness is an indicator for reflecting the channel condition and we define BUP for characterizing the mismatch between the video bitrate and the channel throughput. Accordingly, the adaptive SVC transmission problem is formulated as the adaptive adjustment of video layers based on BUP. This allows us to optimize the attainable video quality, while keeping BUP below a desired level. To estimate BUP, we derive an analytical model based on the large deviation principles. Then, an online layer switching algorithm is proposed using this estimation model, which is capable of accommodating different channel qualities without any prior knowledge of the channel variations and of the video characteristics. We further introduce a perturbation-based layer switching approach for reducing the quality fluctuating issue caused by frequent layer switches, thus improving the viewer’s quality of experience. A system prototype is implemented to evaluate the success of the proposed method. We also conduct simulations in multiuser scenarios with real video traces and the results demonstrate that the proposed algorithm is capable of improving the playback experience, while keeping a low playback interruption rate and quality variation.

IRate: Initial Video Bitrate Selection System for HTTP Streaming

Many HTTP streaming video systems have been developed and widely deployed in recent years. Previous efforts were mainly spent on improving the caching of videos or proposing mid-stream measurement methods to update the best bitrate. However, since the video length is often short, the mid-stream measurement may not even converge to the best bitrate due to insufficient bandwidth estimates. On the other hand, because of diversified Web infrastructure, estimating the actual network quality at the pre-stream stage is increasingly challenging for video service providers. In this paper, we propose IRate, which enables video service providers to proactively profile clients’ streaming performance by carrying out pre-stream measurement in the Content Delivery Network (CDN). With the measurement results, the video stream can start at the best video quality at the onset of streaming. This is especially beneficial to short video clips, which are very popular in the Internet today. IRate is composed of a probe kit and a quality oracle. The probe kit utilizes the pre-stream time window (e.g., user’s think time and pre-roll advertisement) for measuring network quality by running a lightweight measurement script on the Web page to induce probe packets from the IRate middlebox on the server side. With the measurement results, the quality oracle estimates the clients’ streaming performance by determining the highest initial bitrate with a pre-trained decision tree. Our testbed results show that IRate is able to achieve 80% accuracy in determining the bitrate within 10s. By having a better estimate of the best initial bitrate, the buffering time and rebuffering events are significantly reduced in HTTP streaming. Furthermore, the stability and the efficiency in dynamic adaptive streaming over HTTP streaming are also improved by about 40% and 36%, respectively. Our user quality of experience (QoE) experiment further validates that IRate can improve the QoE by more than 6% and the perceived quality of initial quality by 24% in the actual Internet environment.

Inter-frame distortion drift analysis for reversible data hiding in encrypted H.264/AVC video bitstreams

Due to the privacy-preserving requirement for cloud data management, it is necessary to perform data hiding in the encrypted domain. This paper proposes an effective scheme for reversible data hiding in encrypted H.264/AVC video bitstreams. In the encryption phase, the codewords of intra prediction modes, motion vector differences and partial residual coefficients are encrypted without video bit rate increment for preserving the confidentiality of video content. In the data hiding phase, we first present a theoretical analysis of the picture distortion caused by data embedding and the subsequent inter-frame distortion drift. Based on the analysis, we estimate the embedding distortions caused by modifying different residual coefficients and embed data into residual coefficients with different priorities for decreasing the inter-frame distortion drift. The data embedding is implemented by the histogram shifting technique. If the receiver decrypts the encrypted video bitstream and extracts the embedded data, the original video bitstream can be perfectly recovered. In addition, if the receiver decrypts the encrypted video bitstream without extracting the embedded data, the bitstream can still be decoded to obtain the reconstructed video with good quality.

Server Side Solutions For Web-Based Video

In contemporary video streaming systems based on HTTP protocol, video players at the client side are responsible for adjusting video quality to network conditions and user expectations. However, when multiple video clips are streamed simultaneously, an intricate application logic implemented in the video players overlays the TCP mechanism which is responsible for a balanced access to a shared network link. As a result, some video players may not obtain a fair share of network throughput and may be vulnerable to an unstable video bit-rate. Therefore, we propose to simplify the algorithms implemented in the video players, which are responsible for the adjustment of video quality and constrain their functionality only to sending feedback to a server about a state of the player buffer. The main logic of the system is shifted to the server, which is now responsible for bit-rate selection and prioritisation of the video streams transmitted to multiple clients. To verify our proposition, we performed several experiments in a laboratory environment which show that when the server cooperates with the clients, the video players experience fewer quality switches and the system achieves better fairness when allocating network throughput among the video players. However, this comes at the cost of worse utilisation of network bandwidth.

Bitrate allocation among multiple video streams to maximize profit in content delivery networks

Concomitant with rapid advancements in video streaming technology, concurrence video traffic has increased significantly on communication channels. Conflicts often arise among the various video streams on these communication channels when the available bandwidth is limited because the bitrates and transmission range required often vary. This study proposes a server-side-based rate allocation algorithm for content delivery networks (CDNs). Instead of simply considering bitrate selection from the perspective of network and client conditions, the algorithm combines user experience with video bitrate allocation to maximize viewer engagement. First, the values of users are evaluated and a user value computation method is proposed. Second, we developed a profit maximization bitrate allocation approach (PMBAA) that enables both content providers and CDNs to maximize profits by providing guaranteed video quality. At last, the results of test bed experiments and analyses verify that PMBAA enables high-value clients to obtain more preferable bitrates than the HTTP live streaming algorithm developed by Apple Inc.

A model for evaluating sharing policies for network-assisted HTTP adaptive streaming

HTTP adaptive streaming (HAS) has become the dominant technology for streaming video over the Internet. It gained popularity because of its ability to adapt the video quality to the current network conditions and other appealing properties such as usage of off-the-shelf HTTP servers and easy firewall traversal. However, when multiple HAS players share a bottleneck link for streaming, the individual adaptation techniques in the players have difficulties to maintain a stable bitrate and fairly share the network resources. HAS-assisting network elements can solve these performance problems and allow execution of advanced policies for sharing the available bandwidth. Nonetheless, testing and evaluating new sharing policies is costly and time consuming. This motivated us to formulate a model that allows to differentiate between groups of users depending on the type of user or device, and that can describe the mean bitrate of the video streams and how often this bitrate is expected to change during playout. To show how our model can be used, we demonstrate two applications of our model. Furthermore, we validate the model based results against results obtained using our streaming testbed and proxy server based HAS-assistant. The results show that our model is highly accurate for both the mean bitrate and the number of changes in video bitrate. As such, our model is a useful tool for network administrators and internet service providers for evaluating the performance of sharing policies and for managing and provisioning video delivery networks.

An Adaptive Buffering Method for Practical HTTP Live Streaming on Smart OTT STBs

In this paper, we address the channel zapping time problem of video streaming services based on HTTP Live Streaming (HLS) on smart Over-The-Top Set-Top Boxes (OTT STBs). Experimental analysis of the channel zapping time, show that smart OTT STBs inevitably suffer from the accumulated zapping time through channel change request, Internet Group Management Protocol (IGMP) leave/join, synchronization delay, video buffer delay, and STB processing delay when providing HLS services. As a practical solution for the zapping time reduction, an adaptive buffering method is proposed. The proposed method exploits two adaptive buffers added to the basic HLS player. These two adaptive buffers are responsible for constantly buffering previous and next channels relative to the current channel. Implementation and test results show that a stable zapping time less than one second can be achieved even under diverse video bitrate changes and varying network conditions by the proposed adaptive buffering method.

Using the audio of 8-bit video games to monitor web marketing campaigns

Monitoring the performance of a web marketing campaign is usually a long-lasting, low-effort but distracting task, where a user repeatedly glances at some sort of visual analytics tools to check whether the campaign is going well. In this paper, we explore an alternative approach for this task, where the performance of a web marketing campaign is monitored through sonification, using the soundset of popular 8-bit arcade video games. On one hand, sonification would allow a user to be constantly informed about the current state of the campaign without being distracted. On the other hand, the sound metaphors coming from popular 8-bit arcade video games would be able to convey information about the status of the campaign in a simple and effective way (i.e., if the sonification of a campaign resembles the audio of a successful game session, then the campaign is going well). We investigated this idea by developing a prototype system for the sonification of the behavior of a web server activity through a configurable set of sound metaphors. We then analyzed the effectiveness of our approach by conducting a simple experimental study. This was done, first, by sonifying the progress of a given web marketing campaign using the soundset of two popular 8-bit video games: Super Mario Bros and Bubble Bobble. The outcoming soundtrack was then used in a controlled setting to assess the performance of a group of 20 participants listening to our soundtrack under different work conditions.

Identification of canonical neural events during continuous gameplay of an 8-bit style video game

Cognitive neuroscience suffers from a unique and pervasive problem of generalizability. Since neural findings are often interpreted in the context of a specific manipulation during a carefully controlled task, it is hard to transfer knowledge from one task to another. In this report we address problems of generalizability with two methodological advancements. First, we aimed to transcend status quo experimental procedures with a continuous, engaging task environment. To this end, we created a novel 8-bit style continuous space shooter video game that elicits a multitude of goal-oriented events, such as crashing into a wall or blowing up an enemy with a missile. Second, we aimed to objectively define the psychological significance of these events. To achieve this aim, we used pattern classification of EEG data to derive predictive weights from carefully controlled pre-game exemplar events (oddball target detection and gambling wins and losses) and transferred those weights to EEG activities during video game events. All major goal-oriented events (crashes into the wall, crashes into an enemy, missile hit on an enemy) had a significant between-task transfer bias towards oddball target weights in the time range of the canonical P3, indicating the presence of similar salience detection processes. Missile hits on an enemy were specifically identified as gambling wins, confirming the hypothesis that this goal-oriented event was appetitive. These findings suggest that it is possible to identify the contribution of canonical neural activities during otherwise ambiguous and uncontrolled task performance.

A GOP-level fuzzy rate control algorithm for high-delay applications of HEVC

The high -efficiency video coding (HEVC) standard was introduced for high-resolution video contents suitable for many high-delay applications. The bit rate of compressed video is controlled in almost all digital video applications according to the practical constraints such as available channel bandwidth and allowed delay or buffering constraint. In high-delay video applications such as video broadcasting and video streaming applications, variable-bit-rate videos can provide a higher visual quality than constant-bit-rate videos. In this paper, a rate control algorithm (RCA) for high-delay applications of the HEVC standard with buffering constraint is proposed. A fuzzy controller and a virtual buffer are used in the proposed RCA. The fuzzy controller is designed to minimize the fluctuations of quantization parameter (QP) while the buffering constraint is obeyed. It computes a base QP for each group of pictures (GOP) to prevent unnecessary fluctuations of QP at GOP level and thereafter provide a higher visual quality. Experimental results show that not only the bit rate and but also buffering constraints are fully maintained but also the objective quality of compressed video is well preserved. Moreover, the proposed RCA provides smooth QP and peak signal-to-noise ratio close to constant QP case for encoded videos that means high subjective quality.