Networked Video Applications Research Articles

Deep Hierarchical Video Compression

Recently, probabilistic predictive coding that directly models the conditional distribution of latent features across successive frames for temporal redundancy removal has yielded promising results. Existing methods using a single-scale Variational AutoEncoder (VAE) must devise complex networks for conditional probability estimation in latent space, neglecting multiscale characteristics of video frames. Instead, this work proposes hierarchical probabilistic predictive coding, for which hierarchal VAEs are carefully designed to characterize multiscale latent features as a family of flexible priors and posteriors to predict the probabilities of future frames. Under such a hierarchical structure, lightweight networks are sufficient for prediction. The proposed method outperforms representative learned video compression models on common testing videos and demonstrates computational friendliness with much less memory footprint and faster encoding/decoding. Extensive experiments on adaptation to temporal patterns also indicate the better generalization of our hierarchical predictive mechanism. Furthermore, our solution is the first to enable progressive decoding that is favored in networked video applications with packet loss.

Delay-Sensitive Video Computing in the Cloud

While cloud servers provide a tremendous amount of resources for networked video applications, most successful stories of cloud-assisted video applications are presentational video services, such as YouTube and NetFlix. This article surveys the recent advances on delay-sensitive video computations in the cloud, which are crucial to cloud-assisted conversational video services, such as cloud gaming, Virtual Reality (VR), Augmented Reality (AR), and telepresence. Supporting conversational video services with cloud resources is challenging because most cloud servers are far away from the end users while these services incur the following stringent requirements: high bandwidth, short delay, and high heterogeneity. In this article, we cover the literature with a top-down approach: from applications and experience, to architecture and management, and to optimization in and outside of the cloud. We also point out major open challenges, hoping to stimulate more research activities in this emerging and exciting direction.

A direct non-buffer rate control algorithm for real time video compression

Rate control (RC) is crucial in controlling compression bitrates and encoding qualities for networked video applications. In this paper, we propose a direct non-buffer real-time rate control algorithm for video encoding, which has two unique features. First, unlike traditional algorithms which adopt buffers in rate control, the proposed algorithm does not use a buffer in rate regulation which can reduce the delay and improve real-time response. Second, we propose a new Proportional-Integral-Derivative (PID) bit controller to directly control encoding bitrates. In addition, we also develop a simple but effective method for real-time target bit allocation. To the best of our knowledge, this is the first work that conducts video rate control without using a buffer. Our extensive experimental results have demonstrated that the proposed algorithm outperforms the MPEG-4 rate control algorithm by achieving more accurate rate regulation and improving overall coding quality.

A novel total variation based frame layer rate control algorithm for H.264/AVC

Rate control (RC) plays a crucial role in controlling compression bitrates and encoding qualities for networked video applications. In this research, we propose a new total variation (TV) based frame layer rate control algorithm for H.264/AVC. One of its novelties is that a total variation measure, used in image processing field, is proposed to describe encoding distortion in video compression. For intraframes, we present a TV distortion–quantization (DTV–Qstep) model to obtain accurate QP step size (Qstep). Using TV measure to represent frame complexity, we also present an analytic model to calculate Qstep for the initial frame, and develop an effective scene change detection method. In addition, an incomplete derivative proportional integral derivative (IDPID) buffer controller is proposed to reduce the deviation between the current buffer fullness and the target buffer fullness, and minimizes the buffer overflow or underflow. Extensive experimental results show that, compared with JVT-W042, the proposed algorithm successfully achieves more accurate target bit rates, reduces frame skipping, decreases quality fluctuation and improves the overall coding quality.

Quality of Experience Evaluation for Networked Video Services

Rapid development of network technologies has led to a proliferation of networked video applications, making efficient and effective assessment of the related Quality of (user) Experience (QoE) an acute need. In literature, there has been a non-uniform view of Quality of Service (QoS) and therefore the concept of QoE is introduced to avoid the confusion. The difference between QoS and QoE is first clarified in this paper. Then QoE metrics in the current standardized framework for Internet Protocol (IP) based streaming applications in 3G networks, i.e., 3GPP PSS, are introduced. Moreover, objective QoE evaluation for networked video is addressed in respect to perceived video quality, which can be categorized into the parametric-planning model, parametric packet-layer model, parametric bit-stream-layer model, media-layer model, and hybrid model. The related characteristics, state-of-the-art, standardization process and key patents are respectively reviewed.

A No-Reference Quality Assessment System for Video Streaming over RTP

In the paper a no-reference system for quality assessment of video streaming over RTP is proposed for monitoring the quality of networked video. The proposed system is composed of four modules, where the quality assessment module utilizes information extracted from the bit-stream by the modules of RTP header analysis, frame header analysis and display buffer simulation. Taking MPEG-4 encoded video stream over RTP as an example, the process of video quality assessment using the proposed system is described in this paper. The proposed system is featured by its high efficiency without sorting to the original video or video decoding, and therefore well suited for real-time networked video applications.

New rate-distortion modeling and efficient rate control for H.264/AVC video coding

Rate control (RC) is crucial in controlling compression bit rates and qualities for networked video applications. In this paper, we propose a new rate-distortion (R-D) model and an efficient rate control scheme for H.264/AVC video coding, which elegantly resolve the inter-dependency problem between rate-distortion optimization and rate control by eliminating the need of coding complexity prediction for an inter-frame. The objective is to achieve accurate bit rate, obtain optimal video quality while reducing quality variations and simultaneously handling buffer fullness effectively. The proposed algorithm encapsulates a number of new features, including a coding complexity measure for intra-frames, a rate-distortion model, an accurate quantization parameter (QP) estimation for intra-frames, an incremental quantization parameter calculation method for inter-frames, a proportional+integral+derivative (PID) buffer controller, and an intelligent bit-allocation-balancing technique. Our experimental results demonstrate that the proposed scheme outperforms the JVT-G012 solution by providing accurate rate regulation, effectively reducing frame skipping, and finally improving coding quality by up to 1.80 dB.

An adaptive media playout for intra-media synchronization of networked-video applications

Adaptive media playout (AMP) aims to improve the media synchronization quality of streaming applications by regulating the playout time interval among media units (MUs) at a receiver. In this paper, we introduce an AMP scheme based on a discontinuity model for intra-media synchronization of video applications over the best-effort networks. We analyze the temporal distortion (i.e., discontinuity) cases, such as playout pause and skip, to define a unified discontinuity model. Based on the discontinuity model, an objective intra-media synchronization quality metric is defined as the root mean square error (RMSE) of discontinuity. After that, we establish a practical algorithm that enables an efficient playout speed control to enhance the intra-media synchronization quality. Simulation results verify the enhanced intra-media synchronization quality through comparison with a conventional buffer-threshold-based scheme.

Digital Video Transcoding

Video transcoding, due to its high practical values for a wide range of networked video applications, has become an active research topic. We outline the technical issues and research results related to video transcoding. We also discuss techniques for reducing the complexity, and techniques for improving the video quality, by exploiting the information extracted from the input video bit stream.