Video Rate Control Research Articles

Content-based coding tree unit level rate-quantization model for intra-coding in high efficiency video coding standard using convolutional neural network

In almost all video applications, a video rate control algorithm (RCA) is used by the encoder. The RCA tunes the quantization parameter (QP) to match the encoded bit rate to the available capacity of the communication channel or storage media. Conventional RCAs usually utilize a rate-quantization (R-Q) or a rate-distortion (R-D) model for rate control. A content-based R-Q model for intra coding tree units (CTUs) of the high-efficiency video coding standard is proposed. The model is a convolutional neural network that observes pixels of a CTU and its intraprediction reference pixels and it estimates required bit counts for intracoding the CTU for all QP values simultaneously. The proposed model can be easily used by any video RCA. A given RCA just selects a proper QP for which the estimated bit counts are closer to the allocated bit budget. The evaluation results show a high accuracy for the model. According to simulation results, the mean absolute normalized bit error at CTU level is 19.66% and it decreases to 6.85% at the frame level. Compared with similar networks, the proposed structure has a very low computational complexity.

Stochastic joint rate control and resource allocation for wireless video surveillance

Wireless video surveillance has become increasingly prevalent due to the rapid development of cellular communication technology and the low deployment cost of cellular surveillance cameras. However, it is still a challenging task to deliver high-quality low-latency surveillance videos in resource-limited cellular networks due to the fluctuating characteristics of wireless channels. In this paper, we consider a wireless video surveillance system consisting of one base station and multiple cellular surveillance cameras, which upload their videos to the base station. To optimize the long-term system performance in terms of surveillance video quality and latency, we formulate a stochastic optimization problem that jointly considers video rate control and uplink resource allocation. The formulation captures the tradeoff between video quality and delivery latency while considering the different time scales of video rate control and channel resource allocation. With the application of Lyapunov optimization, we transform the problem into a sequence of per-slot deterministic problems, and propose an online joint rate control and resource allocation algorithm to solve these per-slot problems. Specifically, the proposed algorithm decomposes each of the per-slot problems into two independent subproblems, namely, the rate control subproblem at large time scale and the uplink resource allocation subproblem at small time scale. We derive the closed-form solution for the rate control subproblem, and design a fast-convergent iterative algorithm to solve the uplink resource allocation subproblem optimally. We prove the asymptotical optimality of our proposed algorithm. Extensive simulation results validate our theoretical analysis and demonstrate that the proposed algorithm can well balance the surveillance video quality and delivery latency.

FJND-based fuzzy rate control of scalable video for streaming applications

A fuzzy rate controller with buffer constraint in combination with a perceptual quality controller is proposed for streaming applications of the AVC/H.264 scalable (SVC) video. The bit rate of each video layer is controlled separately by the fuzzy controller that adjusts the quantization parameter (QP) on a group of pictures (GOP) basis. The QPs of pictures are computed from the GOP QP by the well-known QP cascading technique. While the fuzzy controller provides the buffering constraint for each video layer, the quality controller tries to improve the perceptual quality of the compressed video based on the foveated just-noticeable distortion (FJND) model. The quality controller regulates the QP of each macroblock around the picture QP based on the visibility threshold of the FJND model. In these applications, the initial buffering allows slight variations of the bit rate leading to produce a variable bit rate (VBR) video bit stream with consistent quality. Experimental results show that the proposed algorithm effectively adapts to the buffer size, while strictly prevents buffer overflow and underflow. In addition, incorporating the perceptual quality controller into the fuzzy rate controller achieves higher perceptual quality at the same bit rate.

Fuzzy logic-based scalable video rate control algorithm for high-delay applications of scalable high-efficiency video coding

The second version of the high-efficiency video coding standard introduced a scalable extension, which is called scalable high-efficiency video coding. It is necessary to design appropriate rate controllers for this extension based on the specific application requirements. According to delay and buffering constraints, rate control algorithms are divided into constant bit rate and variable bit rate (VBR) algorithms. Our focus is on high-delay applications like streaming and broadcast for which VBR controllers are suitable. Our proposed VBR controller utilizes a fuzzy logic-based algorithm to control long-term average bit rate while preventing buffer overflow or underflow. Targeting perceptual quality, the proposed algorithm has been implemented at the group of pictures level to minimize unnecessary fluctuations of quantization parameter and consequently minimizing video quality fluctuations. Experimental results show a high performance in terms of rate control and quality of the compressed video for the proposed algorithm.

Region-based rate control for 3D-HEVC based texture video coding

Bit starvation resulted from inefficient rate control of 3D video coding deteriorates visual quality of synthesized views. Most region based rate control schemes allocate higher bitrate to regions of interest (ROIs), but bits may be still consumed by early coded units. This paper avoids reducing coding bits in high-cost regions of 3D-HEVC. Instead of ROIs that are determined by humans, high-cost regions sensitive to bit starvation are detected. Region-level bit allocation is achieved by curve fitting of coding statistics, and recursive Taylor expansion (RTE) based bit allocation is adopted to optimally estimates the target bitrate and the QP related λ of LCUs in each region. Based on retained earnings in economics, bits are reserved for LCUs that are sensitive to distortions. Experimental results show that the proposed scheme outperforms both the R-λ model and RTE model in bitrate accuracy with similar R-D performance.

Intelligent Video Surveillance Platform for Wireless Multimedia Sensor Networks

Wireless multimedia sensor networks (WMSNs) are increasingly being deployed for surveillance and monitoring applications. WMSNs applications produce large amount of data, which require high transmission rates. An efficient and seamless delivery of multimedia services in WMSNs is still a challenging task. This article proposes an intelligent video surveillance platform (IVSP) for wireless multimedia sensor networks. IVSP presents the design of a networked system for joint rate control and error control of video over resource-constrained embedded devices. First, a combination of two different congestion indicators is introduced to differentiate between congestion levels and handle them accordingly. Second, a feedback-based rate controller is developed to maximize received video quality, in which sensor nodes can adaptively adjust their sending rates. Finally, a different retransmission mechanism for different packets is proposed. Lost packets can be stored temporarily and resend when free channel is available to avoid congestion. The core component of IVSP is an open source hardware platform, which is based on Raspberry Pi sensor nodes. IVSP is extensively evaluated on 7 Raspberry Pi sensor nodes. We present the results of 7-node real-world deployment of IVSP in a video surveillance application and show that it works well in long-term deployments.

Video streaming schemes for industrial IoT

Wireless video applications for Industrial Internet of Things (IoT) are expanding into a multitude of new services. In the example of cloud processing for visual object detection, a camera is connected to the cloud via a local server and a data network, allowing the processing load to be handled in a distributed manner. This service model heavy taxes the data network with potentially unneeded traffic, thus degrading the overall quality of service for all users on the network. Edge computing techniques mitigate the degradation of service quality by partially processing the sensor data at the local server before the data is transmitted to the cloud. This is done according to the level of interest of the captured data which is categorized by machine learning algorithms. However, conventional edge computing is not optimally efficient as further recognition attributes of the captured object data are not considered. This paper presents a model that adds control of the camera video rate by considering the attributes of captured object. We then investigate cost trade-offs using dynamic programming, and evaluates the behavior of proposed method under wireless channel condition using NS-3 simulations. Our results show that by adding intelligent adaptive video rate control to the cloud processing of video data capture can reduce overall system power use while improving system efficiency and subsequently network throughput.

Rate control for non-uniform video in HEVC

Rate control plays an important role in video coding, and most of the existing rate control schemes are based on the assumption that the video contents are uniform. However, for non-uniform video applications, such as screen content videos and movies, previous rate control schemes may cause severe quality degradation, since the bit allocation strategy is improper and the rate-quantization (R-Q) model parameters may extremely inaccurate. In this paper, firstly an adaptive bit allocation algorithm is proposed for the newest video coding standard high efficiency video coding (HEVC), next a key frame detection method is presented, and finally an accurate intra R-Q model, a model parameter adjusting strategy and a bit feedback strategy are developed for the scene change frame. Experimental results show that the proposed rate control method could achieve more accurate bitrates and better rate-distortion (R-D) performance than previous rate control methods.

A Study on the Improvement of Classification and Abnormal Signal Detection Module for Improving an Object Identified in the Video-Rate Control

A Study on the Improvement of Classification and Abnormal Signal Detection Module for Improving an Object Identified in the Video-Rate Control

A Study on the Improvement of Classification and Abnormal Signal Detection Module for Improving an Object Identified in the Video-Rate Control·

A Study on the Improvement of Classification and Abnormal Signal Detection Module for Improving an Object Identified in the Video-Rate Control·

Towards Agile and Smooth Video Adaptation in HTTP Adaptive Streaming

HTTP Adaptive Streaming (HAS) is widely deployed on the Internet for live and on-demand video streaming services. Video adaptation algorithms in the existing HAS systems are either too sluggish to respond to congestion level shifts or too sensitive to short-term network bandwidth variations. Both degrade user video experience. In this paper, we formally study the tradeoff between responsiveness and smoothness in HAS through analysis and experiments. We show that client-side buffered video time is a good feedback signal to guide video adaptation. We then propose novel video rate control algorithms that balance the needs for video rate smoothness and high bandwidth utilization. We show that a small video rate margin can lead to much improved smoothness in video rate and buffer size. We also propose HAS designs that can work with multiple servers and wireless connections. We develop a fully functional HAS system and evaluate its performance through extensive experiments on a network testbed and the Internet. We demonstrate that our HAS designs are highly efficient and robust in realistic network environment.

Region-based bit allocation and rate control for depth video in HEVC

In this paper, we present a novel rate control method with optimized region-based bit allocation for depth video coding. First, a synthetic view distortion oriented segmentation method is proposed to divide depth video into different regions, including texture areas and smooth areas. Then, the expanded exponential distortion-rate (D-R) models and power quantization parameter-rate (QP-R) models are investigated to simulate rate-distortion (R-D) characteristics of different regions. Finally, an optimal bit allocation scheme is developed to adaptively allocate target bit with the division. Experimental results on various video sequences demonstrate that the proposed algorithm achieves excellent R-D efficiency and bit rate accuracy compared with benchmark algorithms.

A Low-Complexity Embedded Compression Codec Design With Rate Control for High-Definition Video

A hardwired design of embedded compression engine targeting the reduction of full high-definition (HD) video transmission bandwidth over the wireless network is developed. It adopts an intra-coding framework and supports both lossless and rate-controlled near lossless compression options. The lossless compression algorithm is based on a simplified Context-Based, Adaptive, Lossless Image Coding (CALIC) scheme featuring pixelwise gradient-adjusted prediction and error-feedback mechanism. To reduce the implementation complexity, an adaptive Golomb-Rice coding scheme in conjunction with a context modeling technique is used in lieu of an adaptive arithmetic coder. With the measures of prediction adjustment, the near lossless compression option can be implemented on top of the lossless compression engine with minimized overhead. An efficient bit-rate control scheme is also developed and can support rate or distortion-constrained controls. For full HD (previously encoded) and nonfull HD test sequences, the lossless compression ratio of the proposed scheme, on average, is 21% and 46%, respectively, better than the Joint Photographic Experts Group-Lossless Standard and the Fast, Efficient Lossless Image Compression System (FELICS) schemes. The near lossless compression option can offer additional 6%–20% bit-rate reduction while keeping the Peak Signal-to-Noise Ratio value 50 dB or higher. The codec is further optimized complexity-wise to facilitate a high-throughput chip implementation. It features a five-stage pipelined architecture and two parallel computing kernels to enhance the throughput. Fabricated using the Taiwan semiconductor manufacturing company 90-nm complementary metal-oxide–semiconductor technology, the design can operate at 200 MHz and supports a 64 frames/s processing rate for full HD videos.

A direct non-buffer rate control algorithm for real time video compression

Rate control (RC) is crucial in controlling compression bitrates and encoding qualities for networked video applications. In this paper, we propose a direct non-buffer real-time rate control algorithm for video encoding, which has two unique features. First, unlike traditional algorithms which adopt buffers in rate control, the proposed algorithm does not use a buffer in rate regulation which can reduce the delay and improve real-time response. Second, we propose a new Proportional-Integral-Derivative (PID) bit controller to directly control encoding bitrates. In addition, we also develop a simple but effective method for real-time target bit allocation. To the best of our knowledge, this is the first work that conducts video rate control without using a buffer. Our extensive experimental results have demonstrated that the proposed algorithm outperforms the MPEG-4 rate control algorithm by achieving more accurate rate regulation and improving overall coding quality.

Modeling and Analysis of Skype Video Calls: Rate Control and Video Quality

Video-conferencing has recently gained its momentum and is widely adopted by end-consumers. But there have been very few studies on the network impacts of video calls and the user Quality-of-Experience (QoE) under different network conditions. In this paper, we study the rate control and video quality of Skype video call, and analyze the network impacts in large-scale networks. We first measure the behaviors of Skype video call on a controlled network testbed. By varying packet loss rate, propagation delay and available network bandwidth, we observe how Skype adjusts its sending rate, FEC redundancy, video rate and frame rate. It is found that Skype is robust against mild packet losses and propagation delays, and can efficiently utilize the available network bandwidth. We also find that it employs an overly aggressive FEC protection strategy. Based on the measurement results, we develop rate control model, FEC model, and video quality model for Skype video calls. Extrapolating from the models, we conduct numerical analysis to study the network impacts. We demonstrate that user back-offs upon quality degradation serve as an effective user-level rate control scheme. We also show that Skype video calls are indeed TCP-friendly and respond to congestion quickly when the network is overloaded. Through a case study of a 4G wireless network, we demonstrate that the proposed models can be used in user-QoE-aware network provisioning.

The Use of Automotive Radars in Video-Based Overtaking Assistance Applications

Overtaking on rural roads may cause severe accidents when oncoming traffic is detected by a driver too late, or its speed is underestimated. Recently proposed cooperative overtaking assistance systems are based on real-time video transmission, where a video stream captured with a camera installed at the windshield of a vehicle is compressed, broadcast through the wireless channel, and displayed to the drivers of vehicles driving behind. In such a system, it is of ultimate importance to deliver video information about the opposite lane with low end-to-end latency and good visual quality. In this paper, we propose reallocating the wireless channel resources in favor of the part of the captured video frame containing the image of the oncoming vehicle. To achieve this goal, we apply automotive radar for oncoming vehicle detection, and we use the image of this vehicle as a region-of-interest (ROI) for the video rate control. We present the theoretical framework, which describes the basics of such an approach and can serve as a useful guideline for the future practical implementation of the overtaking assistance systems. The benefits of our proposal are demonstrated in relation to the practical scenario of H.264/Advance Video Coding (AVC), IEEE 802.11p/Wireless Access for Vehicular Environments (WAVE) intervehicle communication standards, and currently used automotive radars.

실시간 영상 부호화를 위한 H.264/AVC의 비트율 제어 하드웨어 설계

In this paper, the hardware design of rate control for real-time video encoded is proposed. In the proposed method, a quadratic rate distortion model with high-computational complexity is not used when quantization parameter values are being decided. Instead, for low-computational complexity, average complexity weight values of frames are used to calculate QP. For high speed and low computational prediction, the MAD is predicted based on the coded basic unit, using spacial and temporal correlation in sequences. The rate control is designed with the hardware for fast QP decision. In the proposed method, a quadratic rate distortion model with high-computational complexity is not used when quantization parameter values are being decided. Instead, for low-computational complexity, average complexity weight values of frames are used to calculate QP. In addition, the rate control is designed with the hardware for fast QP decision. The execution cycle and gate count of the proposed architecture were reduced about 65% and 85% respectively compared with those of previous architecture. The proposed RC was implemented using Verilog HDL and synthesized with UMC standard cell library. The synthesis result shows that the gate count of the architecture is about 19.1k with 108MHz clock frequency.

Single-Pass Rate Control With Texture and Non-Texture Rate-Distortion Models

One of the challenges in video rate control lies in determining a quantization parameter (Qp) that will be used for both the rate-distortion (R-D) optimization process and the quantization of transform coefficients. In this paper, we attempt to achieve effective rate control with a different approach. By modeling the relationships of distortion, texture bits, non-texture bits, and Qp, we derive the Qp required for both R-D optimization and quantization through Lagrangian optimization. From experiments with several video sequences, we found that our rate control scheme is capable of effective rate control with only a few model updates during encoding. The proposed rate control scheme adapts quickly to the characteristics of the source data and is particularly effective at controlling the rate of videos with high and unpredictable motion content.

Compressed-Sensing-Enabled Video Streaming for Wireless Multimedia Sensor Networks

This paper presents the design of a networked system for joint compression, rate control and error correction of video over resource-constrained embedded devices based on the theory of Compressed Sensing (CS). The objective of this work is to design a cross-layer system that jointly controls the video encoding rate, the transmission rate, and the channel coding rate to maximize the received video quality. First, compressed sensing-based video encoding for transmission over Wireless Multimedia Sensor Networks (WMSNs) is studied. It is shown that compressed sensing can overcome many of the current problems of video over WMSNs, primarily encoder complexity and low resiliency to channel errors. A rate controller is then developed with the objective of maintaining fairness among different videos while maximizing the received video quality. It is shown that the rate of Compressed Sensed Video (CSV) can be predictably controlled by varying only the compressed sensing sampling rate. It is then shown that the developed rate controller can be interpreted as the iterative solution to a convex optimization problem representing the optimization of the rate allocation across the network. The error resiliency properties of compressed sensed images and videos are then studied, and an optimal error detection and correction scheme is presented for video transmission over lossy channels. Finally, the entire system is evaluated through simulation and test bed evaluation. The rate controller is shown to outperform existing TCP-friendly rate control schemes in terms of both fairness and received video quality. The test bed results show that the rates converge to stable values in real channels.

Modeling of Rate and Perceptual Quality of Compressed Video as Functions of Frame Rate and Quantization Stepsize and Its Applications

This paper first investigates the impact of frame rate and quantization on the bit rate and perceptual quality of compressed video. We propose a rate model and a quality model, both in terms of the quantization stepsize and frame rate. Both models are expressed as the product of separate functions of quantization stepsize and frame rate. The proposed models are analytically tractable, each requiring only a few content-dependent parameters. The rate model is validated over videos coded using both scalable and nonscalable encoders, under a variety of encoder settings. The quality model is validated only for a scalable video, although it is expected to be applicable to a single-layer video as well. We further investigate how to predict the model parameters using the content features extracted from original videos. Results show accurate bit rate and quality prediction (average Pearson correlation >;0.99) can be achieved with model parameters predicted using three features. Finally, we apply rate and quality models for rate-constrained scalable bitstream adaptation and frame rate adaptive rate control. Simulations show that our model-based solutions produce better video quality compared with conventional video adaptation and rate control.