Bit Rate Control Research Articles

Quality and Transferred Data Based Video Bitrate Control Method for Web-Conferencing

Quality and Transferred Data Based Video Bitrate Control Method for Web-Conferencing

Long-term adaptive bitrate control mechanism

Long-term adaptive bitrate control mechanism

Adaptive Channel Scheduling for Acceleration and Fine Control of RNN-Based Image Compression

The existing target-dependent scalable image compression network can control the target of the compressed images between the human visual system and the deep learning based classification task. However, in its RNN based structure controls the bit-rate through the number of iterations, where each iteration generates a fixed size of the bit stream. Therefore, a large number of iterations are required at the high BPP, and fine-grained image quality control is not supported at the low BPP. In this paper, we propose a novel RNN-based image compression model that can schedule the channel size per iteration, to reduce the number of iterations at the high BPP and fine-grained bit-rate control at the low BPP. To further enhance the efficiency, multiple network models for various channel sizes are combined into a single model using the slimmable network architecture. The experimental results show that the proposed method achieves comparable performance to the existing method with finer BPP adjustment, increases parameters by only 0.15% and reduces the average amount of computation by 40.4%.

Intelligent Video Ingestion for Real-time Traffic Monitoring

As an indispensable part of modern critical infrastructures, cameras deployed at strategic places and prime junctions in an intelligent transportation system can help operators in observing traffic flow, identifying any emergency situation, or making decisions regarding road congestion without arriving on the scene. However, these cameras are usually equipped with heterogeneous and turbulent networks, making the real-time smooth playback of traffic monitoring videos with high quality a grand challenge. In this article, we propose a lightweight Deep Reinforcement Learning-based approach, namely, sRC-C (smart bitRate Control with a Continuous action space) , to enhance the quality of real-time traffic monitoring by adjusting the video bitrate adaptively. Distinguished from the existing bitrate adjusting approaches, sRC-C can overcome the bias incurred by deterministic discretization of candidate bitrates by adjusting the video bitrate with more fine-grained control from a continuous action space, thus significantly improving the Quality-of-Service (QoS). With carefully designed state space and neural network model, sRC-C can be implemented on cameras with scarce resources to support real-time live video streaming with low inference time. Extensive experiments show that sRC-C can reduce the frame loss counts and hold time by 24% and 15.5%, respectively, even with comparable bandwidth utilization. Meanwhile, compared to the-state-of-art approaches, sRC-C can improve the QoS by 30.4%.

Various transmission codes for the control of bit error rate in both optical wired and wireless communication channels

Abstract This study clarifies the data error rates optimization for OFC/OWC channels based on different transmission codes. These codes that are namely multi bits/symbol digital pulse interval modulation (DPIM), multi bits/symbol pulse position modulation (PPM), nonreturn to zero inverted (NRZI), 4 bit data symbol/5 bit code (4B5B), and Manchester for upgrading optical wired/wireless communication systems. The optical power through OFC/OWC channels, S/N ratio, the output power at the receiver side are stimulated with high bit transmission rates. The effects of coding complexity on the Q-factor, BER, optical power, and electrical received power are also stimulated using both DPIM and PPM coding.

MultiLive: Adaptive Bitrate Control for Low-Delay Multi-Party Interactive Live Streaming

MultiLive: Adaptive Bitrate Control for Low-Delay Multi-Party Interactive Live Streaming

Quality-based Video Bitrate Control for WebRTC-based Teleconference Services

In this article, we propose a quality-based video bitrate control method for web real-time communication (WebRTC)-based teleconferences. Video bitrate is controlled on the basis of quality of service (QoS) parameters such as delay and packet-loss rate in WebRTC. Therefore, the amount of transferred data may increase because media streams are transmitted at excessive quality levels depending on QoS conditions (e.g., the jitter and packet-loss rate are low). An increase in transferred data leads to higher operational cost (i.e., data transferred cost) and affects profitable growth. In the proposed method, quality desired by a service provider is set as TargetQuality, and the video bitrate of each stream is controlled aiming at TargetQuality, thereby suppressing the amount of transferred data while maintaining sufficient quality. The proposed method is implemented to an actual teleconference system and is evaluated in terms of its effect at reducing the amount of transferred data. The results show that the amount of transferred data can be reduced by more than 40% by setting the value of TargetQuality appropriately.

Scalable and Efficient Neural Speech Coding: A Hybrid Design

We present a scalable and efficient neural waveform coding system for speech compression. We formulate the speech coding problem as an autoencoding task, where a convolutional neural network (CNN) performs encoding and decoding as a neural waveform codec (NWC) during its feedforward routine. The proposed NWC also defines quantization and entropy coding as a trainable module, so the coding artifacts and bitrate control are handled during the optimization process. We achieve efficiency by introducing compact model components to NWC, such as gated residual networks and depthwise separable convolution. Furthermore, the proposed models are with a scalable architecture, cross-module residual learning (CMRL), to cover a wide range of bitrates. To this end, we employ the residual coding concept to concatenate multiple NWC autoencoding modules, where each NWC module performs residual coding to restore any reconstruction loss that its preceding modules have created. CMRL can scale down to cover lower bitrates as well, for which it employs linear predictive coding (LPC) module as its first autoencoder. The hybrid design integrates LPC and NWC by redefining LPC's quantization as a differentiable process, making the system training an end-to-end manner. The decoder of proposed system is with either one NWC (0.12 million parameters) in low to medium bitrate ranges (12 to 20 kbps) or two NWCs in the high bitrate (32 kbps). Although the decoding complexity is not yet as low as that of conventional speech codecs, it is significantly reduced from that of other neural speech coders, such as a WaveNet-based vocoder. For wide-band speech coding quality, our system yields comparable or superior performance to AMR-WB and Opus on TIMIT test utterances at low and medium bitrates. The proposed system can scale up to higher bitrates to achieve near transparent performance.

Training Deep Q-Network via Monte Carlo Tree Search for Adaptive Bitrate Control in Video Delivery

Training Deep Q-Network via Monte Carlo Tree Search for Adaptive Bitrate Control in Video Delivery

A neural network-based video bit-rate control algorithm for variable bit-rate applications of versatile video coding standard

Versatile video coding is a new video coding standard that has more capabilities and higher coding efficiency compared with its predecessor. Practical video storage and transmission applications face constrained buffer size and available bandwidth. It is necessary to design the appropriate rate control algorithm to overcome such challenges. In this paper, the non-linear relationship between consumed bits, buffer size, and quantization parameter is estimated by taking the advantages of artificial neural networks, and a rate control algorithm is developed for real-time variable bit rate applications of the versatile video coding standard. The proposed rate control algorithm performs the control action in only one step that results in faster control action. The experimental results show that the proposed algorithm controls the bit-rate as well as the buffer state. Also, the rate–distortion analysis shows that the well-known λ-domain algorithm has only 2.7% bit-rate reduction in comparison with the proposed method.

A Real-Time Low-Power Coding Bit-Rate Control Scheme for High-Efficiency Video Coding in a Multiprocessor System-on-Chip

A real-time high-performance transmission bandwidth-aware (TB-aware) coding bit-rate (CBR) controller design with low power consumption and low hardware complexity is presented in this article for H.265/high-efficiency video coding (HEVC) in a multiprocessor system-on-chip (MPSoC). Previous TB-aware motion estimation designs with CBR-control capability in video coding have focused on algorithm development with precise CBR models, which require a complicated algorithmic derivation according to the system on-demand CBR and are difficult to realize in very large scale integration (VLSI) due to their lack of consideration for hardware implementation and modeling. Consequently, we present a hardware-oriented CBR-control algorithm that uses simple CBR control functions instead of requiring root and exponential operations to realize the real-time low-power design objective for HEVC applications within a mobile MPSoC. Then, an adequate hardware architecture with low hardware complexity is exploited to accomplish a low-power and high-speed VLSI design of a CBR controller for our proposed algorithm. Using diverse video-sized sequences under on-demand system coding-bit-rate constraints, the experimental outcomes demonstrate that the introduced design is capable of low power consumption and high speeds and can utilize low-complexity hardware.

Inter-layer correlation-based adaptive bit allocation for enhancement layer in scalable high efficiency video coding

To facilitate the diversity of network and end-user devices, bit allocation technology must be combined with scalable high efficiency video coding (SHVC) to achieve continuous bitrate variation. However, to date, the multilayer coding feature in SHVC has not been fully utilized. In this paper, a bit allocation algorithm is proposed for the enhancement layer in SHVC. The algorithm includes two parts: a bit allocation method for the initial frame and another for subsequent frames. The bit allocation method for the initial frame consists of a bit allocation factor model constructed based on investigation of the influence of the initial target bitrate on the overall coding performance. For the bit allocation of the subsequent frames, alternate GOP coding is designed and implemented to fully exploit the inter layer correlation. In addition, an adaptive frame layer bit allocation ratio model is deduced according to the rate distortion optimization theory. Experimental results show that the proposed algorithm can improve PSNR by 0.44 dB and 0.41 dB under low delay and random-access configurations, respectively, and achieve high bitrate control accuracy. The algorithm also outperforms the state-of-the-art algorithms in term of PSNR improvement.

Lossy Hyperspectral Image Compression on a Reconfigurable and Fault-Tolerant FPGA-Based Adaptive Computing Platform

This paper describes a novel hardware implementation of a lossy multispectral and hyperspectral image compressor for on-board operation in space missions. The compression algorithm is a lossy extension of the Consultative Committee for Space Data Systems (CCSDS) 123.0-B-1 lossless standard that includes a bit-rate control stage, which in turn manages the losses the compressor may introduce to achieve higher compression ratios without compromising the recovered image quality. The algorithm has been implemented using High-Level Synthesis (HLS) techniques to increase design productivity by raising the abstraction level. The proposed lossy compression solution is deployed onto ARTICo3, a dynamically reconfigurable multi-accelerator architecture, obtaining a run-time adaptive solution that enables user-selectable performance (i.e., load more hardware accelerators to transparently increase throughput), power consumption, and fault tolerance (i.e., group hardware accelerators to transparently enable hardware redundancy). The whole compression solution is tested on a Xilinx Zynq UltraScale+ Field-Programmable Gate Array (FPGA)-based MPSoC using different input images, from multispectral to ultraspectral. For images acquired by the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS), the proposed implementation renders an execution time of approximately 36 s when 8 accelerators are compressing concurrently at 100 MHz, which in turn uses around 20% of the LUTs and 17% of the dedicated memory blocks available in the target device. In this scenario, a speedup of 15.6× is obtained in comparison with a pure software version of the algorithm running in an ARM Cortex-A53 processor.

A frame-level MLP-based bit-rate controller for real-time video transmission using VVC standard

Real-time video transmission is one of the most popular applications that are included in the versatile video coding (VVC) standard. However, real-time applications are encountered with practical limitations, including the buffer size and available bandwidth. In these applications, the buffer overflow and underflow should be strictly prevented and also the bit-rate fluctuation should be suppressed. In this paper, a video bit-rate controller is proposed that completely conforms with the constraints of real-time applications. The proposed controller is based on a multi-layer perceptron (MLP) neural network which estimates the proper quantization parameter (QP) modification at the frame level. The buffer occupancy is directly included in the QP derivation process for robust buffer control. Experimental results show that the proposed bit-rate controller fulfils the buffering constraints and controls the bit-rate accurately. The average bit-rate error of the proposed method is 0.29% while providing a low initial buffering delay of about 0.21 s. Also, the rate-distortion analysis shows that the performance of the proposed method is close to those of the conventional algorithms.

Rate Control for Video-based Point Cloud Compression.

Rate control is a necessary tool for video-based point cloud compression (V-PCC). However, there is no solution specified on this topic yet. In this paper, we propose the first rate control algorithm for V-PCC. Generally, a rate control algorithm is divided into two processes: bit allocation and bitrate control. In V-PCC, the total bits are composed of three parts: the header information including the auxiliary information and occupancy map, the geometry video, and the attribute video. The bit allocation aims to assign the total bits to these three parts. Since the auxiliary information and occupancy map are encoded losslessly, the bit cost of the header information is fixed. Therefore, we only need to assign bits between the geometry and attribute videos. Our first key contribution is the proposed video-level bit allocation algorithm between the geometry and attribute videos to optimize the overall reconstructed point cloud quality. Then we assign geometry and attribute video bits to each group of pictures (GOP), each frame, and each basic unit (BU). Our second key contribution is that we assign zero bits to the BUs with only unoccupied pixels. The unoccupied pixels are useless for the reconstructed quality of the point cloud and therefore should be assigned zero bits. In the bitrate control process, the encoding parameters are determined, and the model parameters are updated for each frame and BU to achieve the target bits. Our third key contribution is that we propose a BU-level model updating scheme to handle the case where various patches may be placed in different positions in neighboring frames. We use the auxiliary information to find the corresponding BU in the previous frame and apply its model parameters to the current BU. The proposed algorithms are implemented in the V-PCC and High Efficiency Video Coding (HEVC) reference software. The experimental results show that the proposed rate control algorithm can achieve significant bitrate savings compared with the state-of-the-art method.

A Frame-Level Constant Bit-Rate Control Using Recursive Bayesian Estimation for Versatile Video Coding

In this paper, we present a frame-level constant bit-rate (CBR) control method using recursive Bayesian estimation (RBE) for Versatile Video Coding (VVC). An R - $\lambda $ model for rate control (RC) has handled the total texture and non-texture bits at a time and has worked reasonably well in High Efficiency Video Coding (HEVC). Nevertheless, if the rate estimation is inaccurately performed, that is, the $R$ and $\lambda $ values for a current frame cannot be linearly modeled with their respective values in the previous frames, the resulting RC performance is degraded. In our work, we adopt the RBE which alternates prediction and update steps not only to precisely estimate the rates, but also to allocate target bits based on the changes in the distortions of the previously coded frames, thus considering the rates and distortions simultaneously. Therefore, an elaborate RC can be performed especially at fluctuating frame complexities. Experimental results show that our RC method outperforms the RC of VVC Test Model (VTM-5.0) in terms of normalized root mean square error (NRMSE) with maximum (average) 34.95% (12.35%) improvement, and maintains higher visual quality consistency in terms of standard deviation of PSNR by 33.07% (22.34%) improvement for All Intra (AI), maximum (average) 44.82% (27.29%) and 22.54% (9.50%) for Low Delay (LD), and maximum (average) 47.35% (39.94%) and 30.35% (18.54%) for Random Access (RA), respectively, compared to the default RC method of the original VTM-5.0.

Mathematical Model of a Network Slicing Approach for Video and Web Traffic

Abstract—This paper presents the study of adaptive bitrate control algorithms used to control the quality of the video stream from video surveillance cameras to remote servers through a live wireless network with video and web traffic. In such a scenario, bitrate control algorithms may select an appropriate video bitrate taking the current state of the camera buffers into account. A mathematical model of video streaming is introduced that allows evaluation of the performance of different bitrate control algorithms.

Optimal Bit Allocation at Frame Level for Rate Control in HEVC

Rate control plays an important role in video coding systems, which makes the output bitrate of a video encoder equal to the target bitrate while minimizing the distortion of the compressed video. However, most of the existing rate control schemes achieve accurate bitrate control at the loss of rate-distortion (R-D) performance. This paper proposes an effective frame level bit allocation method to improve the R-D performance whilst maintaining the high accuracy of the bitrate control in HEVC. First, an improved R-D model is presented at the frame level, which, by making use of the information of encoded frames more completely, achieves higher accuracy with lower bitrate mismatch rate. Second, a group of picture (GOP) level bit allocation approach is introduced with the consideration of the temporal R-D dependency among different GOPs, which can further enhance the R-D performance. Finally, to achieve optimal R-D performance globally, a formulation for optimal frame level bit allocation is developed with a GOP level Lagrange multiplier introduced, which takes into account the coding effects of a current frame on the other frames within the same GOP. A scheme of recursive Taylor expansion is employed to find the GOP level Lagrange multiplier. Experimental results demonstrate that the proposed method can achieve considerable R-D performance improvement and accurate bitrate control. Specifically, our method shows 3.9% and 3.8% BD-rate saving in average compared against the HEVC reference software HM16.7 with rate control in the low-delay B and low-delay P coding structures, respectively.

Evaluation of Throughput Prediction for Adaptive Bitrate Control Using Trace-Based Emulation

Dynamic adaptive video streaming over HTTP (DASH) is widely studied and has been adopted in modern video players to ensure user quality of experience (QoE). In DASH, adaptive bitrate control is a key part whose ultimate goal is to maximize video bitrate while minimizing rebuffering. Throughput prediction plays an important role in helping select the proper video bitrate dynamically. In this paper, we studied the influence of throughput prediction on adaptive video streaming. Because the real-world network is dynamic, different methods need to be tested with large-scale deployments and analyzed statistically. However, this is difficult in academic research. Therefore, we established a reproducible trace-based emulation environment, which enables us to compare different methods quantitatively under the artificially same condition, with limited experiments. The throughput prediction methods are implemented into DASH to evaluate the effect on QoE for video streaming. The results indicate that the prediction method using long short-term memory (LSTM) performs better than the other methods. However, throughput prediction alone is not enough to ensure high QoE. To further improve the QoE, we proposed the decision map method (DMM), where the buffer occupancy is also incorporated to make a selection. By using this decision map, the choice of bitrate can be smarter than that when only prediction information is used. The total QoE is further improved by 32.1% in the ferry trace, which shows the effectiveness of DMM in further improving the performance of throughput prediction in adaptive bitrate control.

A 120 fps High Frame Rate Real-time HEVC Video Encoder with Parallel Configuration Scalable to 4K

This paper describes a new 120 fps (frames per second) real-time HEVC (High Efficiency Video Coding) encoder for HFR (high frame rate) video encoding and transmission. HFR provides more immersive viewing experience features by solving the problems created by fast moving scenes. Temporally scalable encoding with backward compatibility for legacy non-HFR systems is suitable for the rapid spread of HFR content delivery, avoiding the need to distribute multiple bitstreams of the same video with different frame rates. Such temporal scalability requires flexible encoder control functionalities to support newly-customized reference picture structures and dual-stream bitrate control. In this paper, modification in the customizable software architecture of encoder LSIs makes it possible to achieve 120 fps temporally scalable HEVC encoding for existing 60 fps-based systems. The encoder also achieves ${4\mathrm{K}/ 120\;\mathrm{fps}}$ video encoding in real time through the synchronized operation of multiple ${2\mathrm{K}/ 120\;\mathrm{fps}}$ encoders working in parallel. Our evaluations show that the bitrate increase rate from 60 fps to 120 fps under the same objective image quality condition are within the range of less than 57.2 percent in all video sequences and its average value is 53.8 percent. Both values are lower than that of the HM (HEVC reference software). The proposed encoder systems will open the door to the next generation high frame rate UHDTV (ultra high definition television) services.