Bitrate Savings Research Articles

Overview of the Screen Content Support in VVC: Applications, Coding Tools, and Performance

In an increasingly connected world, consumer video experiences have diversified away from traditional broadcast video into new applications with increased use of non-camera-captured content such as computer screen desktop recordings or animations created by computer rendering, collectively referred to as screen content. There has also been increased use of graphics and character content that is rendered and mixed or overlaid together with camera-generated content. The emerging Versatile Video Coding (VVC) standard, in its first version, addresses this market change by the specification of low-level coding tools suitable for screen content. This is in contrast to its predecessor, the High Efficiency Video Coding (HEVC) standard, where highly efficient screen content support is only available in extension profiles of its version 4. This paper describes the screen content support and the five main low-level screen content coding tools in VVC: transform skip residual coding (TSRC), block-based differential pulse-code modulation (BDPCM), intra block copy (IBC), adaptive color transform (ACT), and the palette mode. The specification of these coding tools in the first version of VVC enables the VVC reference software implementation (VTM) to achieve average bit-rate savings of about 41% to 61% relative to the HEVC test model (HM) reference software implementation using the Main 10 profile for 4:2:0 screen content test sequences. Compared to the HM using the Screen-Extended Main 10 profile and the same 4:2:0 test sequences, the VTM provides about 19% to 25% bit-rate savings. The same comparison with 4:4:4 test sequences revealed bit-rate savings of about 13% to 27% for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Y'C_{B}C_{R}$ </tex-math></inline-formula> and of about 6% to 14% for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$R'G'B'$ </tex-math></inline-formula> screen content. Relative to the HM without the HEVC version 4 screen content coding extensions, the bit-rate savings for 4:4:4 test sequences are about 33% to 64% for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Y'C_{B}C_{R}$ </tex-math></inline-formula> and 43% to 66% for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$R'G'B'$ </tex-math></inline-formula> screen content.

The Optimal Real-Time Video Encoding Scheme for Multi-Camera Panoramic System Using NVENC

Today, modern, powerful, and relatively available computer technology allows the proposal of real-time camera systems working with several video streams. There is also a permanent tendency to increase image resolution, what can be useful for example in some image processing methods used in surveillance systems. Given this, an important part of a multi-camera real-time system proposal is optimal and efficient video compression. The modern encoding standards support 4K or 8K video content. One of these standards is H.264 or HEVC (High Efficiency Video Coding) and the popular available encoding technology using GPU (Graphics Processing Unit) acceleration is NVENC (Nvidia Encoding) provided by Nvidia. Many papers deal with the comparison of encoding standards in terms of quality or bitrate savings on testing video sequences [1], [2]. However, this paper is aimed primarily at a comparison of several practical implementations of NVENC technology using different presets of H.264 and HEVC standards. The experiments are mainly focused on testing the encoder’s real-time ability to deal with single video input obtained from the industrial camera using different settings. To do that, some system performance parameters for the continuous monitoring during the NVENC encoding process were chosen. Results in tables I and II are compared, discussed and the best NVENC implementation with encoding scheme is chosen in the context of our briefly introduced novel panoramic system. Finally, the system is fully loaded by the 7 cameras, all streams are encoded using an optimal encoding scheme and a performance limit is established for our hardware configuration.

A Flexible Coding Scheme Based on Block Krylov Subspace Approximation for Light Field Displays with Stacked Multiplicative Layers.

To create a realistic 3D perception on glasses-free displays, it is critical to support continuous motion parallax, greater depths of field, and wider fields of view. A new type of Layered or Tensor light field 3D display has attracted greater attention these days. Using only a few light-attenuating pixelized layers (e.g., LCD panels), it supports many views from different viewing directions that can be displayed simultaneously with a high resolution. This paper presents a novel flexible scheme for efficient layer-based representation and lossy compression of light fields on layered displays. The proposed scheme learns stacked multiplicative layers optimized using a convolutional neural network (CNN). The intrinsic redundancy in light field data is efficiently removed by analyzing the hidden low-rank structure of multiplicative layers on a Krylov subspace. Factorization derived from Block Krylov singular value decomposition (BK-SVD) exploits the spatial correlation in layer patterns for multiplicative layers with varying low ranks. Further, encoding with HEVC eliminates inter-frame and intra-frame redundancies in the low-rank approximated representation of layers and improves the compression efficiency. The scheme is flexible to realize multiple bitrates at the decoder by adjusting the ranks of BK-SVD representation and HEVC quantization. Thus, it would complement the generality and flexibility of a data-driven CNN-based method for coding with multiple bitrates within a single training framework for practical display applications. Extensive experiments demonstrate that the proposed coding scheme achieves substantial bitrate savings compared with pseudo-sequence-based light field compression approaches and state-of-the-art JPEG and HEVC coders.

A Trellis Based Temporal Rate Allocation and Virtual Reference Frames for High Efficiency Video Coding

The High Efficiency Video Coding (HEVC) standard has now become the most popular video coding solution for video conferencing, broadcasting, and streaming. However, its compression performance is still a critical issue for adopting a large number of emerging video applications with higher spatial and temporal resolutions. To advance the current HEVC performance, we propose an efficient temporal rate allocation solution. The proposed method adaptively allocates the compression bitrate for each coded picture in a group of pictures by using a trellis-based dynamic programming approach. To achieve this task, we trained the trellis-based quantization parameter for each frame in a group of pictures considering the temporal layer position. We further improved coding efficiency by incorporating our proposed framework with other inter prediction methods such as a virtual reference frame. Experiments showed around 2% and 5% bitrate savings with our trellis-based rate allocation method with and without a virtual reference frame compared to the conventional HEVC standard, respectively.

Convolutional Neural Network-Based Occupancy Map Accuracy Improvement for Video-Based Point Cloud Compression

In video-based point cloud compression (V-PCC), a dynamic point cloud is projected onto geometry and attribute videos patch by patch for compression. In addition to the geometry and attribute videos, an occupancy map video is compressed into a V-PCC bitstream to indicate whether a two-dimensional (2D) point in the projected geometry video corresponds to any point in three-dimensional (3D) space. The occupancy map video is usually downsampled before compression to obtain a tradeoff between the bitrate and the reconstructed point cloud quality. Due to the accuracy loss in the downsampling process, some noisy points are generated, which leads to severe objective and subjective quality degradation of the reconstructed point cloud. To improve the quality of the reconstructed point cloud, we propose using a convolutional neural network (CNN) to improve the accuracy of the occupancy map video. We mainly make the following contributions. First, we improve the accuracy of the occupancy map video by formulating the problem as a binary segmentation problem since the pixel values of the occupancy map video are either 0 or 1. Second, in addition to the downsampled occupancy map video, we introduce a reconstructed geometry video as the other input of the CNN to provide more useful information in order to indicate the occupancy map video. To the best of our knowledge, this is the first learning-based work to improve the performance of V-PCC. Compared to state-of-the-art schemes, our proposed CNN-based approach achieves much more accurate occupancy map videos and significant bitrate savings.

Performance Comparison of Emerging EVC and VVC Video Coding Standards with HEVC and AV1

<i>The recent dramatic increase in video content consumption requires efficient video coding standards, which is specifically true for ultrahigh-definition (UltraHD) resolutions, such as 4K and 8K (i.e., 3840&#x00D7;2160 or 7680&#x00D7;4320 resolutions in terms of luma samples, respectively). The well-known high-efficiency video coding (HEVC) [H.265/Moving Pictures Expert Group (MPEG)-H] standard was approved in 2013. Although HEVC provides approximately 50&#x0025; coding gain compared to its predecessor advanced video coding (AVC) (H.264/MPEG-4), its adoption is still relatively slow. In addition, larger bitrate savings than those provided by HEVC are currently in demand. At the same time, work on the versatile video coding (VVC) and essential video coding (EVC) standards started in 2018. After intensive development efforts that continued for two and a half years, these two video coding standards have been recently finalized. VVC (H.266/MPEG-I) was developed jointly by the MPEG and the International Telecommunication Union-Telecommunication (ITU-T) Video Coding Experts Group (VCEG). On the other hand, EVC (MPEG-5) is an MPEG-only effort. In this paper, we compare the performance of EVC and VVC, in terms of both coding gains and computational complexity, to their predecessor&#x2014;the HEVC video coding standard. In addition, given the growing popularity of the AV1 video codec, which was recently developed by the Alliance for Open Media (AOM), we also include AV1 as an alternative baseline and provide corresponding comparison results. According to the experimental results, which have been carried out in a constant bitrate (CBR) mode, EVC provides about 30&#x0025; bitrate savings compared to HEVC for encoding 4K/2160p entertainment content (such as VoD) in terms of Bj&#x00F8;ntegaard-delta bitrate (BD-BR) peak-signal-to-noise ratio (PSNR)_YUV, while introducing an encoding computational complexity increase of approximately five times. VVC provides larger bitrate savings of about 40&#x0025; at a price of a significant encoding computational complexity increase of more than nine times. When the performance of HEVC CBR encoding (i.e., with the rate control disabled) was compared to that of AV1 VBR encoding (i.e., with the rate control enabled), it was found that AV1 provides bitrate savings of about 20&#x0025; compared to HEVC for encoding 4K/2160p video sequences, as a tradeoff of an encoding computational complexity increase by a factor of approximately four. The authors find both EVC and VVC to be very promising successors to HEVC in terms of coding gains and computational complexity, but the jury is still out on the speed of their adoption</i>.

Gaussian-Wiener Representation and Hierarchical Coding Scheme for Focal Stack Images

Focal stack images (FoSIs) are a set of 2D images that captured one scene with serial focal depths. The redundancies of FoSIs mainly come from gradual focused depths changes rather than motion of objects. Conventional coding schemes cannot fully exploit such redundancies, leading to coding inefficiency. In this paper, we propose a new Gaussian-Wiener representation to model the gradual focused depths changes among FoSIs. In the representation, image degradation-restoration relations are utilized to describe the focus-defocus changing characteristics of FoSIs. Based on this representation, we propose a new hierarchical coding scheme for fully exploiting the inter-frame redundancies of FoSIs. In the scheme, a Gaussian-Wiener representation based inter prediction (GWR-IP) is presented by embedding Gaussian convolution and Wiener deconvolution into normal video encoder. Block-wise focus-defocus changing of FoSIs can be predicted in bi-directional manner by solving optimization problem. For higher coding efficiency, a Gaussian-Wiener representation based hierarchical prediction structure (GWR-HPS) is also designed and applied in the coding scheme. The proposed coding scheme is performed on 10 test sequences, including 5 synthetic scenes and 5 realistic scenes. Experimental results show that proposed coding scheme can obtain 2.640 dB PSNR gains and 51.830% bitrate savings on average of all test sequences in Low Delay P configuration, 2.123 dB PSNR gains and 43.975% bitrate savings in Low Delay B configuration, and 1.044 dB PSNR gains and 26.078% bitrate savings in Random Access configuration. Particularly, it achieves up to 65.544% bit rate savings and 3.901 dB PSNR increments for test sequence I09 in Low Delay P configuration. Furthermore, ablation test demonstrates that Gaussian representation contributes more on coding performance than Wiener representation and GWR-HPS.

Rate–distortion optimization of multi-exposure image coding for high dynamic range image coding

High dynamic range (HDR) images have many practical applications because they offer an extended dynamic range and a more realistic visual experience. A HDR image is usually stored in floating-point format, so pre-processing is required to make the HDR image compatible with coding standards. A transfer function is also used to achieve better coding efficiency. Typically, HDR images are generated using several low dynamic range (LDR) images with different exposures. Instead of compressing the HDR image when it is generated from images with multiple exposures, this study proposes a technique to compress the multi-exposure images. The HDR image generation, as well as the multi-exposure images fusion, can be realized in the decoder. The proposed framework encodes the multi-exposure images using MV-HEVC where the inter-view redundancy is well exploited when an accurate intensity-mapping function between the multi-exposure images has been established. Multi-exposure image coding is used to produce a high-quality HDR image so the rate–distortion optimization (RDO) is modified by considering both the reconstruction quality of the current block and its effect on the multi-exposure fused image. A Lagrange multiplier is modified to maintain a balance between the rate and the modified distortion during the RDO process. Compared to encoding the generated HDR image using HEVC range extension, the experimental results show that the proposed technique achieves significant bitrate savings for equivalent quality in terms of HDR-VDP-2.

Spatiotemporal Generative Adversarial Network-Based Dynamic Texture Synthesis for Surveillance Video Coding

Dynamic texture refers to the content in video sequences that is characterized by spatial repetition and temporal variation, such as swaying foliage and flowing water. It is a great challenge to compress the dynamic textures efficiently in the current prediction/transform hybrid video coding framework. However, these textures have little information for machine vision, and human visual perception is less sensitive to the textures than to the structures. Thus, we propose a spatiotemporal generative adversarial network (GAN) based dynamic texture synthesis method for surveillance video coding. We detect and remove the dynamic texture content at encoder side, which is irrelevant to machine vision. We generate the dynamic texture content using the proposed GAN at decoder side, so that the reconstructed videos can be observed by human without deteriorating perceptual quality. Specifically, we design a GAN network to synthesize dynamic textures by exploiting the correlation between spatial and temporal neighbors; we present a surveillance video coding scheme with the dynamic texture detection/synthesis method; we build a high-quality dynamic texture dataset, and we collect a dynamic texture testing dataset that goes beyond the existing video coding test datasets by focusing on surveillance scenes. The proposed video coding scheme has been implemented on top of the High Efficiency Video Coding (HEVC) reference software. Experiments have been conducted to evaluate the quantitative and qualitative performance of the proposed coding scheme. Our method achieves 7.4% and 7.6% bit-rate savings in low-delay-B and low-delay-P settings, respectively, at similar visual quality levels in comparison with HEVC.

Light field image coding with flexible viewpoint scalability and random access

This paper proposes a novel light field image compression approach with viewpoint scalability and random access functionalities. Although current state-of-the-art image coding algorithms for light fields already achieve high compression ratios, there is a lack of support for such functionalities, which are important for ensuring compatibility with different displays/capturing devices, enhanced user interaction and low decoding delay. The proposed solution enables various encoding profiles with different flexible viewpoint scalability and random access capabilities, depending on the application scenario. When compared to other state-of-the-art methods, the proposed approach consistently presents higher bitrate savings (44% on average), namely when compared to pseudo-video sequence coding approach based on HEVC. Moreover, the proposed scalable codec also outperforms MuLE and WaSP verification models, achieving average bitrate saving gains of 37% and 47%, respectively. The various flexible encoding profiles proposed add fine control to the image prediction dependencies, which allow to exploit the tradeoff between coding efficiency and the viewpoint random access, consequently, decreasing the maximum random access penalties that range from 0.60 to 0.15, for lenslet and HDCA light fields.

Video quality evaluation and testing verification of H.264, HEVC, VVC and EVC video compression standards

This paper is a comparative analysis with respect to video compression, coding efficiency with respect to performance and quality of the compressed video of Advanced Video Coding (H.264), High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC) and MPEG-5 Essential Video Coding (EVC). The approach used for coding and comparative analysis is by keeping the video quality (QP) and group of pictures (GOP) constant for all the four codecs. Performance with respect to quality and coding efficiency is tabulated with respect to objective quality metrics analysis like (PSNR, M-SSIM, Bitrate, compression ratio) and subjective quality analysis. From both these techniques, it is observed that the VVC is superior to EVC, HEVC and H.264 in terms of compression. Therefore, more bitrate savings is observed for VVC which is 15% to 45% more when compared to EVC. Due to the complexity of the encoder block, VVC takes almost twice the time taken to encode EVC and HEVC taking the least time to encode the video bit streams. The conclusion of the quality of video keeping system parameters constant, showed VVC performing better than EVC, followed by HEVC and then H.264 for both objective and subjective analysis. Time taken to decode the encoded sequences is almost same with very less delta difference between the video codecs. The codec configurations with algorithmic enhancements is also explained in this paper.

Adaptive Video Encoding for Different Video Codecs

By 2022, we expect video traffic to reach 82% of the total internet traffic. Undoubtedly, the abundance of video-driven applications will likely lead internet video traffic percentage to a further increase in the near future, enabled by associate advances in video devices' capabilities. In response to this ever-growing demand, the Alliance for Open Media (AOM) and the Joint Video Experts Team (JVET) have demonstrated strong and renewed interest in developing new video codecs. In the fast-changing video codecs' landscape, there is thus, a genuine need to develop adaptive methods that can be universally applied to different codecs. In this study, we formulate video encoding as a multi-objective optimization process where video quality (as a function of VMAF and PSNR), bitrate demands, and encoding rate (in encoded frames per second) are jointly optimized, going beyond the standard video encoding approaches that focus on rate control targeting specific bandwidths. More specifically, we create a dense video encoding space (offline) and then employ regression to generate forward prediction models for each one of the afore-described optimization objectives, using only Pareto-optimal points. We demonstrate our adaptive video encoding approach that leverages the generated forward prediction models that qualify for real-time adaptation using different codecs (e.g., SVT-AV1 and x265) for a variety of video datasets and resolutions. To motivate our approach and establish the promise for future fast VVC encoders, we also perform a comparative performance evaluation using both subjective and objective metrics and report on bitrate savings among all possible pairs between VVC, SVT-AV1, x265, and VP9 codecs.

Frequency Domain Long-Term Prediction for Low Delay General Audio Coding

In this paper we propose a long-term prediction method for low delay transform domain general audio coders. This Frequency Domain Joint Harmonics Prediction (FDJHP) method operates directly in the Modified Discrete Cosine Transform (MDCT) domain and can enhance the coding efficiency, even under very low frequency resolutions. We compare this new method with state-of-the-art MDCT based methods by analyzing bitrate savings and by a listening test using test signals with strong harmonic components. The results indicate that it outperforms an existing method, which also directly operates in the frequency domain. Additionally, we show how it can be combined with the existing techniques into an adaptive system, where the different methods can complement each other.

Quality-Driven Variable Frame-Rate for Green Video Coding in Broadcast Applications

The Digital Video Broadcasting (DVB) has proposed to introduce the Ultra-High Definition services in three phases: UHD-1 phase 1, UHD-1 phase 2 and UHD-2. The UHD-1 phase 2 specification includes several new features such as High Dynamic Range (HDR) and High Frame-Rate (HFR). It has been shown in several studies that HFR (+100 fps) enhances the perceptual quality and that this quality enhancement is content-dependent. On the other hand, HFR brings several challenges to the transmission chain including codec complexity increase and bit-rate overhead, which may delay or even prevent its deployment in the broadcast echo-system. In this paper, we propose a Variable Frame Rate (VFR) solution to determine the minimum (critical) frame-rate that preserves the perceived video quality of HFR video. The frame-rate determination is modeled as a 3-class classification problem which consists in dynamically and locally selecting one frame-rate among three: 30, 60 and 120 frames per second. Two random forests classifiers are trained with a ground truth carefully built by experts for this purpose. The subjective results conducted on ten HFR video contents, not included in the training set, clearly show the efficiency of the proposed solution enabling to locally determine the lowest possible frame-rate while preserving the quality of the HFR content. Moreover, our VFR solution enables significant bit-rate savings and complexity reductions at both encoder and decoder sides.

Improving compression efficiency of HEVC using perceptual coding

The introduction of the new coding tools in HEVC has brought significant bitrate savings compared to the previous standard, H.264/AVC. In both standards, the mean square error (MSE) is used for measuring distortion in the rate distortion optimization process of the coding unit structure and mode selection. However, MSE is not a good measure to use for measuring visual quality as it poorly correlates with human perception. Integration of a video quality metric based on the characteristics of the Human Visual System (HVS) inside the rate distortion optimization procedure is expected to improve the compression efficiency of video coding. In this paper, the PSNR-HVS measure is used in the rate distortion optimization process for the coding unit structure and mode selection. In the first step, we find the scaling factor for the Lagrangian multiplier based on the proposed perceptual approach. In the second step, we find optimal Lagrangian multiplier depending on the quantization parameter. The compression efficiency of the proposed approach is compared to that of HEVC. Simulations prove that the proposed approach yields higher compression efficiency.

Adaptive CU split prediction and fast mode decision for 3D-HEVC texture coding based on just noticeable difference model

3D-high efficiency video coding (3D-HEVC) is the latest extension of HEVC standard. Through adopting novel coding tools, 3D-HEVC could improve the compression efficiency of multi-view texture video. However, with the great bitrate savings, 3D-HEVC brings the extreme complexity. To reduce the encoding time of texture video, a fast 3D-HEVC method based on just noticeable difference (JND) model is proposed in this paper. The JND model is introduced to analyze the coding tree block (CTB) characteristics of the texture image. Moreover, combining with adaptive coding unit (CU) split prediction and fast mode decision, the proposed scheme could skip some CTBs of texture video at the early stage without a normal coding procedure. Simulation results demonstrate that the proposed scheme can reduce 51.5% encoding runtime on average with negligible loss of subjective quality.

Efficient Projected Frame Padding for Video-Based Point Cloud Compression

The state-of-the-art 2D-based dynamic point cloud (DPC) compression algorithm is the video-based point cloud compression (V-PCC) developed by the Moving Pictures Experts Group (MPEG). It first projects the DPC patch by patch from 3D to 2D and organizes the projected patches into a video. The video is then efficiently compressed by High Efficiency Video Coding. However, there are many unoccupied pixels that may have a significant influence on the coding efficiency. These unoccupied pixels are currently padded using either the average of 4-neighbors for the geometry or the push-pull algorithm for the color attribute. While these algorithms are simple, the unoccupied pixels are not handled in the most efficient way. In this paper, we divide the unoccupied pixels into two groups: those that should be occupied and those that should not be occupied according to the occupancy map. We then design padding algorithms tailored to each group to improve the rate-distortion performance of the V-PCC reference software, for both the geometry and the color attribute. The first group is the unoccupied pixels that should be occupied according to the block-based occupancy map. We attempt to pad those pixels using the real points in the original DPC to improve the quality of the reconstructed DPC. Additionally, we attempt to maintain the smoothness of each block so as not to negatively influence the video compression efficiency. The second group is the unoccupied pixels that were correctly identified as unoccupied according to the block-based occupancy map. These pixels are useless for improving the reconstructed quality of the DPC. Therefore, we attempt to minimize the bit cost of these pixels without considering their reconstruction qualities. The bit cost is determined by the residue of these pixels obtained by subtracting the prediction pixels from the original pixels. Therefore, we propose padding the residue using the average residue of the occupied pixels in order to minimize the bit cost. The proposed algorithms are implemented in the V-PCC and the corresponding HEVC reference software. The experimental results show the proposed algorithms can bring significant bitrate savings compared with the V-PCC.

A signal adaptive diffusion filter for video coding: Mathematical framework and complexity reductions

In this paper we combine video compression and modern image processing methods. We construct novel iterative filter methods for prediction signals based on Partial Differential Equation (PDE) based methods. The mathematical framework of the employed diffusion filter class is given and some desirable properties are stated. In particular, two types of diffusion filters are constructed: a uniform diffusion filter using a fixed filter mask and a signal adaptive diffusion filter that incorporates the structures of the underlying prediction signal. The latter has the advantage of not attenuating existing edges while the uniform filter is less complex. The filters are embedded into a software based on HEVC with additional QTBT (Quadtree plus Binary Tree) and MTT (Multi-Type-Tree) block structure. In this setting, several measures to reduce the coding complexity of the tool are introduced, discussed and tested thoroughly. The coding complexity is reduced by up to 70% while maintaining over 80% of the gain. Overall, the diffusion filter method achieves average bitrate savings of 2.27% for Random Access having an average encoder runtime complexity of 119% and 117% decoder runtime complexity. For individual test sequences, results of 7.36% for Random Access are accomplished.

Learned Representation of Satellite Image Series for Data Compression

Real-time transmission of satellite video data is one of the fundamentals in the applications of video satellite. Making use of the historical information to eliminate the long-term background redundancy (LBR) is considered to be a crucial way to bridge the gap between the compressed data rate and the bandwidth between the satellite and the Earth. The main challenge lies in how to deal with the variant image pixel values caused by the change of shooting conditions while keeping the structure of the same landscape unchanged. In this paper, we propose a representation learning based method to model the complex evolution of the landscape appearance under different conditions by making use of the historical image series. Under this representation model, the image is disentangled into the content part and the style part. The former represents the consistent landscape structure, while the latter represents the conditional parameters of the environment. To utilize the knowledge learned from the historical image series, we generate synthetic reference frames for the compression of video frames through image translation by the representation model. The synthetic reference frames can highly boost the compression efficiency by changing the original intra-frame prediction to inter-frame prediction for the intra-coded picture (I frame). Experimental results show that the proposed representation learning-based compression method can save an average of 44.22% bits over HEVC, which is significantly higher than that using references generated under the same conditions. Bitrate savings reached 18.07% when applied to satellite video data with arbitrarily collected reference images.

Per Clip Lagrangian Multiplier Optimisation for HEVC

The majority of internet traffic is video content. This drives the demand for video compression in order to deliver high quality video at low target bitrates. This paper investigates the impact of adjusting the rate distortion equation on compression performance. An constant of proportionality, k, is used to modify the Lagrange multiplier used in H.265 (HEVC). Direct optimisation methods are deployed to maximise BD-Rate improvement for a particular clip. This leads to up to 21% BD-Rate improvement for an individual clip. Furthermore we use a more realistic corpus of material provided by YouTube. The results show that direct optimisation using BD-rate as the objective function can lead to further gains in bitrate savings that are not available with previous approaches.