High Dynamic Range Video Compression Research Articles

Uniform Color Space-Based High Dynamic Range Video Compression

Recently, there has been a significant progress in the research and development of the high dynamic range (HDR) video technology and the state-of-the-art video pipelines are able to offer a higher bit depth support to capture, store, encode, and display HDR video content. In this paper, we introduce a novel HDR video compression algorithm, which uses a perceptually uniform color opponent space, a novel perceptual transfer function to encode the dynamic range of the scene, and a novel error minimization scheme for accurate chroma reproduction. The proposed algorithm was objectively and subjectively evaluated against four state-of-the-art algorithms. The objective evaluation was conducted across a set of 39 HDR video sequences, using the latest x265 10-bit video codec along with several perceptual and structural quality assessment metrics at 11 different quality levels. Furthermore, a rating-based subjective evaluation ( $n=40$ ) was conducted with six sequences at two different output bitrates. Results suggest that the proposed algorithm exhibits the lowest coding error amongst the five algorithms evaluated. Additionally, the rate–distortion characteristics suggest that the proposed algorithm outperforms the existing state-of-the-art at bitrates ≥ 0.4 bits/pixel.

An Adaptive Quantizer for High Dynamic Range Content: Application to Video Coding

In this paper, we propose an adaptive perceptual quantization method to convert the representation of high dynamic range (HDR) content from the floating point data type to integer, which is compatible with the current image/video coding and display systems. The proposed method considers the luminance distribution of the HDR content, as well as the detectable contrast threshold of the human visual system, in order to preserve more contrast information than the perceptual quantizer (PQ) in integer representation. Aiming to demonstrate the effectiveness of this quantizer for HDR video compression, we implemented it in a mapping function on the top of the HDR video coding system based on high efficiency video coding standard. Moreover, a comparison function is also introduced to decrease the additional bit-rate of side information, generated by the mapping function. Objective quality measurements and subjective tests have been conducted in order to evaluate the quality of the reconstructed HDR videos. Subjective test results have shown that the proposed method can improve, in a significant manner, the perceived quality of some reconstructed HDR videos. In the objective assessment, the proposed method achieves improvements over PQ in terms of the average bit-rate gain for metrics used in the measurement.

Improved HEVC $$\lambda $$ λ -domain rate control algorithm for HDR video

Widespread high dynamic range (HDR) video distribution via transmission and broadcast is imminent in the near future. However, the rate control (RC) algorithms in video coding standards, like high-efficiency video coding (HEVC), are optimized and designed for low dynamic range (LDR) content—making them inefficient in the rate-distortion (RD) sense, when applied to HDR video compression and distribution. In this paper, we propose a non-normative change to the HM16.2 HEVC $$\lambda $$ -domain RC algorithm to achieve high-fidelity HDR video distribution. A new $$\lambda $$ -QP relation is modeled for HDR content after assessing the suitable RD model. The results indicate that the proposed RC algorithm outperforms the default RC algorithm in HEVC, achieving on average performance gains of 1.40 dB, 0.685 units, and 0.0155 units in terms of PU-PSNR, HDR-VDP-2.2 Q factor, and HDR-VQM, respectively. Further, proposed method has bit estimation accuracy similar to the default HEVC RC algorithm. Also, the subjective evaluations corroborate the significance of the performance gained in the objective metrics.

Adaptive perceptual quantizer for high dynamic range video compression

Although high dynamic range (HDR) videos are a very fascinating way to represent real-world scenes, they need a huge amount of memory to store and transmit due to the high bit-depth. Thus, it is a major challenge for HDR video coding to efficiently compress them without sacrificing perceptual quality. Perceptual Quantizer (PQ) transfer function provides a solution to this problem, which is adopted as the HEVC Main 10 Profile-based Anchor. However, PQ is not adaptive to HDR contents, thus reducing the coding efficiency. In this paper, we propose adaptive transfer function based on PQ for HDR video compression, called adaptive PQ. Different from PQ which uses a fixed mapping curve from luminance to luma, the proposed transfer function adaptively maps luminance to luma according to HDR contents. Thus, adaptive PQ is able to efficiently utilize possible luma values. Moreover, adaptive PQ achieves better perceptual uniformity in the luminance range than PQ. Experimental results demonstrate that adaptive PQ achieves a significant performance improvement in HDR video coding over PQ in terms of visual quality and bitrate.

HDR video quality assessment: Perceptual evaluation of compressed HDR video

Compared with standard dynamic range (SDR) video, the high dynamic range (HDR) video can provide us significantly enhanced viewing experience. In particular, compared to SDR video, the HDR video has better contrast and preserves more details for the same scene. With the rapid development of HDR video compression technology, there is a lack of trusted quality measure of HDR video compression. In order to facilitate the future development of objective HDR quality assessment, we build a HDR video quality assessment database, in which the bitstream is created by compressing a series of HDR video sequences. In the compression, the quantization parameters (QP) are set to 12 levels according to the configuration of the codec. The subjective quality of each bitstream is rated by 22 viewers. It is revealed that the subject viewers have arrived at a reasonable agreement on the subjective quality of different QP levels. This paper presents the results of subjective quality assessment of HDR compressed video, which also exhibits that there is significant room to further improve the objective HDR video quality assessment algorithms.

Spatio-temporal constrained tone mapping operator for HDR video compression

With the growing popularity of high dynamic range (HDR) imaging, efficient compression techniques are demanded, as HDR video entails typically higher raw data rate than traditional video. For this purpose, we introduce a hybrid spatially and temporally constrained content-adaptive tone mapping operator (TMO) to convert the input HDR video into a tone mapped video sequence, which is then encoded using the high efficiency video coding (HEVC) standard. The proposed TMO simultaneously exploits intra-frame spatial redundancies and preserves inter-frame temporal coherence of the tone mapped video sequence. Extensive experimental results show that the developed spatio-temporal TMO (ST-TMO) solution yields higher coding performance than existing frame-by-frame TMO’s, and compares favorably with state-of-the-art methods based on a fixed transfer function.

An evaluation of power transfer functions for HDR video compression

High dynamic range (HDR) imaging enables the full range of light in a scene to be captured, transmitted and displayed. However, uncompressed 32-bit HDR is four times larger than traditional low dynamic range (LDR) imagery. If HDR is to fulfil its potential for use in live broadcasts and interactive remote gaming, fast, efficient compression is necessary for HDR video to be manageable on existing communications infrastructure. A number of methods have been put forward for HDR video compression. However, these can be relatively complex and frequently require the use of multiple video streams. In this paper, we propose the use of a straightforward Power Transfer Function (PTF) as a practical, computationally fast, HDR video compression solution. The use of PTF is presented and evaluated against four other HDR video compression methods. An objective evaluation shows that PTF exhibits improved quality at a range of bit-rates and, due to its straightforward nature, is highly suited for real-time HDR video applications.

Objective and subjective evaluation of High Dynamic Range video compression

A number of High Dynamic Range (HDR) video compression algorithms proposed to date have either been developed in isolation or only-partially compared with each other. Previous evaluations were conducted using quality assessment error metrics, which for the most part were developed for qualitative assessment of Low Dynamic Range (LDR) videos. This paper presents a comprehensive objective and subjective evaluation conducted with six published HDR video compression algorithms. The objective evaluation was undertaken on a large set of 39 HDR video sequences using seven numerical error metrics namely: PSNR, logPSNR, puPSNR, puSSIM, Weber MSE, HDR-VDP and HDR-VQM. The subjective evaluation involved six short-listed sequences and two ranking-based subjective experiments with hidden reference at two different output bitrates with 32 participants each, who were tasked to rank distorted HDR video footage compared to an uncompressed version of the same footage. Results suggest a strong correlation between the objective and subjective evaluation. Also, non-backward compatible compression algorithms appear to perform better at lower output bit rates than backward compatible algorithms across the settings used in this evaluation.

Videoshop: A new framework for spatio-temporal video editing in gradient domain

This paper proposes a new framework for video editing in gradient domain. The spatio-temporal gradient fields of target videos are modified and/or mixed to generate a new gradient field which is usually not integrable. We compare two methods to solve this “mixed gradient problem”, i.e., the variational method and loopy belief propagation. We propose a 3D video integration algorithm, which uses the variational method to find the potential function whose gradient field is closest to the mixed gradient field in the sense of least squares. The video is reconstructed by solving a 3D Poisson equation. The main contributions of our framework lie in three aspects: first, we derive a straightforward extension of current 2D gradient technique to 3D space, thus resulting in a novel video editing framework, which is very different from all current video editing software; secondly, we propose using a fast and accurate 3D discrete Poisson solver which uses diagonal multigrids to solve the 3D Poisson equation, which is up to twice as fast as a simple conventional multigrid algorithm; finally, we introduce a set of new applications, such as face replacement and painting, high dynamic range video compression and graphcut based video compositing. A set of gradient operators is also provided to the user for editing purposes. We evaluate our algorithm using a variety of examples for image/video or video/video pairs. The resulting video can be seamlessly reconstructed.

Backward compatible high dynamic range MPEG video compression

To embrace the imminent transition from traditional low-contrast video (LDR) content to superior high dynamic range (HDR) content, we propose a novel backward compatible HDR video compression (HDR MPEG) method. We introduce a compact reconstruction function that is used to decompose an HDR video stream into a residual stream and a standard LDR stream, which can be played on existing MPEG decoders, such as DVD players. The reconstruction function is finely tuned to the content of each HDR frame to achieve strong decorrelation between the LDR and residual streams, which minimizes the amount of redundant information. The size of the residual stream is further reduced by removing invisible details prior to compression using our HDR-enabled filter, which models luminance adaptation, contrast sensitivity, and visual masking based on the HDR content. Designed especially for DVD movie distribution, our HDR MPEG compression method features low storage requirements for HDR content resulting in a 30% size increase to an LDR video sequence. The proposed compression method does not impose restrictions or modify the appearance of the LDR or HDR video. This is important for backward compatibility of the LDR stream with current DVD appearance, and also enables independent fine tuning, tone mapping, and color grading of both streams.