Backward Compatible High Dynamic Range Research Articles

Improvement of JPEG XT Floating-Point HDR Image Coding Using Region Adaptive Prediction

Displays capable of representing high-dynamic range (HDR) images were recently released in the digital consumer electronics market, and consumers have become increasingly interested in HDR images. However, the majority of today’s digital imaging devices are still low dynamic range (LDR) using an 8-bit representation for each RGB color components; hence, a backward-compatible HDR image format with an existing LDR-based imaging environment is required. Accordingly, JPEG XT, a new HDR image coding standard providing JPEG backward compatibility was established by the JPEG committee based on the market needs. JPEG XT consists of a base and a residual layer. Both layers are independently encoded by a legacy JPEG encoder. Most importantly, the base layer contains a tone-mapped LDR version of the HDR image for compliance with a legacy JPEG decoder. To date, three profiles have been defined for JPEG XT to encode HDR images, which are represented using floating-point values. This paper proposes a new image-coding scheme that uses a region adaptive prediction method with modified current specifications on JPEG XT Part 7. The proposed method adaptively predicts the HDR pixel value using different prediction information for each block based on the ratio between the LDR and HDR blocks, which is invariant to the tone-mapping operator, while the existing JPEG XT profiles globally predict the HDR values using an inverse transformation of the LDR values. Experimental results using various sample images show the superiority of the proposed method in terms of objective and visual evaluations.

Gradient-Based Tone Mapping for Rate-Distortion Optimized Backward-Compatible High Dynamic Range Compression.

This paper addresses the problem of designing a global tone mapping operator for rate distortion optimized backward compatible compression of high dynamic range (HDR) images. We address the problem of tone mapping design for two different use cases leading to two different minimization problems. The first problem considered is the minimization of the distortion on the reconstructed HDR signal under a rate constraint on the standard dynamic range (SDR) layer. The second problem remains the same minimization with an additional constraint to preserve a good quality for the SDR signal. Both the distortion and rate are expressed as a function of the spatial gradient in HDR images. Experiments show that the proposed rate and distortion models based on the HDR image gradient accurately predict the real image rate and distortion measures. Experimental results show that for the first minimization, the optimal rate-distortion performances are achieved, and that the second optimization yields the best tradeoff between rate-distortion performance and quality preservation of the SDR signal.

Evaluation of JPEG XT for high dynamic range cameras

Modern cameras can be used to capture high dynamic range (HDR) images by taking multiple low dynamic range (LDR) pictures at different exposure settings. However, DSLR cameras often store the multiple exposure pictures, which increases storage requirements proportionally to the number of exposures, while smartphones and tablets only store a tone-mapped representation of the HDR image instead of the original HDR data. In this paper, we investigate the bit rate reduction that could be achieved by storing HDR images compressed with JPEG XT, a JPEG backward-compatible HDR image compression standard, over multiple exposures JPEG. We also investigate the quality increase that could be achieved by fusing images with linear values and native sensor bit depth over pictures with sRGB gamma encoded values and 8 bits per sample. To investigate the impact of the different parameters on the performance, we perform a set of experiments with 14 contents, 4 resolutions (ranging from 2.5 megapixels (MP) to 16MP), 4 exposures configurations (2, 3, 5, and 7 exposures), 5 tone mapping operators (TMOs) used to generate the JPEG XT base layer, and 5 quality levels representative of values commonly used in smartphones and DSLR cameras. Results show that the performance is dependent on the TMO and number of exposures, with maximum average bit rate savings ranging from 10% for two exposures up to 70% for seven exposures. Additionally, quality improvements of 1.7dB to 5.1dB can be achieved when using the native image sensor data format.

Methods for improving the tone mapping for backward compatible high dynamic range image and video coding

Backward compatibility for high dynamic range image and video compression forms one of the essential requirements in the transition phase from low dynamic range (LDR) displays to high dynamic range (HDR) displays. In a recent work [1], the problems of tone mapping and HDR video coding are originally fused together in the same mathematical framework, and an optimized solution for tone mapping is achieved in terms of the mean square error (MSE) of the logarithm of luminance values. In this paper, we improve this pioneer study in three aspects by considering its three shortcomings. First, the proposed method [1] works over the logarithms of luminance values which are not uniform with respect to Human Visual System (HVS) sensitivity. We propose to use the perceptually uniform luminance values as an alternative for the optimization of tone mapping curve. Second, the proposed method [1] does not take the quality of the resulting tone mapped images into account during the formulation in contrary to the main goal of tone mapping research. We include the LDR image quality as a constraint to the optimization problem and develop a generic methodology to compromise the trade-off between HDR and LDR image qualities for coding. Third, the proposed method [1] simply applies a low-pass filter to the generated tone curves for video frames to avoid flickering during the adaptation of the method to the video. We instead include an HVS based flickering constraint to the optimization and derive a methodology to compromise the trade-off between the rate-distortion performance and flickering distortion. The superiority of the proposed methodologies is verified with experiments on HDR images and video sequences.

Optimizing a Tone Curve for Backward-Compatible High Dynamic Range Image and Video Compression

For backward compatible high dynamic range (HDR) video compression, the HDR sequence is reconstructed by inverse tone-mapping a compressed low dynamic range (LDR) version of the original HDR content. In this paper, we show that the appropriate choice of a tone-mapping operator (TMO) can significantly improve the reconstructed HDR quality. We develop a statistical model that approximates the distortion resulting from the combined processes of tone-mapping and compression. Using this model, we formulate a numerical optimization problem to find the tone-curve that minimizes the expected mean square error (MSE) in the reconstructed HDR sequence. We also develop a simplified model that reduces the computational complexity of the optimization problem to a closed-form solution. Performance evaluations show that the proposed methods provide superior performance in terms of HDR MSE and SSIM compared to existing tone-mapping schemes. It is also shown that the LDR image quality resulting from the proposed methods matches that produced by perceptually-based TMOs.

Backward compatible high dynamic range MPEG video compression

To embrace the imminent transition from traditional low-contrast video (LDR) content to superior high dynamic range (HDR) content, we propose a novel backward compatible HDR video compression (HDR MPEG) method. We introduce a compact reconstruction function that is used to decompose an HDR video stream into a residual stream and a standard LDR stream, which can be played on existing MPEG decoders, such as DVD players. The reconstruction function is finely tuned to the content of each HDR frame to achieve strong decorrelation between the LDR and residual streams, which minimizes the amount of redundant information. The size of the residual stream is further reduced by removing invisible details prior to compression using our HDR-enabled filter, which models luminance adaptation, contrast sensitivity, and visual masking based on the HDR content. Designed especially for DVD movie distribution, our HDR MPEG compression method features low storage requirements for HDR content resulting in a 30% size increase to an LDR video sequence. The proposed compression method does not impose restrictions or modify the appearance of the LDR or HDR video. This is important for backward compatibility of the LDR stream with current DVD appearance, and also enables independent fine tuning, tone mapping, and color grading of both streams.