A Perception‐Guided Framework for Cross‐Media HDR ‐to‐ SDR Reproduction

Abstract. 3D reconstruction relies on accurate detection, extraction, description and matching of image features. This is even truer for complex architectural scenes that pose needs for 3D models of high quality, without any loss of detail in geometry or color. Illumination conditions influence the radiometric quality of images, as standard sensors cannot depict properly a wide range of intensities in the same scene. Indeed, overexposed or underexposed pixels cause irreplaceable information loss and degrade digital representation. Images taken under extreme lighting environments may be thus prohibitive for feature detection/extraction and consequently for matching and 3D reconstruction. High Dynamic Range (HDR) images could be helpful for these operators because they broaden the limits of illumination range that Standard or Low Dynamic Range (SDR/LDR) images can capture and increase in this way the amount of details contained in the image. Experimental results of this study prove this assumption as they examine state of the art feature detectors applied both on standard dynamic range and HDR images.

Recently, deep learning has been proven to be a promising approach in standard dynamic range (SDR) image compression. However, due to the wide luminance distribution of high dynamic range (HDR) images and the lack of large standard datasets, developing a deep model for HDR image compression is much more challenging. To tackle this issue, we view HDR data as distributional shifts of SDR data and the HDR image compression can be modeled as an out-of-distribution generalization (OoD) problem. Herein, we propose a novel out-of-distribution (OoD) HDR image compression framework (OoDHDR-codec). It learns the general representation across HDR and SDR environments, and allows the model to be trained effectively using a large set of SDR datases supplemented with much fewer HDR samples. Specifically, OoDHDR-codec consists of two branches to process the data from two environments. The SDR branch is a standard blackbox network. For the HDR branch, we develop a hybrid system that models luminance masking and tone mapping with white-box modules and performs content compression with black-box neural networks. To improve the generalization from SDR training data on HDR data, we introduce an invariance regularization term to learn the common representation for both SDR and HDR compression. Extensive experimental results show that the OoDHDR codec achieves strong competitive in-distribution performance and state-of-the-art OoD performance. To the best of our knowledge, our proposed approach is the first work to model HDR compression as OoD generalization problems and our OoD generalization algorithmic framework can be applied to any deep compression model in addition to the network architectural choice demonstrated in the paper. Code available at https://github.com/caolinfeng/OoDHDR-codec.

Broadcasting high dynamic range (HDR) video has been demonstrated as largely preferred when compared to standard dynamic range (SDR), mainly due to its capability of representing more details in dark and bright regions. Additionally, progress over the last decades on creation, compression, transmission and rendering of HDR content signals a forthcoming deployment of HDR broadcasting services. Lack of a widely supported recommendation regarding bandwidth allocation for HDR compressed streams or a unique compression approach, prevent faster deployment of such services. This paper investigates the performance of a dual-layer backward-compatible compression codec, when compared to state-of-the-art HDR compression strategies, in terms of perceived quality. The evaluated system is a dual-layer compression scheme enabling the transmission of a backward-compatible SDR stream along with an HDR stream, reconstructed from the residual-based enhancement layer and SDR mapping (i.e. prediction). Comparison is made to two compression strategies realizing uncompromised SDR or HDR through the use of single layer systems multiplexed with metadata. Metadata contains information necessary to map HDR into SDR or SDR into HDR streams. Our conclusion provides guidance regarding the compression strategy to use as well as bandwidth allocation for HDR delivery, ensuring both SDR and HDR contents with perceptually acceptable quality.

High dynamic range (HDR) image and video technology can provide significant picture quality improvement over the standard dynamic range (SDR). However, when HDR content is represented on an SDR display, dynamic range compression may result in image quality deterioration. To address this problem, we propose an optimized human visual system (HVS) response model-based tone-mapping algorithm to preserve the perceptual responses between the HDR image and its tone-mapped image. First, we measure the HVS response differences using a 2D histogram when an HDR image is displayed on an HDR device and when its tone-mapped image is displayed on an SDR device. Then, we formulate an optimization problem to minimize the differences. By efficiently solving the optimization problem, we obtain an optimal tone-mapping curve. Experimental results on actual displays demonstrate that the proposed algorithm provides superior image quality compared with conventional algorithms in terms of both subjective and objective evaluations.

In recent research in representing realistic appearance of materials, HDR(High Dynamic Range) images constructed from LDR(Low Dynamic Range) images taken with different exposure time using a digital camera are used to measure radiance of materials like BRDF. However, in order to reproduce the original color of the materials, they require color correction of the HDR images. Before constructing an HDR image, the color of each LDR image is corrected by using the ICC profile or other method. Then they make a color corrected HDR image from the color corrected LDR images [Goesele et al. 2001]. We call this method the LDR-level color correction. However, each pixel value of LDR images represents not only color but also intensity. Since the pixel values of the LDR images are modified for color correction before making an HDR image, the intensity of each pixel is also changed. In this paper, we propose a color correction method at HDR-level, where the correction is made after the HDR image is constructed. This gives a much better reproduction of the original color of a given material in our experiment.

Brightness is an attribute of visual perception used to describe the intensity of the light entering the eye. Since human perception is not linearly related to light intensity, characterizing brightness is a challenging task. In standard dynamic range (SDR) imagery, brightness is often quantified using the average picture level (APL), which is the average of all pixels’ code values normalized by the maximum allowed signal code value. APL provides a simple and commonly used brightness metric for SDR; however, its validity for high dynamic range (HDR) content has never been assessed. Due to the higher luminance range that HDR supports, HDR content is encoded using a different transfer function than SDR. Thus, a different distribution of pixel code values is to be expected between HDR and SDR content. In this work, we evaluate the efficiency of the APL metric to quantify the brightness of HDR content. We describe, using patches and professionally graded images, pixel distributions where the APL fails to distinguish relative brightness between pairs of images. To overcome APL shortcomings, we propose a brightness metric based on the geometric mean (GM) and variance of an image’s luma code values. We then conduct two subjective experiments to compare the efficiency of APL and our metric. Results show that the proposed metric predicts more accurately the relative brightness between two frames. However, our results also suggest that a simple statistical model, although useful as a general guideline, cannot be considered accurate enough for HDR content. Therefore, we hypothesize that work on a spatial model might provide a still better fit in characterizing brightness in HDR.

High dynamic range(HDR) imaging is a technique to represent the wider range of luminance from the lightest and darkest area of an image than normal digital imaging techniques. These techniques merge multiple images, called as LDR(low dynamic range) or SDR(standard dynamic range) images which have proper luminance with different exposure steps, to cover the entire dynamic range of real scenes. In the initial techniques, a series of acquisition process for LDR images according to exposure steps are required. However, several acquisition process of LDR images induce ghost artifact for HDR images due to moving objects. Recent researches have tried to reduce the number of LDR images with optimal exposure steps to eliminate the ghost artifacts. Nevertheless, they still require more than three times of acquisition processes, resulting ghosting artifacts. In this paper, we propose an HDR imaging from a single Bayer image with arbitrary exposures without additional acquisition processes. This method first generates new LDR images which are corresponding to each average luminance from user choices, based on Exposure LUTs(look-up tables). Since the LUTs contains relationship between uniform-gray patches and their average luminances according to whole exposure steps in a camera, new exposure steps for any average luminance can be easily estimated by applying average luminance of camera-output image and corresponding exposure step to LUTs. Then, objective LDR images are generated with new exposure steps from the current input image. Additionally, we compensate the color generation of saturated area by considering different sensitivity of each RGB channel from neighbor pixels in the Bayer image. Resulting HDR images are then merged by general method using captured images and estimated images for comparison. Observer's preference test shows that HDR images from the proposed method provides similar appearance with the result images using captured images.

Digital source identification is one of the most important problems in the field of multimedia forensics. While Standard Dynamic Range (SDR) images are commonly analyzed, High Dynamic Range (HDR) images are a less common research subject, which leaves space for further analysis. In this paper, we present a novel database of HDR and SDR images captured in different conditions, including various capturing motions, scenes and devices. As a possible application of this dataset, the performance of the well-known reference pattern noise-based source identification algorithm was tested on both kinds of images. Results have shown difficulties in source identification conducted on HDR images, due to their complexity and wider dynamic range. It is concluded that capturing conditions and devices themselves can have an impact on source identification, thus leaving space for more research in this field.

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Deploying separate high dynamic range (HDR) and standard dynamic range (SDR) production facilities to cover the same event simultaneously can be economically prohibitive. In this case study, we present a new method for creating both ultrahigh-definition (UHD) HDR and HD SDR streams from a single UHD HDR production workflow, developed through a series of real-world UHD HDR production trials in 2018 and 2019, concluding with the 2019 Football Association Challenge Cup (FA Cup), a major U.K. sporting event. We describe the findings from those trials and the lessons learned and include descriptions of what worked well and what worked less well. The workflow solution presented allows for the integration of both HDR camera sources and SDR camera sources (e.g., super slow-motion and specialist cameras), prerecorded or archive SDR content and SDR graphics into a single HDR production environment. It can exploit the backward-compatible nature of the hybrid log-gamma (HLG) HDR solution and recent advances in color volume management. We show how a production’s two outputs, HDR and SDR, can be made using current SDR shading/racking monitoring, with only a single HDR vision guarantee monitor</i> .

In this work, we address the problem of efficiently and accurately retrieving High Dynamic Range (HDR) images and displaying the retrieval results in Standard Dynamic Range (SDR) devices. The proposed approach considers a set of global descriptors for modeling an HDR image using vector of features that combines two color attributes: the color histogram based on the Hue-Saturation-Value (HSV) space and the color moments. The dissimilarity between images is measured using Manhattan distance. To evaluate the effectiveness of our retrieval system and conduct our tests, we created a collection of various HDR images. Experimental results demonstrate that the proposed method yields satisfactory results. It can accurately retrieve the similar HDR images. Moreover, we applied our retrieval method on Low Dynamic Range (LDR) version of the same dataset. The comparisons confirm that the features extracted from HDR images are more effective than those extracted from LDR ones.

High dynamic range (HDR) technology is one of various research topics in computer vision technologies, which has been actively being applied to cutting-edge consumer electronics products such as premium UHDTV, premium smart phones and etc. In case of consumer camera devices or mobile camera phones, HDR scene capture mode is commonly provided by combining differently-exposed images with standard dynamic range (SDR). In consumer premium TVs, there exist two kinds of HDR processing technologies: one is a tone mapping (TM) to render HDR content on SDR display, and the other one is an inverse tone mapping (iTM) to render a SDR content on HDR display. Especially, in the case of HDR ton mapping, some metadata for adjusting radiometric difference between a reference mastering display and various consumer TV displays were standardized as well as HDR signal encoding method. Currently there exist various organizations related to HDR standards: UHD-Alliance, SMPTE, CTA, ITU-R, MPEG, DVB and etc. Those HDR standard ecosystem members include movie studio, broadcasting, OTTs, TV/STB manufacturers, and etc. They have made efforts to create a next big opportunity of brand-new HDR technology on the basis of 4K UHD media. We will here review HDR technologies, standards, delivery system, and its CE industry trends.

Thanks to the recent developments in both High Dynamic Range (HDR) content creation and HDR consumer displays, HDR technology is now mature enough for consumer TV broadcasting. Considering that HDR displays have been introduced to the consumer market just recently, backward compatible broadcasting approaches that can also address the Standard Dynamic Range (SDR) displays are preferred during this transition period. While several backward compatible single layer approaches have been proposed for HDR content delivery, there is no comprehensive study on the subjective quality of the delivered content on both SDR and HDR displays. In this work, we investigate the performance of two single layer HDR delivery pipelines, namely HDR10 and SDR10 through three sets of subjective test experiments. The effect of different video preprocessing and post-processing methods used in the HDR10 and SDR10 pipelines on the visual quality of the HDR and SDR outputs is investigated in these comprehensive tests. The analysis of the results shows that the HDR10 pipeline can achieve superior HDR quality compared to that of the SDR10 while the SDR subjective quality of the HDR10 pipeline is comparable to that of the SDR10 pipeline. Therefore, it is concluded that HDR10 can be used as a backward compatible single layer pipeline addressing both HDR and SDR displays. It is also shown in this work that by addressing the display in 4:4:4 mode, the SDR subjective quality of the HDR10 pipeline is further improved compared to the 4:2:0 mode. The findings of this work can be used as guidelines for addressing both HDR and SDR displays with HDR10 pipeline.

Considerable progress has been made toward developing standard dynamic range (SDR) blind video quality assessment (BVQA) models that do not require any baseline reference for quality prediction. However, there is no such method for the high dynamic range (HDR) content. Unlike SDR video, HDR video represents a high-fidelity representation of the real-world scene by preserving the wide luminance range and color gamut. Therefore, SDR BVQA models are not suitable for HDR BVQA. Towards ameliorating this, a first-of-its-kind BVQA model for HDR content is presented in this work. The proposed HDR blind video quality model (HDR-BVQM) is inspired by the spatio-temporal natural scene statistics model, previously employed in SDR blind quality assessment metrics. To build our proposed model, we first develop a comprehensive subjective HDR video quality dataset, including 228 distorted videos generated through three different (H.264, HEVC, packet drop) distortion processes from 19 pristine HDR videos. The developed dataset is then used to extract HDR relevant features, which vary in different distortion types, to train and test the proposed HDR-BVQM. The features are based on the pointwise, pairwise log-derivative, and motion coherence based statistics. Finally, detailed validation and performance comparison is performed with full-reference HDR and no-reference SDR quality assessment methods. The results reveal that the quality prediction by HDR-BVQM correlates with the human judgment of quality.

High dynamic range (HDR) imaging techniques have been working constantly, actively, and validly in the fault detection and disease diagnosis in the astronomical and medical fields, and currently they have also gained much more attention from digital image processing and computer vision communities. While HDR imaging devices are starting to have friendly prices, HDR display devices are still out of reach of typical consumers. Due to the limited availability of HDR display devices, in most cases tone mapping operators (TMOs) are used to convert HDR images to standard low dynamic range (LDR) images for visualization. But existing TMOs cannot work effectively for all kinds of HDR images, with their performance largely depending on brightness, contrast, and structure properties of a scene. To accurately measure and compare the performance of distinct TMOs, in this paper develop an effective and efficient no-reference objective quality metric which can automatically assess LDR images created by different TMOs without access to the original HDR images. Our model is shown to be statistically superior to recent full- and no-reference quality measures on the existing tone-mapped image database and a new relevant database built in this work.

Human observers are able to make fine discriminations of surface gloss. What cues are they using to perform this task? In previous studies, we identified two reflection-related cues-the contrast of the reflected image (c, contrast gloss) and the sharpness of reflected image (d, distinctness-of-image gloss)--but these were for objects rendered in standard dynamic range (SDR) images with compressed highlights. In ongoing work, we are studying the effects of image dynamic range on perceived gloss, comparing high dynamic range (HDR) images with accurate reflections and SDR images with compressed reflections. In this paper, we first present the basic findings of this gloss discrimination study then present an analysis of eye movement recordings that show where observers were looking during the gloss discrimination task. The results indicate that: 1) image dynamic range has significant influence on perceived gloss, with surfaces presented in HDR images being seen as glossier and more discriminable than their SDR counterparts; 2) observers look at both light source highlights and environmental interreflections when judging gloss; and 3) both of these results are modulated by surface geometry and scene illumination.