Inverse Tone Mapping Operator Research Articles

Analysis and simulation of the effect of large optical range difference of common path coherent-dispersion spectrometer on the detection of exoplanet radial velocities

The Exoplanet Explorer common-path coherent-dispersion spectrometer (CODES) utilizes a unique combination of an asymmetric common path Sagnac interferometer and a low to medium resolution spectrometer. The ideal optical range difference (OPD) interval for CODES is OPD ∈ {15.06 mm, 19.45 mm}; however, the OPD of CODES is 64.3 mm to achieve better detection accuracy. Though they will increase the accuracy of detection, large OPDs outside of the ideal interval will also reduce the contrast of the interference fringes, making phase changes more hazy. This may also significantly affect the radial velocity inversion and reduce CODES's instrumental accuracy. This study creates an inverse tone mapping operator based on the photographic model and designs an inverse tone mapping algorithm called CODESCE. The outcomes of the experiments demonstrate that the tone mapping algorithm CODESCE in this work is appropriate for enhancing the contrast of interference fringe images with high OPD, and it can enhance the contrast of interference fringes by three orders of magnitude when OPD = 64.3 mm; the processed interference fringes are located in the range of interference fringe curves of the optimal OPD. By comparison with other current approaches, the suggested algorithm yields superior processing outcomes.

HDR-LFNet: Inverse tone mapping using fusion network

To capture the real-world luminance values, High Dynamic Range (HDR) image processing has been developed. HDR images have a richer content than the widely-used Standard Dynamic Range (SDR) images, and are used in a number of situations, e.g. in film industry. As HDR displays are more and more commercially available, we need to be able to process HDR images as well as SDR ones (for example, devising denoising algorithms, inpainting or anti-aliasing). The most powerful methods to process images are now deep neural networks. However, the training of such networks calls for a lot of images, and HDR images datasets are relatively small.One way to generate HDR images is inverse tone mapping operators (iTMOs). They are algorithms that expand the dynamic range of SDR images. In this paper, we propose HDR-LFNet, a novel iTMO, and its HDR training dataset. Our method merges several existing handcrafted iTMOs, combined in a supervised neural network to produce an HDR output. Our lightweight network requires less training images than state-of-the-art methods, and has faster training phase. Besides, the quality of the generated images is similar to the state-of-the-art. We present the architecture as well as the subjective and experimental evaluations of our method.

A dynamic range adjustable inverse tone mapping operator based on human visual system

A dynamic range adjustable inverse tone mapping operator based on human visual system

HDR-Like Image from Pseudo-Exposure Image Fusion: A Genetic Algorithm Approach

Image distortion during multiple-exposure capture is a common scenario, and most of the existing legacy image content is in the low dynamic range (LDR). For both of these problems, pseudo multiple exposure image generation from a single image is a possible solution. Hence, in this article, we propose a novel method for high dynamic range (HDR)-like image generation by fusing pseudo-multi-exposure images that are produced from only one single image. The proposed pseudo-exposure image generation method is based on intensity mapping function (IMF). First, our system learns the relation between different exposure images with the help of genetic algorithm and generates long and short exposure images. In the second step, we fuse images with different exposures to generate the HDR-like image. Our method outperforms several inverse tone mapping operator (ITMO)-based methods. The simplicity and good performance of the proposed method makes it suitable for mobile phones and other consumer electronics. We also develop a demo software to validate the proposed method.

A review on high dynamic range (HDR) image quality assessment

Abstract This paper presents a literature review on the method of measuring high dynamic range (HDR) image quality. HDR technology can help maximize user satisfaction level when using HDR images-based visual services. The advance of HDR technology indirectly presents a more difficult challenge to the image quality assessment method due to the high sensitivity of the human visual system (HVS) to various kinds of distortions that may arise in HDR images. This is related to the process of HDR image generation, which in general can be classified into two broad categories: the formation using the multiple exposure fusion (MEF) method and the inverse tone mapping operator (ITMO) method. In this paper, we will outline how HDR image quality measurement method works and describe some examples of these measurement methods which are related to the way the HDR images are fabricated. From these methods, it can be seen that most of them are still focused on full-reference and no-reference quality models. We argue that there is still room for the development of reduced-reference HDR image quality assessment.

A Fully Automatic Content Adaptive Inverse Tone Mapping Operator With Improved Color Accuracy

High Dynamic Range (HDR) technology offers a higher visual quality compared to its Standard Dynamic Range (SDR) counterpart, as it tries to imitate the way our eyes perceive brightness and color information. Converting SDR content to HDR format - using inverse Tone Mapping Operators (iTMOs) - to take advantage of the superior visual quality offered by HDR displays, is an attractive proposition to SDR content owners and real-time broadcasters. In this paper, we propose a novel content adaptive iTMO that works in the perceptual domain to model the sensitivity of the human eye to brightness changes in different areas of a scene. To preserve the overall visual impression, our proposed iTMO utilizes an entropy-based brightness segmentation, which also makes our method content adaptive. In addition, we propose a novel perception-based color adjustment method that can maintain the color accuracy between input SDR and generated HDR frames. By performing the color adjustment in the perceptual domain, our iTMO prevents hue shifts and generates HDR colors that closely follow their SDR counterparts. Our subjective evaluations indicate that our proposed method outperforms other state-of-the-art methods by an average of 81% in terms of visual quality, and 76% in terms of how closely the HDR colors match their SDR counterparts. In addition to subjective evaluations, we also performed objective evaluations using the HDR-VDP 2.2 and PU-SSIM metrics and concluded that, on average, our proposed iTMO outperforms the state-of-the-art methods in terms of these two metrics.

Supervised Learning in Automatic Selection of Preferred Inverse Tone-Mapping Operator for HDR Display

Supervised Learning in Automatic Selection of Preferred Inverse Tone-Mapping Operator for HDR Display

A Perception-Based Inverse Tone Mapping Operator for High Dynamic Range Video Applications

The drastic visual improvements introduced by High Dynamic Range (HDR) technologies open new markets for a wide range of industries. Among them, a significant opportunity is offered to owners of legacy Standard Dynamic Range (SDR) content that can be converted to the new standard to take advantage of the enhanced capabilities of the HDR displays. Similarly, since SDR broadcasting infrastructure will continue to be around for the time being, such conversion process is becoming an obvious necessity. To this end, different approaches have tried to efficiently convert SDR images and videos to HDR format, a procedure well known as inverse Tone Mapping. In this paper, we propose a novel high visual quality video inverse Tone Mapping Operator (iTMO) that addresses the inadequacies of the state-of-the-art methods, resulting in high visual quality HDR videos that match the capabilities of the HDR technology. Our approach is based on human visual perception and employs a segmentation method according to the Human Visual System (HVS) sensitivity to brightness changes in different regions of the frame and constructs the mapping curve using the brightness distribution information of these regions. Our iTMO uses a hybrid approach to achieve an optimal balance between the overall contrast and brightness of the output HDR frame by maximizing a weighted sum of contrast and brightness difference between input SDR and generated HDR frame. The proposed iTMO works equally well for all levels of brightness, eliminating any visual artifacts by dynamically maintaining changes at non-perceivable levels. Subjective and objective evaluations validated the superior visual performance of our proposed iTMO over state-of-the-art methods.

Fully-automatic inverse tone mapping algorithm based on dynamic mid-level tone mapping

Abstract High Dynamic Range (HDR) displays can show images with higher color contrast levels and peak luminosities than the common Low Dynamic Range (LDR) displays. However, most existing video content is recorded and/or graded in LDR format. To show LDR content on HDR displays, it needs to be up-scaled using a so-called inverse tone mapping algorithm. Several techniques for inverse tone mapping have been proposed in the last years, going from simple approaches based on global and local operators to more advanced algorithms such as neural networks. Some of the drawbacks of existing techniques for inverse tone mapping are the need for human intervention, the high computation time for more advanced algorithms, limited low peak brightness, and the lack of the preservation of the artistic intentions. In this paper, we propose a fully-automatic inverse tone mapping operator based on mid-level mapping capable of real-time video processing. Our proposed algorithm allows expanding LDR images into HDR images with peak brightness over 1000 nits, preserving the artistic intentions inherent to the HDR domain. We assessed our results using the full-reference objective quality metrics HDR-VDP-2.2 and DRIM, and carrying out a subjective pair-wise comparison experiment. We compared our results with those obtained with the most recent methods found in the literature. Experimental results demonstrate that our proposed method outperforms the current state-of-the-art of simple inverse tone mapping methods and its performance is similar to other more complex and time-consuming advanced techniques.

Dynamic Range Expansion Using Cumulative Histogram Learning for High Dynamic Range Image Generation

In modern digital photographs, most images have low dynamic range (LDR) formats, which means that the range of light intensities from the darkest to the brightest is much lower than the range that can be perceived by the human eye. Therefore, to visualize images as naturally as possible on devices that display them in high dynamic range (HDR) format, the LDR images need to be converted into HDR images. The aim of this study was to develop an adaptive inverse tone mapping operator (iTMO) that can convert a single LDR image into a realistic HDR image based on artificial neural networks. In contrast to conventional iTMO algorithms, our technique was developed by learning the complicated relationship between various LDR–HDR pair images, which enabled nearly ground-truth HDR images to be generated from various types of LDR images. The novel learning technique is called cumulative histogram-based learning and color difference learning. The superior performance of our technique over conventional methods was assessed through objective evaluations of various types of LDR and HDR images.

Local Inverse Tone Mapping for Scalable High Dynamic Range Image Coding

Tone mapping operators (TMOs) and inverse TMOs (iTMOs) are important for scalable coding of high dynamic range (HDR) images. Because of the high nonlinearity of local TMOs, it is very difficult to estimate the iTMO accurately for a local TMO. In this letter, we present a two-layer local iTMO estimation algorithm using an edge-preserving decomposition technique. The low dynamic range (LDR) image is first linearized and then decomposed into a base layer and a detail layer via a fast edge-preserving decomposition method. The base layer of the HDR image is generated by subtracting the LDR detail layer from the HDR image. An iTMO function is finally estimated by solving a novel quadratic optimization problem formulated on the pair of base layers rather than the pair of HDR and LDR images as in existing methods. Experimental results show that the proposed two-layer iTMO can recover the HDR accurately so that it is possible to use these local TMOs in scalable HDR image coding schemes.

Inverse Tone Mapping Operator Using Sequential Deep Neural Networks Based on the Human Visual System

Conventional digital displays show images with much less dynamic range than that of human visual perception. High-dynamic range (HDR) displays are being developed for viewing images with higher dynamic range than that of conventional low-dynamic range (LDR) images. However, to view existing LDR images on HDR displays, an inverse tone mapping operator (ITMO), which is a process to extend the dynamic range of LDR images, is required. In this paper, we propose an adaptive ITMO by effectively learning the differences between LDR and HDR images using a sequential learning process: dynamic range learning followed by color difference learning. Our proposed method enables visualization of colors similar to real-world colors better than conventional ITMOs by learning color differences based on the human visual system properties. For the objective comparison, seven different evaluation metrics optimized for HDR image evaluation were used. Our method resulted in 10%–25% improved HDR-visual difference predictor-2.2 values over those of other ITMOs. Other metrics also demonstrated the superior performance of our method. For the subjective comparison, eight human observers evaluated the estimated HDR images in terms of color appearance and overall preferences. Our method received an average 4.7 out of 5 score, whereas other ITMOs received below 3.5 scores in the evaluation of color appearance. The objective and subjective evaluations’ results showed that our proposed method outperformed the conventional ITMOs in estimating the dynamic range and color appearance of ground-truth HDR images.

A model of perceived dynamic range for HDR images

For High Dynamic Range (HDR) content, the dynamic range of an image is an important characteristic in algorithm design and validation, analysis of aesthetic attributes and content selection. Traditionally, it has been computed as the ratio between the maximum and minimum pixel luminance, a purely objective measure; however, the human visual system's perception of dynamic range is more complex and has been largely neglected in the literature. In this paper, a new methodology for measuring perceived dynamic range (PDR) of chromatic and achromatic HDR images is proposed. PDR can benefit HDR in a number of ways: for evaluating inverse tone mapping operators and HDR compression methods; aesthetically; or as a parameter for content selection in perceptual studies. A subjective study was conducted on a data set of 36 chromatic and achromatic HDR images. Results showed a strong agreement across participants' allocated scores. In addition, a high correlation between ratings of the chromatic and achromatic stimuli was found. Based on the results from a pilot study, five objective measures (pixel-based dynamic range, image key, area of bright regions, contrast and colorfulness) were selected as candidates for a PDR predictor model; two of which have been found to be significant contributors to the model. Our analyses show that this model performs better than individual metrics for both achromatic and chromatic stimuli.

Rendering of HDR images to improve brightness discrimination

The impression of quality of images can be enhanced on a high dynamic range (HDR) displays. Generally, a conventional 8-bit image can be processed to an HDR image by inverse tone mapping operators. Among the operators, brightness discrimination mapping by applying brightness adaptation model attempted to mimic the human visual system. In this paper, we use a brightness adaptation model to derive a brightness discrimination mapping algorithm for HDR displays. The proposed algorithm maximizes a function, which represents the local and global brightness discrimination range by exploiting characteristics of the human visual system. Enhancement of details is verified by visualizing HDR images from dark to bright regions. Improvement of dynamic range is quantified by measuring increased discrimination ratio.

Evaluating the Color Fidelity of ITMOs and HDR Color Appearance Models

With the increasing availability of high-dynamic-range (HDR) displays comes the need to remaster existing content in a way that takes advantage of the extended range of luminance and contrast that such displays offer. At the same time, it is crucial that the creative intent of the director is preserved through such changes as much as possible. In this article, we compare several approaches for dynamic range extension to assess their ability to correctly reproduce the color appearance of standard dynamic range (SDR) images on HDR displays. A number of state-of-the-art inverse tone mapping operators (ITMOs) combined with a standard chromatic adaptation transform (CAT) as well as some HDR color appearance models have been evaluated through a psychophysical study, making use of an HDR display as well as HDR ground-truth data. We found that global ITMOs lead to the most reliable performance when combined with a standard CAT, while more complex methods were found to be more scene dependent, and often less preferred than the unprocessed SDR image. HDR color appearance models, albeit being the most complete solutions for accurate color reproduction, were found to not be well suited to the problem of dynamic range expansion, suggesting that further research may be necessary to provide accurate color management in the context of inverse tone mapping.

Dynamic range independent image quality assessment

The diversity of display technologies and introduction of high dynamic range imagery introduces the necessity of comparing images of radically different dynamic ranges. Current quality assessment metrics are not suitable for this task, as they assume that both reference and test images have the same dynamic range. Image fidelity measures employed by a majority of current metrics, based on the difference of pixel intensity or contrast values between test and reference images, result in meaningless predictions if this assumption does not hold. We present a novel image quality metric capable of operating on an image pair where both images have arbitrary dynamic ranges. Our metric utilizes a model of the human visual system, and its central idea is a new definition of visible distortion based on the detection and classification of visible changes in the image structure. Our metric is carefully calibrated and its performance is validated through perceptual experiments. We demonstrate possible applications of our metric to the evaluation of direct and inverse tone mapping operators as well as the analysis of the image appearance on displays with various characteristics.

A framework for inverse tone mapping

In recent years many tone mapping operators (TMOs) have been presented in order to display high dynamic range images (HDRI) on typical display devices. TMOs compress the luminance range while trying to maintain contrast. The inverse of tone mapping, inverse tone mapping, expands a low dynamic range image (LDRI) into an HDRI. HDRIs contain a broader range of physical values that can be perceived by the human visual system. We propose a new framework that approximates a solution to this problem. Our framework uses importance sampling of light sources to find the areas considered to be of high luminance and subsequently applies density estimation to generate an expand map in order to extend the range in the high luminance areas using an inverse tone mapping operator. The majority of today’s media is stored in the low dynamic range. Inverse tone mapping operators (iTMOs) could thus potentially revive all of this content for use in high dynamic range display and image based lighting (IBL). Moreover, we show another application that benefits quick capture of HDRIs for use in IBL.