Tone Mapping Function Research Articles

Tone Mapping Method Based on the Least Squares Method

Tone mapping is used to compress the dynamic range of image data without distortion. To compress the dynamic range of HDR images and prevent halo artifacts, a tone mapping method is proposed based on the least squares method. Our method first uses weights for the estimation of the illumination, and the image detail layer is obtained by the Retinex model. Then, a global tone mapping function with the parameter is used to compress the dynamic range, and the parameter is obtained by fitting the function to the histogram equalization. Finally, the detail layer and the illumination layer are fused to obtain the LDR image. The experimental results show that the proposed method can efficiently restore real-world scene information while preventing halo artifacts. Therefore, tone mapping quality index and mean Weber contrast of the tone-mapped image are 8% and 12% higher than the closest competition tone mapping method.

TM-Net: A Neural Net Architecture for Tone Mapping.

Tone mapping functions are applied to images to compress the dynamic range of an image, to make image details more conspicuous, and most importantly, to produce a pleasing reproduction. Contrast Limited Histogram Equalization (CLHE) is one of the simplest and most widely deployed tone mapping algorithms. CLHE works by iteratively refining an input histogram (to meet certain conditions) until convergence, then the cumulative histogram of the result is used to define the tone map that is used to enhance the image. This paper makes three contributions. First, we show that CLHE can be exactly formulated as a deep tone mapping neural network (which we call the TM-Net). The TM-Net has as many layers as there are refinements in CLHE (i.e., 60+ layers since CLHE can take up to 60 refinements to converge). Second, we show that we can train a fixed 2-layer TM-Net to compute CLHE, thereby making CLHE up to 30× faster to compute. Thirdly, we take a more complex tone-mapper (that uses quadratic programming) and show that it too can also be implemented - without loss of visual accuracy-using a bespoke trained 2-layer TM-Net. Experiments on a large corpus of 40,000+ images validate our methods.

Texture Management for Glossy Objects Using Tone Mapping.

In this paper, we proposed a method for matching the color and glossiness of an object between different displays by using tone mapping. Since displays have their own characteristics, such as maximum luminance and gamma characteristics, the color and glossiness of an object when displayed differs from one display to another. The color can be corrected by conventional color matching methods, but the glossiness, which greatly changes the impression of an object, needs to be corrected. Our practical challenge was to use tone mapping to correct the high-luminance part, also referred to as the glossy part, which cannot be fully corrected by color matching. Therefore, we performed color matching and tone mapping using high dynamic range images, which can record a wider range of luminance information as input. In addition, we varied the parameters of the tone-mapping function and the threshold at which the function was applied to study the effect on the object’s appearance. We conducted a subjective evaluation experiment using the series category method on glossy-corrected images generated by applying various functions to each display. As a result, we found that the differences in glossiness between displays could be corrected by selecting the optimal function for each display.

Use of Local Region Maps on Convolutional LSTM for Single-Image HDR Reconstruction

Low dynamic range (LDR) images captured by consumer cameras have a limited luminance range. As the conventional method for generating high dynamic range (HDR) images involves merging multiple-exposure LDR images of the same scene (assuming a stationary scene), we introduce a learning-based model for single-image HDR reconstruction. An input LDR image is sequentially segmented into the local region maps based on the cumulative histogram of the input brightness distribution. Using the local region maps, SParam-Net estimates the parameters of an inverse tone mapping function to generate a pseudo-HDR image. We process the segmented region maps as the input sequences on long short-term memory. Finally, a fast super-resolution convolutional neural network is used for HDR image reconstruction. The proposed method was trained and tested on datasets including HDR-Real, LDR-HDR-pair, and HDR-Eye. The experimental results revealed that HDR images can be generated more reliably than using contemporary end-to-end approaches.

Unified implementation of global high dynamic range image tone-mapping algorithms.

High dynamic range (HDR) images and video require tone-mapping for display on low dynamic range (LDR) screens. Many tone-mapping operators have been proposed to convert HDR content to LDR, but almost each has a different implementation structure and requires a different execution time. We propose a unified structure that can represent any global tone-mapping algorithm with an array of just 256 coefficients. These coefficients extracted offline for every HDR image or video frame can be used to convert them to LDR in real time using linear interpolation. The produced LDR images are identical to the images produced by the original implementation of the algorithm. This unified implementation can replicate any global tone-mapping function and requires very low and fixed execution time, which is independent of algorithm and type of content and depends only on image size. Experimental studies are presented to show the accuracy and time efficiency of the proposed implementation.

Real-Time Deep Image Retouching Based on Learnt Semantics Dependent Global Transforms.

Although artists' actions in photo retouching appear to be highly nonlinear in nature and very difficult to characterize analytically, we find that the net effects of interactively editing a mundane image to a desired appearance can be modeled, in most cases, by a parametric monotonically non-decreasing global tone mapping function in the luminance axis and by a global affine transform in the chrominance plane that are weighted by saliency. This allows us to simplify the machine learning problem of mimicking artists in photo retouching to constructing a deep artful image transform (DAIT) using convolutional neural networks (CNN). The CNN design of DAIT aims to learn the image-dependent parameters of the luminance tone mapping function and the affine chrominance transform, rather than learning the end-to-end pixel level mapping as in the mainstream methods of image restoration and enhancement. The proposed DAIT approach reduces the computation complexity of the neural network by two orders of magnitude, which also, as a side benefit, improves the robustness and generalization capability at the inference stage. The high throughput and robustness of DAIT lend itself readily to real-time video enhancement as well after a simple temporal processing. Experiments and a Turing-type test are conducted to evaluate the proposed method and its competitors.

Backlit images enhancement using global tone mappings and image fusion

The authors present a method for the enhancement of backlit images, i.e. images in which the main source of light is behind the photography subject. These images contain, simultaneously, very dark and very bright regions. In this situation, a single tone mapping function is unable to enhance the whole image. They propose the use of several such tone mappings, some of them enhancing the dark regions while others enhancing the bright regions, and then the combination of all these results using an image fusion algorithm. Qualitative and quantitative results confirm the validity of the proposed method.

Differential Evolution-based Approach for Tone-Mapping of High Dynamic Range Images

Recently, high dynamic range (HDR) imaging has received significant attention from research community as well as the industrial companies due to valuable applications of HDR images in better visualization and analysis. However, HDR images need to be converted to low dynamic range (LDR) images for viewing on standard LDR display screens. Several tone-mapping operators have been proposed for the conversion, however, so far, no significant works have been reported employing artificial intelligence to achieve better enhancement of the output images. In this paper, we present an optimization-based approach, to enhance the quality of the tone-mapped LDR images using metaheuristics. More specifically, the optimization process is based on the differential evolution (DE) algorithm which takes tone-mapping function of an existing histogram-based method as initial guess and refines the histogram bins iteratively leading to progressive enhancement of the quality of LDR image. The final results produced by the proposed optimized histogram-based approach (OHbA) showed better performance compared to the existing state-of-the-art tone-mapping algorithms.

Optimized Tone Mapping Function for Contrast Enhancement Considering Human Visual Perception System

Conventional contrast enhancement methods, including global and local enhancements, produce enhanced images with some limitations. Global contrast enhancement does not take the local characteristics into consideration, and therefore, the enhancement performance could be limited. On the other hand, a local contrast enhancement method achieves a noticeable improvement, but it generates unnatural improvement results compared with the input image. Due to the complementary characteristics of these two methods, it is hard to achieve remarkable contrast enhancement without visual artifacts. To overcome the limitations, we propose a new tone mapping function for contrast enhancement using an optimization approach that is subject to constraints such as the output image needing to be enhanced naturally and noticeably. Since contrast enhancement without artificiality is possible when the enhancement process mimics the human eye, we model the human visual perception system, and then, the model is incorporated into the proposed tone mapping function. Consequently, the contrast of the image is adaptively enhanced according to a region that is more attractive to a person. The experimental results demonstrate that the proposed algorithm outperforms other contrast enhancement methods in terms of both objective and subjective criteria.

An Adaptive Global and Local Tone Mapping Algorithm Implemented on FPGA

We present a fast global and locally adaptive tone mapping algorithm and its field-programmable gate array (FPGA) implementation. The specially designed tone mapping function, which is based on local histogram equalization, controls global, and local characteristics individually. In contrast to other tonemap operators, our algorithm manages light/dark halos separately and by using local tonemap function alone, it can effectively suppress noise. We validated the effectiveness of our algorithms using subjective and objective assessment. Using an average of the bins, we achieve fast smoothed local histogram estimation with fewer bins while maintaining high accuracy. Our new implementation method requires minimal data access and reduced memory as it operates with a downscaled frame size of 240 × 135 pixels. Relative local area size is 248 × 248 @Full-HD resolution (1920 × 1080). For low-latency pixel output, the system performs the tone mapping using pixel information from the previous frame. When we implemented the system on FPGA (TB-7K-325TIMG and Xilinx Kintex-7), we achieved lightweight hardware as the total usage rate is about 25% of the available FPGA resource. Using an online 1080p video we demonstrate, a real-time video processing using our hardware tone mapping system.

An optimization framework for inverse tone mapping using a single low dynamic range image

Conventional inverse tone-mapping (ITM) methods tend to produce contrast distortions such as contrast loss and contrast reversal in reconstructed high dynamic range (HDR) images. This paper proposes a novel ITM optimization framework based on the assumption that the input low dynamic range (LDR) image is similar to the LDR image obtained by tone mapping a true HDR image. In the proposed framework, an HDR image is initially reconstructed by applying a conventional tone-mapping function in a reverse manner, and then the reconstructed HDR image is iteratively modified toward the optimum HDR image by minimizing the difference between the input LDR image and a tone-mapped LDR image obtained from the reconstructed HDR image. The experimental results demonstrate that the proposed framework effectively reconstructs a high-quality HDR image and outperforms other conventional methods in terms of objective quality.

Learning-Based Tone Mapping Operator for Efficient Image Matching

In this paper, we propose a new framework to optimally tone map the high dynamic range (HDR) content for image matching under drastic illumination variations. Since tone mapping operators (TMO) have traditionally been used for displaying HDR scenes, their design is suboptimal when used for computer vision tasks, such as image matching. We address this suboptimality by proposing a two-step framework, consisting of: first, a luminance-invariant guidance model based on a support vector regressor (SVR) to optimally adapt the tone mapping function for image matching; and second, an energy maximization model to generate appropriate training samples for learning the SVR. At each step, we collectively address both stages of keypoint detection and descriptor extraction in the feature matching framework. By locally altering the intrinsic characteristics of the tone mapping function, the learned guidance model facilitates the extraction of local invariant features in the presence of illumination variations. We demonstrate that the proposed TMO significantly outperforms perceptually driven state-of-the-art TMOs on a dataset of HDR scenes characterized by challenging lighting variations, such as day/night transitions.

Illumination consistency based on single low dynamic range images

In this paper, a light consistency solution for generating high dynamic range (HDR) images based on a single low dynamic range image(LDR) is proposed, and the virtual object is rendered by illumination. The solution can reduce the time of image acquisition and processing, and solve the problems caused by the limitations of image acquisition equipment. The solution is divided into three stages: image preprocessing, high dynamic range image generation and virtual object relighting. Firstly, in the stage of image pretreatment, the wavelet noise reduction method based on a Gaussian mixture model is used to remove image noise and avoid image detail distortion. The inverse camera response function is utilized to linearize the image, the pixel brightness range is expanded based on the inverse tone mapping function, and the threshold segmentation method is combined with flooding Gaussian smoothing to calculate the highlight spread diagram to compensate for scene highlights lost during camera shooting. Then, the extended dynamic range image is interpolated linearly by using the specular expansion image to get the high dynamic range image. Based on the analysis and experimental simulation, compared with other methods, it is found that using a single low-dynamic-range image can greatly reduce the time of image acquisition and processing and reduce the limitations of image acquisition equipment, while maintaining good light fusion. Based on the simulation results, the efficiency of light consistency processing is improved.

High dynamic range image tone mapping based on asymmetric model of retinal adaptation

Global tone mapping operators using the symmetrical retinal response model to light tend to produce a low dynamic range (LDR) image that exhibits loss of details of its corresponding high dynamic range (HDR) image in a bright or dark area. In this paper, we introduce a new asymmetric sigmoid curve (ASC) based on the model of retinal adaptation encompassing symmetrical S-shaped curve, and present two global tone mapping operators by using the ASC. In the proposed method, an ASC-based tone mapping function is obtained by using a well-known classic photography technique, called zone system. In addition, a contrast-enhancing tone mapping function is introduced by formulating a bi-criteria optimization problem with the luminance histogram of an input HDR image and the ASC-based mapping function. Experimental results demonstrate that the proposed method enhances the global contrast while preserving image details in the tone-mapped LDR image. Moreover, the objective assessment results using an image quality metric indicate that the proposed method shows a high performance to state-of-the-art global tone mapping operators.

Content Aware Image Pre-Compensation.

The goal of image pre-compensation is to process an image such that after being convolved with a known kernel, will appear close to the sharp reference image. In a practical setting, the pre-compensated image has significantly higher dynamic range than the latent image. As a result, some form of tone mapping is needed. In this paper, we show how global tone mapping functions affect contrast and ringing in image pre-compensation. We further enhance contrast and reduce ringing by considering the visual saliency. Specifically, we prioritize contrast preservation in salient regions while tolerating more blurriness elsewhere. For quantitative analysis, we design new metrics to measure the contrast of an image with ringing. Specifically, we set out to find its "equivalent ringing-free" image that matches its intensity histogram and uses its contrast as the measure. We illustrate our approach on projector defocus compensation and visual acuity enhancement. Compared with the state-of-the-art, our approach significantly improves the contrast. We also perform user studies to demonstrate that our method can effectively improve the viewing experience for users with impaired vision.

Learning-Based Restoration of Backlit Images

Backlighting is a commonly encountered ill illumination condition that can cause serious degradation of image quality. In this paper, we propose a learning-based spatially adaptive technique of optimal tone mapping to restore backlit images. Object surfaces illuminated from behind in a scene are detected by a soft binary classifier that is constructed via supervised learning. Two optimal tone mapping functions, one for backlit regions and the other for the remainder of the image, are used and their outputs are fused to restore illegible surface details in backlit regions and at the same time improve contrast in overexposed regions, if any. Experimental results demonstrate the superior performance of the proposed new technique over existing image enhancement techniques on backlit photographs.

Learning Tone Mapping Function for Dehazing

Learning Tone Mapping Function for Dehazing

Simulating Visual Contrast Reduction during Nighttime Glare Situations on Conventional Displays

Bright glare in nighttime situations strongly decreases human contrast perception. Nighttime simulations therefore require a way to realistically depict contrast perception of the user. Due to the limited luminance of popular as well as specialized high-dynamic range displays, physical adaptation of the human eye cannot yet be replicated in a physically correct manner in a simulation environment. To overcome this limitation, we propose a method to emulate the adaptation in nighttime glare situations using a perception-based model. We implemented a postprocessing tone mapping algorithm that simulates the corresponding contrast reduction effect for a night-driving simulation with glares from oncoming vehicles headlights. During glare, tone mapping reduces image contrast in accordance with the incident veiling luminance. As the glare expires, the contrast starts to normalize smoothly over time. The conversion of glare parameters and elapsed time into image contrast during the readaptation phase is based on extensive user studies carried out first in a controlled laboratory setup. Additional user studies have then been conducted in field tests to ensure validity of the derived time-dependent tone-mapping function and to verify transferability onto real-world traffic scenarios.

인간 시각 인지에 기반을 둔 높은 동적폭을 갖는 영상 보정 방법

For last several decades, the color correction methods have been proposed for HDR(high dynamic range) images. However, color distortion problems take place after correcting the colors such as halos, dominant color as well known. Accordingly, this article presents a novel approach in which the method consists of tone-mapping method and cone response function. In the proposed method, the tone mapping method is used to improve the contrast in the given HDR image based on chromatic and achromatic based on the CIEXYZ tristimulus value, expressed in c/m. The cone response function is used to deal with mismatch between corrected image and displayed image as well as to estimate various human visual effects based on the CMCAT2000 color appearance model. The experimental results show that the proposed method yields better performance of color correction over the conventional method in subjective and quantitative quality, and color reproduction.

Modeling a color-rendering operator for high dynamic range images using a cone-response function

Tone-mapping operators are the typical algorithms designed to produce visibility and the overall impression of brightness, contrast, and color of high dynamic range (HDR) images on low dynamic range (LDR) display devices. Although several new tone-mapping operators have been proposed in recent years, the results of these operators have not matched those of the psychophysical experiments based on the human visual system. A color-rendering model that is a combination of tone-mapping and cone-response functions using an XYZ tristimulus color space is presented. In the proposed method, the tone-mapping operator produces visibility and the overall impression of brightness, contrast, and color in HDR images when mapped onto relatively LDR devices. The tone-mapping resultant image is obtained using chromatic and achromatic colors to avoid well-known color distortions shown in the conventional methods. The resulting image is then processed with a cone-response function wherein emphasis is placed on human visual perception (HVP). The proposed method covers the mismatch between the actual scene and the rendered image based on HVP. The experimental results show that the proposed method yields an improved color-rendering performance compared to conventional methods.