Tone-mapped Images Research Articles

Temporally Consistent Tone Mapping of Images and Video Using Optimal K-means Clustering

The field of high dynamic range imaging addresses the problem of capturing and displaying the large range of luminance levels found in the world, using devices with limited dynamic range. In this paper we present a novel tone mapping algorithm that is based on K-means clustering. Using dynamic programming we are able to not only solve the clustering problem efficiently, but also find the global optimum. Our algorithm runs in hbox {O}(N^2K) for an image with N input luminance levels and K output levels. We show that our algorithm gives comparable results to state-of-the-art tone mapping algorithms, but with the additional large benefit of a minimum of parameters. We show how to extend the method to handle video input. We test our algorithm on a number of standard high dynamic range images and video sequences and give qualitative and quantitative comparisons to a number of state-of-the-art tone mapping algorithms.

Tone mapping with contrast preservation and lightness correction in high dynamic range imaging

In real-world environments, the human visual system perceives a wide range of luminance in a scene. However, the full range of tones in a high dynamic range (HDR) scene cannot be displayed on standard display devices due to their low dynamic range (LDR). Therefore, tone mapping is necessary to faithfully display a HDR scene on an LDR display device. Existing tone mapping methods have problems because details and contrast in a scene are not preserved faithfully, and they also distort the colors in a scene. Thus, we propose a tone mapping method for preserving the detail in an HDR scene using a weighted least squares filter, which preserves the global contrast in a scene by using a competitive learning neural network, before applying a tone reproduction operator to preserve the color without shifting the lightness. According to the Helmholtz–Kohlrausch effect, the perception of brightness depends on the lightness, chroma, and hue of a color. For example, the luminance of pixels with specific colors such as red and blue is low in an HDR scene. The proposed method corrects the lightness of pixels according to the color (i.e., lightness, chroma, and hue) of a tone-mapped image. Experimental results with several test sets of images demonstrated that the proposed tone mapping method with contrast preservation and lightness correction is more suitable for dynamic range compression than existing tone mapping methods, while it also preserves the color of a scene in an effective way.

Adaptive contrast enhancement using edge-based lighting condition estimation

This paper proposes a new approach to image contrast enhancement that improves the perceptual visual quality by considering the lighting condition and minimizing the structural distortion to a tolerable level. The proposed method consists of the following two major steps: lighting condition estimation and contrast enhancement processes. In the first step, the proposed method estimates the lighting condition by calculating the dynamic range along the edges of the image. In the second step, the method adaptively adjusts the luminance by considering both the estimated lighting condition and the order of luminance levels in order to improve the perceptual visual quality. In addition, the method properly reduces the structural distortion. Experimental results show that the proposed method improved the perceptual visual quality of various images by increasing the average structural fidelity, enhancement performance measure, entropy, and tone-mapped image quality index by up to 11%, 133%, 16%, and 11%, respectively, compared to the benchmark methods.

A Tone Mapping Algorithm Based on Multi-scale Decomposition

A new Tone Mapping( TM) algorithm based on multi-scale decomposition was proposed to solve a High Dynamic Range( HDR) image displayed on an ordinary display device. The algorithm decomposed a HDR image into multiple scales using a Local Edge-Preserving( LEP) filter to smooth the details of the image effectively, while still retaining the salient edges. Then a dynamic range compression function with parameters was proposed according to the characteristics of the decomposed layers and the request of compression. By changing the parameters, the coarse scale layer was compressed and the fine scale layer was boosted, which resulted in compressing the dynamic range of the image and boosting the details. Finally, by restructuring the image and restoring the color, the image after mapping had a good visual quality. The experimental results demonstrate that the proposed method is better than the algorithm proposed by Gu et al.( GU B, LI W J, ZHU M Y, et al. Local edge-preserving multiscale decomposition for high dynamic range image tone mapping [J]. IEEE Transactions on Image Processing, 2013, 22( 1) : 70- 79) and Yeganeh et al.( YEGANEH H, WANG Z. Objective quality assessment of tonemapped images [J]. IEEE Transactions on Image Processing, 2013, 22( 2) : 657- 667) in naturalness, structural fidelity and quality assessment; moreover, it avoids the halo artifacts which is a common problem existing in the local tone mapping algorithms. The algorithm can be used for the tone mapping of the HDR image.

Blind Quality Assessment of Tone-Mapped Images Via Analysis of Information, Naturalness, and Structure

High dynamic range (HDR) imaging techniques have been working constantly, actively, and validly in the fault detection and disease diagnosis in the astronomical and medical fields, and currently they have also gained much more attention from digital image processing and computer vision communities. While HDR imaging devices are starting to have friendly prices, HDR display devices are still out of reach of typical consumers. Due to the limited availability of HDR display devices, in most cases tone mapping operators (TMOs) are used to convert HDR images to standard low dynamic range (LDR) images for visualization. But existing TMOs cannot work effectively for all kinds of HDR images, with their performance largely depending on brightness, contrast, and structure properties of a scene. To accurately measure and compare the performance of distinct TMOs, in this paper develop an effective and efficient no-reference objective quality metric which can automatically assess LDR images created by different TMOs without access to the original HDR images. Our model is shown to be statistically superior to recent full- and no-reference quality measures on the existing tone-mapped image database and a new relevant database built in this work.

Evaluation of feature point detection in high dynamic range imagery

This paper evaluates the suitability of High Dynamic Range (HDR) imaging techniques for Feature Point (FP) detection under demanding lighting conditions. The FPs are evaluated in HDR, tone mapped HDR, and traditional Low Dynamic Range (LDR) images. Eleven global and local tone mapping operators are evaluated and six widely used FP detectors are used in the experiments (Harris, Shi–Tomasi, DoG, Fast Hessian, FAST, and BRISK). The distribution and repeatability rate of FPs are studied under changes of camera viewpoint, camera distance, and scene lighting. The results of the experiments show that current FP detectors cannot cope with HDR images well. The best contemporary solution is thus tone mapping of HDR images using a local tone mapper as a pre-processing step.

Image smoothing with generalized random walks: Algorithm and applications

A novel generalized random walks model based algorithm for image smoothing is presented. Unlike previous image smoothing methods, the proposed method performs image smoothing in a global weighted way based on graph notation, which can preserve important features and edges as much as possible. Based on the new random walks model, input image information and user defined smoothing scale information are projected to a graph, our method calculates the probability that a random walker starting at each pixel node position will first reach one of the pre-defined terminal node to achieve image smoothing, which goes to solving a system of linear equations, the system can be solved efficiently by lots of methods. Theoretical analysis and experimental results are reported to illustrate the usefulness and potential applicability of our algorithm on various computer vision fields, including image enhancement, edge detection, image decomposition, high dynamic range (HDR) image tone mapping and other applications.

Stream Processor for Real-Time Inverse Tone Mapping of Full-HD Images

In this paper, an architecture design of a hardware accelerator capable to expand the dynamic range of low dynamic range images to the 32-bit high dynamic range counterpart is presented. The processor implements on-the-fly calculation of the edge-preserving bilateral filtering and luminance average, to elaborate a full-HD (1920 $ \times $ 1080 pixels) image in 16.6 ms (60 frames/s) on field-programmable logic (FPL), by processing the incoming pixels in streaming order, without frame buffers. In this way, the design avoids the use of external DRAM and can be tightly coupled with acquiring devices, thus to enable the implementation of smart sensors. The processor complexity can be configured with different area/speed ratios to meet the requirements of different target platforms from FPLs to ASICs, obtaining, in both implementations, state-of-the-art performances.

Robust Matching Cost Function for Stereo Correspondence Using Matching by Tone Mapping and Adaptive Orthogonal Integral Image.

Real-world stereo images are inevitably affected by radiometric differences, including variations in exposure, vignetting, lighting, and noise. Stereo images with severe radiometric distortion can have large radiometric differences and include locally nonlinear changes. In this paper, we first introduce an adaptive orthogonal integral image, which is an improved version of an orthogonal integral image. After that, based on matching by tone mapping and the adaptive orthogonal integral image, we propose a robust and accurate matching cost function that can tolerate locally nonlinear intensity distortion. By using the adaptive orthogonal integral image, the proposed matching cost function can adaptively construct different support regions of arbitrary shapes and sizes for different pixels in the reference image, so it can operate robustly within object boundaries. Furthermore, we develop techniques to automatically estimate the values of the parameters of our proposed function. We conduct experiments using the proposed matching cost function and compare it with functions employing the census transform, supporting local binary pattern, and adaptive normalized cross correlation, as well as a mutual information-based matching cost function using different stereo data sets. By using the adaptive orthogonal integral image, the proposed matching cost function reduces the error from 21.51% to 15.73% in the Middlebury data set, and from 15.9% to 10.85% in the Kitti data set, as compared with using the orthogonal integral image. The experimental results indicate that the proposed matching cost function is superior to the state-of-the-art matching cost functions under radiometric variation.

Modeling a color-rendering operator for high dynamic range images using a cone-response function

Tone-mapping operators are the typical algorithms designed to produce visibility and the overall impression of brightness, contrast, and color of high dynamic range (HDR) images on low dynamic range (LDR) display devices. Although several new tone-mapping operators have been proposed in recent years, the results of these operators have not matched those of the psychophysical experiments based on the human visual system. A color-rendering model that is a combination of tone-mapping and cone-response functions using an XYZ tristimulus color space is presented. In the proposed method, the tone-mapping operator produces visibility and the overall impression of brightness, contrast, and color in HDR images when mapped onto relatively LDR devices. The tone-mapping resultant image is obtained using chromatic and achromatic colors to avoid well-known color distortions shown in the conventional methods. The resulting image is then processed with a cone-response function wherein emphasis is placed on human visual perception (HVP). The proposed method covers the mismatch between the actual scene and the rendered image based on HVP. The experimental results show that the proposed method yields an improved color-rendering performance compared to conventional methods.

English

English

Gradient Domain Guided Image Filtering.

Guided image filter (GIF) is a well-known local filter for its edge-preserving property and low computational complexity. Unfortunately, the GIF may suffer from halo artifacts, because the local linear model used in the GIF cannot represent the image well near some edges. In this paper, a gradient domain GIF is proposed by incorporating an explicit first-order edge-aware constraint. The edge-aware constraint makes edges be preserved better. To illustrate the efficiency of the proposed filter, the proposed gradient domain GIF is applied for single-image detail enhancement, tone mapping of high dynamic range images and image saliency detection. Both theoretical analysis and experimental results prove that the proposed gradient domain GIF can produce better resultant images, especially near the edges, where halos appear in the original GIF.

FSITM: A Feature Similarity Index For Tone-Mapped Images

In this work, based on the local phase information of images, an objective index, called the feature similarity index for tone-mapped images (FSITM), is proposed. To evaluate a tone mapping operator (TMO), the proposed index compares the locally weighted mean phase angle map of an original high dynamic range (HDR) to that of its associated tone-mapped image calculated using the output of the TMO method. In experiments on two standard databases, it is shown that the proposed FSITM method outperforms the state-of-the-art index, the tone mapped quality index (TMQI). In addition, a higher performance is obtained by combining the FSITM and TMQI indices. The MATLAB source code of the proposed metric(s) is available at https://www.mathworks.com/matlabcentral/fileexchange/59814.

Visual-Salience-Based Tone Mapping for High Dynamic Range Images

Visual saliency aims to predict the attentional gaze of observers viewing a scene, and it is thus highly demanded for tone mapping of high dynamic range (HDR) images. In this paper, novel saliency-aware weighting and edge-aware weighting are introduced for HDR images. They are incorporated into an existing guided image filter to form a perceptually guided image filter. The saliency-aware weighting and the proposed filter are applied to design a new local tone-mapping algorithm for HDR images such that both extreme light and shadow regions can be reproduced on conventional low dynamic range displays. In particular, the proposed filter is applied to decompose the luminance of the input HDR image into a base layer and a detail layer. The saliency-aware weighting is then adopted to design a saliency-aware global tone mapping for the compression of the base layer. The proposed filter preserves sharp edges in the base layer better than the existing guided filter. Halo artifacts are thus significantly reduced in the tone-mapped image. Moreover, the visual quality of the tone-mapped image, especially attention-salient regions, is improved by the saliency-aware weighting.

HDR image encoding using reconstruction functions based on piecewise linear approximations

Two-layer encoding schemes for HDR images can not only reduce the storage requirements, but more importantly they can also ensure backward compatibility as imaging technology makes the transition from LDR to HDR. In this paper, we present a two-layer lossy encoding scheme for HDR images, which models each channel of the HDR image as a piecewise linear function of its tone-mapped version, to reduce the dynamic range of the residual image and achieve a better compression. The tone-mapped image and the residual image for each channel are saved as two separate LDR images, and these along with the piecewise linear models, encode the details of the HDR image. The encoded images are compatible to both HDR and non-HDR enabled devices for visualization and processing. Detailed comparison with similar existing state of the art two-layer techniques is presented to show the effectiveness of our proposed scheme.

English

2 ABSTRACT: This paper deals with the assessment of measuring the quality of tone mapped images by considering the Tone Mapping Operators (TMOs) that convert high dynamic range (HDR) to low dynamic range (LDR) images provide practically useful tools for the visualization of HDR images on standard LDR displays. Different TMOs create different tone mapped images, and a natural question is which one has the best quality. Without an appropriate quality measure, different TMOs cannot be compared, and further improvement is directionless. Subjective rating may be a reliable evaluation method, but it is expensive and time consuming, and more importantly, is difficult to be embedded into optimization frameworks. Here we propose an objective quality assessment algorithm for tone mapped images by combining two concepts one of them is a multi scale signal fidelity measure on the basis of a modified structural similarity index and the other followed by a naturalness measure on the basis of intensity statistics of natural images. Validations using independent subject-rated image databases show good correlations between subjective ranking score and the proposed tone-mapped image quality index (TMQI). The proposed measure not only provides an overall quality score of an image, but also creates multi-scale quality maps that reflect the structural fidelity variations across scale and space.

Tone-mapping and dynamic range compression using dynamic cone response

Over the last several decades, the tone-mapping and color correction methods have been incorporated into the display device such as LCD/LED display device or the photographic print. However, conventional methods are based on Gray-World Assumption using spatial smoothing filter to estimate the local illumination component. For this reason, most of these methods have several problems like visual perception problem, scale the contrast, gray-out, color distortion (dominate color, halo-artifact) and so on. Accordingly, this paper presents a color correction method using the dynamic cone response function based on human visual perception. The proposed method consists of tone-mapping and dynamic cone response. The tone-mapped image is obtained by using chromatic and achromatic colors. Thereafter, the resultant image is processed through the dynamic cone response function, and it is to estimate the dynamic response of human eye as well as to deal with mismatch between a real scene and the rendered image. The experimental results show that the proposed method yields better performance of color correction over the conventional methods.

Hardware Implementation of an Automatic Rendering Tone Mapping Algorithm for a Wide Dynamic Range Display

Tone mapping algorithms are used to adapt captured wide dynamic range (WDR) scenes to the limited dynamic range of available display devices. Although there are several tone mapping algorithms available, most of them require manual tuning of their rendering parameters. In addition, the high complexities of some of these algorithms make it difficult to implement efficient real-time hardware systems. In this work, a real-time hardware implementation of an exponent-based tone mapping algorithm is presented. The algorithm performs a mixture of both global and local compression on colored WDR images. An automatic parameter selector has been proposed for the tone mapping algorithm in order to achieve good tone-mapped images without manual reconfiguration of the algorithm for each WDR image. Both algorithms are described in Verilog and synthesized for a field programmable gate array (FPGA). The hardware architecture employs a combination of parallelism and system pipelining, so as to achieve a high performance in power consumption, hardware resources usage and processing speed. Results show that the hardware architecture produces images of good visual quality that can be compared to software-based tone mapping algorithms. High peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) scores were obtained when the results were compared with output images obtained from software simulations using MATLAB.

A study of hardware-friendly methods for gradient domain tone mapping of high dynamic range images

A large class of techniques for image processing is based on manipulation of the gradient field of an image. These techniques have high computational complexity, due to the need to solve an inverse problem, taking the form of a Poisson equation, to find the output image that best matches a manipulated gradient field. This work studies hardware-friendly techniques, appropriate for implementation on a field-programmable gate array implementation and embeddable inside a camera, for approximating this solution. Fattal's operator for the dynamic range compression of high dynamic range images is studied as a representative example application. A family of methods, inspired by Fattal's operator but with significantly lower computational complexity, solves the inverse problem in a moving window of small size. In this paper, a study is conducted to understand the role that the boundary conditions and the size of the window play in the quality of the resulting output image and the size of the hardware. The impact of using single-scale and multi-scale approaches to compute the attenuation factors needed for Fattal's operator is also considered.

An Improved Method Based on Fast Bilateral Filter for High Dynamic Image Rendering

This paper presents an improved high dynamic range image tone mapping method based on fast bilateral filtering. The algorithm first applied a bilateral filtering to the luminance channel of the image, the image is decomposed into an HDR base layer and an LDR detail layer. Then the HDR base layer is blurred with bilateral filtering again, get the details portion of the base layer, at the same time the dynamic range of the global base layer is compressed. Finally, the detail component and the compressed HDR base layer are recombined and the result is tone-mapped image for displaying. For color image, the color restoration converts luminance value into RGB color. Experimental results show that the proposed technique performed better than the conventional bilateral filtering, preserving more details and enhancing local contrast, giving decent visual effect and avoiding additional artifacts.