Tone Mapping Research Articles

On Wide Dynamic Range Tone Mapping CMOS Image Sensor

The dynamic range of a natural scene often covers over 6 decades of intensity from bright to dark areas. Typical image sensors, however, have limited ability to capture this dynamic range available in nature. Even after designing specific wide dynamic range (WDR) image sensors, displaying them on conventional media with limited ability requires computationally complex tone mapping.This paper proposed a novel CMOS pixel which can capture and perform tone mapping during data acquisition. The pixel requires a reference voltage to generate tone mapped response. A number of different reference signals are proposed and generated which can perform WDR operation. Nevertheless, fixed pattern noise (FPN) effects the performance of these pixel. A pixel model with simple parameter extraction procedure is described for a typical tone mapping operator. This model is then used to obtain a simple procedure for pixel calibration leading to reduced FPN. The new proposed pixel response is able to capture upto 6 decades of light intensity and reported FPN correction procedure produces 1% of FPN contrast error.

Comparison between Digital Tone-Mapping Operators and a Focal-Plane Pixel-Parallel Circuit

In a previous work, we proposed a color extension of an existing focal-plane tone-mapping operator (FPTMO) and introduced circuit modifications that led to smaller sensing array area and simpler off-chip white-balance operations. The proposed FPTMO chip was fabricated and its experimental test setup is under development. In the present work, we systematically compare the previously proposed FPTMOs with conventional digital tone-mapping operators (DTMOs), both in terms of overall image quality and execution time. To assess the image quality, we use the Tone-Mapped image Quality Index (TMQI) metric, the Blind Tone-Mapped Quality Index (BTMQI) metric, and an image colorfulness metric. By statistically comparing the results given by both metrics, we verify that the proposed FPTMOs achieve results similar to those obtained with DTMOs, occasionally outperforming them. To compare the tone-mapping operators execution times, we perform a detailed complexity analysis. Results show that system-level software FPTMO descriptions (FPTMO functionality described in software) rank among the fastest DTMOs and that the hardware FPTMO (described in software) speed-up depends on the target frame rate. For 30-frames-per-second color image generation, the fastest FPTMO is 15 times faster than the DTMO with highest TMQI and BTMQI metric scores, and 170 times faster than the DTMO with best colorfulness metric score.

Image enhancement with lightness correction and image sharpening based on characteristics of vision for elderly persons

In general, as people age the functions of their sensory organs deteriorate. For example, in the eye of an elderly person, the visual field becomes dark owing to a decrease in the amount of light reaching the retina. Moreover, objects appear blurred due to a decrease in contrast sensitivity. In particular, the contrast sensitivity decreases remarkably as the spatial frequency increases. In this study, we propose an image enhancement method that considers the visual characteristics of elderly persons. The proposed method combines lightness correction and amplification of high spatial frequency components to obtain images with optimized visibility for elderly persons. In particular, lightness enhancement that guarantees color gamut via tone mapping is conducted. In addition, image sharpening is applied based on the inverse characteristic function of the contrast sensitivity curve. The experiments were performed using several digital color images to confirm the effectiveness of the proposed method.

HDR image reproduction based on visual achromatic response

High-dynamic range image reproduction using tone-mapping algorithms are used for image processing to reduce the dynamic range of a real scene to be displayed on low-dynamic range devices. General tone-mapping methods are associated with the determination of local parameters for tone compression and the halo artifact connected to detail enhancements. This paper proposes a lightness-preserved tone-mapping method and a halo-artifact reduction method based on the property of visual achromatic responses. For the reproduction of the visual perception of the real scene on displays, the sensitivity parameter of the sigmoidal transfer curves is optimized for the lightness-preserved tone mapping, and anti-halo image decomposition method is applied for the halo-artifact reduction. In results, it is confirmed that the proposed method improves the performance of the tone-mapping method effectively.

Hierarchical morphological graph signal multi‐layer decomposition for editing applications

We address the problem of editing signals such as 2D color images or 3D colored meshes that are represented under the general framework of graph signals. As state-of-the-art editing approaches decompose an image into several layers in order to manipulate them, we propose a hierarchical multi-layer decomposition of graph signals that relies on morphological filtering. Since morphological filtering operators require a complete lattice, a dedicated approach for the morphological processing of vectorial data on graphs is used. By iterating the application of morphological filterings of decreasing sizes, the graph signal is decomposed into several detail layers, each capturing a given detail level. Editing applications such as abstraction, sharpness enhancement and tone mapping are shown to illustrate the benefits of the proposed approach.

Information measure based tone mapping of outdoor LDR image for maximum scale‐invariant feature transform extraction

A novel tone mapping method referred to here as information measure-based tone mapping is proposed for extracting the maximum number of scale-invariant feature transform (SIFT) feature points from outdoor low dynamic range (LDR) images. To this end, first, an image is transformed to YUV colour space in which every pixel is assigned two information scores: forward and inverse scores. The forward and inverse information scores represent the number of neighbour pixels that differ in their intensities from the pixel under consideration by a certain threshold for the respective YUV and inverse YUV images. The proposed tone mapping is then carried out in such a way as to emphasise the pixels of high information scores defined by a weighted sum of forward and inverse information scores. Experimental results show that the proposed method offers superior performance to state-of-the-art methods in terms of the number of features extracted and matched.

Color Volume and Hue-preservation in HDR Tone Mapping

The electrical-electrical transfer function (EETF), described in International Telecommunications Union—Radiocommunication (ITU-R) BT.2390–7, offers four variations, which allow high luminance imagery to be tone-mapped to a lower luminance in different ways. This is useful when converting high dynamic range (HDR) encoding spaces or rendering HDR content on screens that have limited performance. However, for some input colors, these variations introduce drastic changes in hue, in some cases, and do not restrict the output to the desired color volume, in other cases. In this article, a fifth variation is proposed, that preserves the input color’s hue while also restricting the output to the specified color volume. This proposed variation is compared and shown to outperform the four standardized variations.

Study of naturalness in tone-mapped images

Nowadays, images can be obtained in various ways such as capturing photos in single-exposure mode, applying Multiple Exposure Fusion algorithms to generate an image from multiple shoots of the same scene, mapping High Dynamic Range (HDR) images to Standard Dynamic Range (SDR) images, converting raw formats to displayable formats, or applying post-processing techniques to enhance image quality, aesthetic quality,…When looking at some photos, one might have a feeling of unnaturalness. This paper deals with the problem of developing a model firstly to estimate if an image looks natural or not to humans and the second purpose is to try to understand how the unnaturalness feeling is induced by a photo: Are there specific unnaturalness clues or is unnaturalness a general feeling when looking at a photo? The study focuses on SDR images, especially on tone-mapped images. The first contribution of the paper is the setting of an experiment gathering human naturalness opinions on 1900 SDR images mainly obtained from tone mapping operators. Based on the collected data, the second contribution is to study the efficiency of different feature types including handcrafted features and learned features for image naturalness analysis. A binary classification model is then developed based on the determined features to classify if an image looks natural or unnatural.

Exploiting Local Degradation Characteristics and Global Statistical Properties for Blind Quality Assessment of Tone-Mapped HDR Images

Tone mapping operators (TMOs) are developed to convert a high dynamic range (HDR) image into a low dynamic range (LDR) one for display with the goal of preserving as much visual information as possible. However, image quality degradation is inevitable due to the dynamic range compression during the tone-mapping process. This accordingly raises an urgent demand for effective quality evaluation methods to select a high-quality tone-mapped image (TMI) from a set of candidates generated by distinct TMOs or the same TMO with different parameter settings. A key element to the success of TMI quality evaluation is to extract effective features that are highly consistent with human perception. Towards this end, this paper proposes a novel blind TMI quality metric by exploiting both local degradation characteristics and global statistical properties for feature extraction. Several image attributes including texture, structure, colorfulness and naturalness are considered either locally or globally. The extracted local and global features are aggregated into an overall quality via regression. Experimental results on two benchmark databases demonstrate the superiority of the proposed metric over both the state-of-the-art blind quality models designed for synthetically distorted images (SDIs) and the blind quality models specifically developed for TMIs.

Tone Mapped High Dynamic Range Image Quality Assessment Techniques: Survey and Analysis

The dynamic range of luminance value obtained by natural scenes frequently extent than the images that are captured by recently developed imaging sensors. In this real world, various practical scenarios desire to attain high dynamic range (HDR) illumination. So a large number of HDR imaging techniques were developed during the past decades. But, still there exist some limitations, so we performed survey in this area to overcome these limitations. The consistent methods developed for tone mapped HDR images is found to be meaningful and momentous as this techniques are found to be more beneficial for performing the comparison process for the tone mapping operators (TMOs) and also to advance the improvements in TMO techniques. During past decades, numbers of Image Quality Assessment (IQA) methods have been emerged to estimate the tone mapped HDR images. Recent advancements that are obtained in IQA techniques of tone mapped HDR images are reviewed. These IQA methods are normally categorized into two types they are full reference, and no/blind reference (NR). This type performs the QA process based on the information’s that are obtained from the reference images. NR-IQA is highly recommended by various datasets to assess HDR images as it does not require any perfect reference image for tone mapped HDR image. In this review, we discuss some valuable and interesting IQA methods that are widely employed to assess the image quality. Yet, there exist a prospective to perform a various research works in this field.

Tone-Mapped Image Quality Assessment for Electronics Displays by Combining Luminance Partition and Colorfulness Index

Tone mapping operators (TMOs) reproduce the high dynamic range (HDR) images on low dynamic range (LDR) consumer electronics devices such as monitors or printers. To accurately measure and compare the performance of different TMOs, this article proposes a tone-mapped images (TMIs) luminance partition model and corresponding quality measure. First, each tone-mapped (TM) image is segmented into highlight region (HR), dark region (DR) and midtone region (MR) based on luminance partition. Second, local entropies and contrast features are extracted in the HR and DR, and color-based features are captured in the MR. Meanwhile, the gray-level co-occurrence matrix (GLCM) and Canny operator are utilized to measure the microstructural distortions and halo effects, respectively. Finally, all extracted features are combined and trained together with subjective ratings to form a regression model using support vector regression (SVR). Experimental results show that the proposed method outperforms the state-of-the-art no-reference (NR) methods. Specifically, the spearman correlation coefficients (SRCC) values of our method reach 0.83 and 0.76 on the tone-mapped image database (TMID) and the ESPL-LIVE HDR database, respectively.

Structure revealing of low-light images using wavelet transform based on fractional-order denoising and multiscale decomposition

Images captured in low-light environment often lower its quality due to low illumination and high noise. Hence, the low visibility of images notably degrades the overall performance of multimedia and vision systems that are typically designed for high-quality inputs. To resolve this problem, numerous algorithms have been proposed in extant literature to improve the visual quality of low-light images. However, existing approaches are not good at improving overexposed portions and produce unnecessary distortion, which leads to poor visibility in images. Therefore, in this paper, a new model is proposed to prevent overenhancement, handle uneven illumination, and suppress noise in underexposed images. Firstly, the input image is converted into HSV color space. Then, the obtained V component is decomposed into high- and low-frequency subbands using the dual-tree complex wavelet transform. Secondly, a denoised model based on fractional-order anisotropic diffusion is applied on high-pass subbands. Thirdly, multiscale decomposition is used to extract more details from low-pass subbands, and inverse transformation is performed to compute final V. Next, sigmoid function and tone mapping are used on V-channel to prevent data loss and achieve robust results. Finally, the image is reconstructed and converted to RGB color space to achieve enhanced performance. Comparative experimental statistics show that the proposed method achieves high efficacy and outperforms the traditional approaches in terms of overall performance.

Quantization with Perception for Performance Improvement in HEVC for HDR Content

Human eye always reveals a non –linear understanding, for the disturbances caused by the lossy image and video coding. This is mainly because of the masking capability of the human eye to conceal the attributes such as contrast, luminous, spatial and temporal frequencies. To have a distortion less and efficient video encoding for the high dynamic video range content by eluding the invisible messages in the video that causes disturbances the paper puts forth the quantization with perception utilizing the luminous masking. The methodology utilized, computes the tone mapping to scale every frames in the HDR and later quantizes on unit basis with perception tuning. For this purpose the mechanism put forth incorporates the reference model of the HEVC with the extension range of the HEVC. The proposed model was validated by evaluating the reduction incurred in each rate of bit compared to the HDR range extension. The results acquired proved to have an enhancement in terms of the savings endured in the bit rate compared to the High efficient video coding that relied on the high dynamic range visible difference predictor-II

Blind quality assessment for tone-mapped images based on local and global features

In order to show high dynamic range (HDR) images by traditional displays, various tone-mapping operators have been designed to convert HDR images into low dynamic range (LDR) images recently. However, how to estimate the visual quality of LDR images effectively is still challenging. In this paper, we propose a novel blind quality assessment method for tone-mapped images with the consideration of naturalness and the perceptual characteristics of human visual system (HVS). First, we design parametric models that describe characteristics of chromatic information in tone-mapped images and extract quality-aware features based on global statistics model to characterize the naturalness of tone-mapped images. Second, motivated by perceptual characteristics that the HVS is highly adaptive to the image texture, we employ local texture features to capture the quality degradation of tone-mapped images. Support vector regression (SVR) is used to train the quality prediction model from features to human ratings. Experimental results indicate that the proposed metric can get better performance in predicting the visual quality of tone-mapped images than the state-of-the-art methods.

Photographic High-Dynamic-Range Scalar Visualization.

We propose a photographic method to show scalar values of high dynamic range (HDR) by color mapping for 2D visualization. We combine (1) tone-mapping operators that transform the data to the display range of the monitor while preserving perceptually important features, based on a systematic evaluation, and (2) simulated glares that highlight high-value regions. Simulated glares are effective for highlighting small areas (of a few pixels) that may not be visible with conventional visualizations; through a controlled perception study, we confirm that glare is preattentive. The usefulness of our overall photographic HDR visualization is validated through the feedback of expert users.

Retina inspired tone mapping method for high dynamic range images.

The limited dynamic range of regular screens restricts the display of high dynamic range (HDR) images. Inspired by retinal processing mechanisms, we propose a tone mapping method to address this problem. In the retina, horizontal cells (HCs) adaptively adjust their receptive field (RF) size based on the local stimuli to regulate the visual signals absorbed by photoreceptors. Using this adaptive mechanism, the proposed method compresses the dynamic range locally in different regions, and has the capability of avoiding halo artifacts around the edges of high luminance contrast. Moreover, the proposed method introduces the center-surround antagonistic RF structure of bipolar cells (BCs) to enhance the local contrast and details. Extensive experiments show that the proposed method performs robustly well on a wide variety of images, providing competitive results against the state-of-the-art methods in terms of visual inspection, objective metrics and observer scores.

Research on defect detection method for high-reflective-metal surface based on high dynamic range imaging

The high dynamic range imaging (HDRI) technology can realistically reproduce the scene and acquire more image information for analysis and processing. In this paper, a defect detection method based on HDRI is proposed to solve the visual inspection problem of industrial parts with high-reflective surface. Firstly, a fast calibration method is put forward to increase the calibration speed of the camera response function (CRF). Then, a novel algorithm named adaptive tone mapping based on color correction model in gradient domain is designed to adaptively adjust image chromaticity information. Finally, adaptive threshold segmentation and Haar-like feature extraction algorithm are used to detect surface defects in synthesized high dynamic range (HDR) images. The experimental results show that the HDR images processed by our algorithm can meet the detection requirements for high-reflective-metal surface. The number of sampling points required by our fast calibration algorithm of CRF is 1/10 times than that of the traditional algorithm, and the running time in the program is about 1/4 times than that of the traditional algorithm. What’s more, our improved local tone mapping algorithm is obviously improved in information entropy, contrast and average gradient.

Backlit images enhancement using global tone mappings and image fusion

The authors present a method for the enhancement of backlit images, i.e. images in which the main source of light is behind the photography subject. These images contain, simultaneously, very dark and very bright regions. In this situation, a single tone mapping function is unable to enhance the whole image. They propose the use of several such tone mappings, some of them enhancing the dark regions while others enhancing the bright regions, and then the combination of all these results using an image fusion algorithm. Qualitative and quantitative results confirm the validity of the proposed method.

Comparing common still image quality metrics in recent High Dynamic Range (HDR) and Wide Color Gamut (WCG) representations

There are an increasing number of databases describing subjective quality responses for HDR (high dynamic range) imagery with various distortions. The dominant distortions across the databases are those that arise from video compression, which are primarily perceived as achromatic, but there are some chromatic distortions due to 422 and other chromatic sub-sampling. Tone mapping from the source HDR levels to various levels of reduced capability SDR (standard dynamic range) are also included in these databases. While most of these distortions are achromatic, tone-mapping can cause changes in saturation and hue angle when saturated colors are in the upper hull of the of the color space. In addition, there is one database that specifically looked at color distortions in an HDR-WCG (wide color gamut) space. From these databases we can test the improvements to well-known quality metrics if they are applied in the newly developed color perceptual spaces (i.e., representations) specifically designed for HDR and WCG. We present results from testing these subjective quality databases to computed quality using the new color spaces of Jzazbz and ICTCP, as well as the commonly used SDR color space of CIELAB.

Tone Mapping High Dynamic Range Images by Hessian Multiset Canonical Correlations

Tone mapping algorithms reproduce high dynamic range (HDR) images on low dynamic range images in the standard display devices such as LCD, CRT, projectors, and printers. In this paper, we propose a statistical clustering-based tone mapping technique that would be able to adapt the local content of an image as well as its color. At first, the HDR image is partitioned into many overlapped color patches and we disintegrate each color patch into three segments: patch mean, color variation and color structure. Then based on the color structure component, the extracted color patches are clustered into a number of clusters by k-means clustering technique. For each cluster, the statistical signal processing technique namely Hessian multi set canonical correlations (HesMCC) has been produced to ascertain the transform matrix. Moreover, the HesMCC are fundamentally utilized for performing the dimensionality reduction of patches and to form effective tone mapped images. Contrasting with the current strategies, the procedures in the proposed clustering-based strategy can better adapt image color and its local structures by exploiting the image in the worldwide repetition. Experimental results show that the running time of the proposed method is less about 88.32%, 92%, 68.9%, and 29.4%, while comparing with other existing tone mapping methods.