High Dynamic Range Tone Mapping Research Articles

Subjective Quality Assessment of Compressed Tone-Mapped High Dynamic Range Videos.

High Dynamic Range (HDR) videos are able to represent wider ranges of contrasts and colors than Standard Dynamic Range (SDR) videos, giving more vivid experiences. Due to this, HDR videos are expected to grow into the dominant video modality of the future. However, HDR videos are incompatible with existing SDR displays, which form the majority of affordable consumer displays on the market. Because of this, HDR videos must be processed by tone-mapping them to reduced bit-depths to service a broad swath of SDR-limited video consumers. Here, we analyze the impact of tone-mapping operators on the visual quality of streaming HDR videos. To this end, we built the first large-scale subjectively annotated open-source database of compressed tone-mapped HDR videos, containing 15,000 tone-mapped sequences derived from 40 unique HDR source contents. The videos in the database were labeled with more than 750,000 subjective quality annotations, collected from more than 1,600 unique human observers. We demonstrate the usefulness of the new subjective database by benchmarking objective models of visual quality on it. We envision that the new LIVE Tone-Mapped HDR (LIVE-TMHDR) database will enable significant progress on HDR video tone mapping and quality assessment in the future. To this end, we make the database freely available to the community at https://live.ece.utexas.edu/research/LIVE TMHDR/index.html.

A Dataset and Model for the Visual Quality Assessment of Inversely Tone-Mapped HDR Videos.

To enhance the viewer experience of standard dynamic range (SDR) video content on high dynamic range (HDR) displays, inverse tone mapping (ITM) is employed. Objective visual quality assessment (VQA) models are needed for effective evaluation of ITM algorithms. However, there is a lack of specialized VQA models for assessing the visual quality of inversely tone-mapped HDR videos (ITM-HDR-Videos). This paper addresses both an algorithmic and a dataset gap by introducing a novel SDR referenced HDR (SD-R-HD) VQA model tailored for ITM-HDR-Videos, along with the first public dataset specifically constructed for this purpose. The innovations of the SD-R-HD VQA model include 1) utilizing available SDR video as a reference signal, 2) extracting features that characterize standard ITM operations such as global mapping and local compensation, and 3) directly modeling interframe inconsistencies introduced by ITM operations. The newly created ITM-HDR-VQA dataset comprises 200 ITM-HDR-Videos annotated with mean opinion scores, gathered over 320 man-hours of psychovisual experiments. Experimental results demonstrate that the SD-R-HD VQA model significantly outperforms existing state-of-the-art VQA models.

High-Dynamic-Range Tone Mapping in Intelligent Automotive Systems

Intelligent driver assistance systems are becoming increasingly popular in modern passenger vehicles. A crucial component of intelligent vehicles is the ability to detect vulnerable road users (VRUs) for an early and safe response. However, standard imaging sensors perform poorly in conditions of strong illumination contrast, such as approaching a tunnel or at night, due to their dynamic range limitations. In this paper, we focus on the use of high-dynamic-range (HDR) imaging sensors in vehicle perception systems and the subsequent need for tone mapping of the acquired data into a standard 8-bit representation. To our knowledge, no previous studies have evaluated the impact of tone mapping on object detection performance. We investigate the potential for optimizing HDR tone mapping to achieve a natural image appearance while facilitating object detection of state-of-the-art detectors designed for standard dynamic range (SDR) images. Our proposed approach relies on a lightweight convolutional neural network (CNN) that tone maps HDR video frames into a standard 8-bit representation. We introduce a novel training approach called detection-informed tone mapping (DI-TM) and evaluate its performance with respect to its effectiveness and robustness in various scene conditions, as well as its performance relative to an existing state-of-the-art tone mapping method. The results show that the proposed DI-TM method achieves the best results in terms of detection performance metrics in challenging dynamic range conditions, while both methods perform well in typical, non-challenging conditions. In challenging conditions, our method improves the detection F2 score by 13%. Compared to SDR images, the increase in F2 score is 49%.

Adaptive Enhancement of High Greyscale Weld Images Based on the New Pixel Self-transformation Method

X-ray nondestructive testing technology is widely used in the weld detection of oil and gas pipeline industry. The industrial welding manufacturing field produces 12-bit and higher high grey scale images that cannot be displayed directly on an 8-bit display. In this paper, we propose an adaptive pixel self-transformation method for high greyscale weld images. The method takes into account the characteristics of weld images and human perception. Firstly, a high dynamic range tone mapping model is designed, which converts the original RAW data into optical observable data. Secondly, in order to solve the problem that the traditional pixel self-transformation method can only adapt to the input with 256 levels of greyscale and cannot process the high-bit image data. An adaptive high-bit pixel self-transformation is proposed. Finally, the method of this paper was tested on 12-bit X-ray films. The experimental results show that the designed method, which improves the recognizability of weld images and can effectively obtain weld images with outstanding details, is an efficient weld feature enhancement algorithm.

A lightweight exposure bracketing strategy for HDR imaging without access to camera raw

A lightweight learning-based exposure bracketing strategy is proposed in this paper for high dynamic range (HDR) imaging without access to camera RAW. Some low-cost, power-efficient cameras, such as webcams, video surveillance cameras, sport cameras, mid-tier cellphone cameras, and navigation cameras on robots, can only provide access to 8-bit low dynamic range (LDR) images. Exposure fusion is a classical approach to capture HDR scenes by fusing images taken with different exposures into a 8-bit tone-mapped HDR image. A key question is what the optimal set of exposure settings are to cover the scene dynamic range and achieve a desirable tone. The proposed lightweight neural network predicts these exposure settings for a 3-shot exposure bracketing, given the input irradiance information from 1) the histograms of an auto-exposure LDR preview image, and 2) the maximum and minimum levels of the scene irradiance. Without the processing of the preview image streams, and the circuitous route of first estimating the scene HDR irradiance and then tone-mapping to 8-bit images, the proposed method gives a more practical HDR enhancement for real-time and on-device applications. Experiments on a number of challenging images reveal the advantages of our method in comparison with other state-of-the-art methods qualitatively and quantitatively.

Image declipping: Saturation correction in single images

High dynamic range (HDR) images present fine details in a scene and are visually more appealing than low dynamic range (LDR) images, since they contain a greater dynamic range of color gamut. HDR compatible displays are currently high-cost, therefore tone-mapping algorithms have widely been used to obtain high quality images for LDR screens with a lower cost. However, tone-mapped images may contain clipped pixel regions, which should be corrected to retrieve the lost information, to acquire visually pleasing LDR images. In a single image, the recovery of color and texture information in clipped regions is challenging, yet an attractive research field in image processing. Although there are several algorithms present in literature, developing a general framework for different types of image content is hard to achieve. This study proposes a single image declipping method based on linear embeddings, difference of pixels and block-search. Experimental results carried out on a tone-mapped HDR image dataset and LDR images demonstrate that the proposed algorithm is able to successfully recover saturated pixels in various types of images. Detailed statistical and visual comparisons show that this approach produces superior results on average for both tone-mapped and LDR images when compared to existing techniques.

Blind Tone-Mapped HDR Image Quality Measurement by Analysis of Low-Level and High-Level Perceptual Characteristics

Tone-mapping technology aims to transform High Dynamic Range (HDR) images into standard dynamic range images for visualizing on standard screens. However, various visual artifacts will inevitably be induced during the tone-mapping process. To quantify the visual quality degradation for Tone-Mapped Images (TMIs) accurately and automatically, a robust blind TMI quality measurement method is developed by analysis of low-level and high-level perceptual characteristics. Specifically, for low-level visual features, considering that the tone-mapping operator is prone to destroy the local image contrast, which conveys vital structural information, we resort to the joint statistical features based on gradient map and Laplacian of Gaussian response to portray the local image contrast variation in TMIs. In addition, Chromatic Local Binary Pattern (CLBP) is leveraged to measure the colorfulness degradation over four chromatic descriptor maps. For high-level quality-aware features, deep-learned features are utilized by a pre-trained convolutional neural network to characterize the semantics variation. Finally, the extracted low-level and high-level features are combined and mapped into an overall quality score by regression function. Extensive experiments show that our metric achieves a higher performance over two publicly available TMI benchmark databases.

Quantitative Measurement of Perceptual Attributes and Artifacts for Tone-Mapped HDR Display

Measuring electronic display quality, as perceived by human observers, has attracted high attention in current consumer displays. With limited dynamic range of consumer-level standard dynamic range (SDR) displays, high dynamic range (HDR) scenes are often rendered by different tone mapping operators (TMOs). Due to the lack of quantitative measurement and analysis of tone mapping in the existing image quality measurement (IQM) methods, it is of considerable significance to establish new IQM protocols that can differentiate the display quality of SDR electronic devices. We first propose an IQM model to exhibit the essential perceptual attributes and artifacts that are peculiar to tone-mapping. Furthermore, we characterize the overall image quality (OiQ) resulting from linear regression and various machine learning techniques. Finally, the execution of without HDR reference ablation experiments demonstrate the relative contribution of these attribute measurement to the OiQ. The use of IQM protocols helps with well-founded quality measurement between TMOs during tone-mapping processing. Our effort is not only useful to get into the tone-mapping field or when implementing a TMO, but it also sets the stage for quantitative measurement of TMOs. By monitoring these attributes and artifacts after different tone-mapping process, user-driven or optimal display are made possible.

Tone mapping high dynamic range images based on region-adaptive self-supervised deep learning

This paper presents a region-adaptive self-supervised deep learning (RASSDL) technique for high dynamic range (HDR) image tone mapping. The RASSDL tone mapping operator (TMO) is a convolutional neural network (CNN) trained on local image regions that can seamlessly tone map images of arbitrary sizes. The training of RASSDL TMO is through the design of a self-supervising target that automatically adapts to the local image regions based on their information contents. The self-supervising target is designed to ensure the tone-mapped output achieves a balance between preserving the relative contrast of the original scene and the visibilities of the fine details to achieve faithful reproduction of the HDR scene. Distinguishing from many existing TMOs that require manual tuning of parameters, RASSDL is parameter-free and completely automatic. Experimental results demonstrate that RASSDL TMO can achieve state-of-the-art performance in terms of preserving overall contrasts, revealing fine details, and being free from visual artifacts.

Blind Quality Evaluator of Tone-Mapped HDR and Multi-Exposure Fused Images for Electronic Display

The problem of reproducing high dynamic range (HDR) images on electronic display and photography with restricted dynamic range has gained a lot of interest in the consumer electronics community. There exist various approaches to this issue, e.g., tone mapping operators (TMOs) and multi-exposure fusion algorithms (MEFs). Many existing image quality assessment (IQA) methods have been proposed to compare images of quality degradation generated by TMOs/MEFs. Although promising performances have been achieved, they seldom consider local specific artifacts difference (i.e., abnormal exposure and color cast) related with the TMOs/MEFs. To address this limitation, this paper proposes a Blind Quality Evaluator of Tone-Mapped HDR and Multi-Exposure Fused Images (BQE-TM/MEFI). First, two purpose-designed segment models are utilized to distinguish well-exposedness dense patches (WEDPes) and non-WEDPes, color cast patches (CCPes) and non-CCPes respectively. Second, multiple quality-perception features are extracted to measure local artifacts: 1) structure and sharpness features from WEDPes, 2) saturation features from non-CCPes, and 3) edge structure features. Then, three new low-complexity regional features (over-exposure ratio, entropy and color confidence index) are calculated based on over-exposure segmentation model. Finally, all extracted features are aggregated into a machine-learning regression model to pool a quality score. The simplicity and good performance of the proposed method makes it suitable for electronic displays and other consumer electronics.

Blind Quality Index for Tone-mapped HDR Images and Multi-Exposure Fused Images

Tone mapping operators (TMOs) and multi-exposure fusion algorithms (MEFs) are employed to visualize the informative contents of high dynamic range (HDR) images on the standard dynamic range (SDR) devices, but also lead to visual quality degradation. Thus, it is urgent to evaluate the perceptual quality of tone-mapped images (TMIs) and multi-exposure fusion images (MEFIs). This paper proposes a blind quality index for TMIs/MEFIs by capturing exposure, colorfulness and structure characteristic features. The connection between feature space and associated subjective ratings is established via a regression model. The proposed method is tested on the public database and compared to the state-of-the-art blind image quality assessment (IQA) methods. The experimental results show that the proposed method achieves superior performance.

Fast Single-Image HDR Tone-Mapping by Avoiding Base Layer Extraction.

The tone-mapping algorithm compresses the high dynamic range (HDR) information into the standard dynamic range for regular devices. An ideal tone-mapping algorithm reproduces the HDR image without losing any vital information. The usual tone-mapping algorithms mostly deal with detail layer enhancement and gradient-domain manipulation with the help of a smoothing operator. However, these approaches often have to face challenges with over enhancement, halo effects, and over-saturation effects. To address these challenges, we propose a two-step solution to perform a tone-mapping operation using contrast enhancement. Our method improves the performance of the camera response model by utilizing the improved adaptive parameter selection and weight matrix extraction. Experiments show that our method performs reasonably well for overexposed and underexposed HDR images without producing any ringing or halo effects.

Color Volume and Hue-preservation in HDR Tone Mapping

The electrical-electrical transfer function (EETF), described in International Telecommunications Union—Radiocommunication (ITU-R) BT.2390–7, offers four variations, which allow high luminance imagery to be tone-mapped to a lower luminance in different ways. This is useful when converting high dynamic range (HDR) encoding spaces or rendering HDR content on screens that have limited performance. However, for some input colors, these variations introduce drastic changes in hue, in some cases, and do not restrict the output to the desired color volume, in other cases. In this article, a fifth variation is proposed, that preserves the input color’s hue while also restricting the output to the specified color volume. This proposed variation is compared and shown to outperform the four standardized variations.

Exploiting Local Degradation Characteristics and Global Statistical Properties for Blind Quality Assessment of Tone-Mapped HDR Images

Tone mapping operators (TMOs) are developed to convert a high dynamic range (HDR) image into a low dynamic range (LDR) one for display with the goal of preserving as much visual information as possible. However, image quality degradation is inevitable due to the dynamic range compression during the tone-mapping process. This accordingly raises an urgent demand for effective quality evaluation methods to select a high-quality tone-mapped image (TMI) from a set of candidates generated by distinct TMOs or the same TMO with different parameter settings. A key element to the success of TMI quality evaluation is to extract effective features that are highly consistent with human perception. Towards this end, this paper proposes a novel blind TMI quality metric by exploiting both local degradation characteristics and global statistical properties for feature extraction. Several image attributes including texture, structure, colorfulness and naturalness are considered either locally or globally. The extracted local and global features are aggregated into an overall quality via regression. Experimental results on two benchmark databases demonstrate the superiority of the proposed metric over both the state-of-the-art blind quality models designed for synthetically distorted images (SDIs) and the blind quality models specifically developed for TMIs.

Tone Mapping High Dynamic Range Images by Hessian Multiset Canonical Correlations

Tone mapping algorithms reproduce high dynamic range (HDR) images on low dynamic range images in the standard display devices such as LCD, CRT, projectors, and printers. In this paper, we propose a statistical clustering-based tone mapping technique that would be able to adapt the local content of an image as well as its color. At first, the HDR image is partitioned into many overlapped color patches and we disintegrate each color patch into three segments: patch mean, color variation and color structure. Then based on the color structure component, the extracted color patches are clustered into a number of clusters by k-means clustering technique. For each cluster, the statistical signal processing technique namely Hessian multi set canonical correlations (HesMCC) has been produced to ascertain the transform matrix. Moreover, the HesMCC are fundamentally utilized for performing the dimensionality reduction of patches and to form effective tone mapped images. Contrasting with the current strategies, the procedures in the proposed clustering-based strategy can better adapt image color and its local structures by exploiting the image in the worldwide repetition. Experimental results show that the running time of the proposed method is less about 88.32%, 92%, 68.9%, and 29.4%, while comparing with other existing tone mapping methods.

FFTMI: Features Fusion for Natural Tone-Mapped Images Quality Evaluation

Tone-mapping is a crucial step in the task towards displaying high dynamic range (HDR) images on standard displays. Given the number of possible ways to tone-map such images, development of an objective quality criterion, enabling selection of the most suitable tone-mapping operator (TMO) and setting its parameters in order to maximize the quality of the reproduction, is of high interest. In this paper, a new objective metric for natural tone-mapped images is proposed. It is based on a fusion of several perceptually relevant features that have been carefully selected using an appropriate feature selection procedure. The outcome of the selection also provides a valuable insight into the importance of particular perceptual aspects when judging the quality of tone-mapped HDR content. The performance of the resulting combination of features is thoroughly evaluated with respect to three publicly available databases and compared to several relevant state-of-the-art criteria. The proposed approach is shown to significantly outperform the tested metrics and can, therefore, be considered a competitive alternative for tone-mapped images evaluation.

Referenceless Quality Evaluation of Tone-Mapped HDR and Multiexposure Fused Images

Nowadays, the standard dynamic range (SDR) image acquired at a fixed exposure exposes weakness in portraying fine-grained details of real scenes. The high dynamic range (HDR) image and other types of SDR images generated by multiexposure fusion techniques provide us new choices for scene representation. To display on SDR screens, an HDR image must be tone-mapped to an SDR one. Since different tone-mapping/fusion algorithms produce images with varying visual quality levels, it naturally desires a quality evaluation model for comparison. This article proposes an effective model in the absence of the reference image. By analyzing the characteristics of tone-mapped HDR and multiexposure fused images, we first extract multiple quality-sensitive features from the following aspects: 1) colorfulness; 2) exposure; and 3) naturalness. Then, the model is built by bridging all extracted features and associated subjective ratings via support vector regression. Extensive experiments on publicly available databases prove the superiority of our model over the state-of-the-art referenceless quality evaluation ones.

Tone-mapped High Dynamic Range Image Restoration with a Mask-applied Neural Network

Tone-mapped High Dynamic Range Image Restoration with a Mask-applied Neural Network

An extended variational image decomposition model for color image enhancement

Abstract Variational image decomposition model has been widely used in computer vision and computational photography to decompose an image into a luminance layer and a reflectance layer. Existing method perform decomposition either on the intensity channel of a color image or on each of the RGB channels separately. However, these methods fail to exploit effectively the correlation among color channels, and will lose much color edges and textures during the decomposition process. This paper presents an extended variational image decomposition model, which works directly on the color image and decomposes it into a luminance layer, a reflectance layer and a color layer by solving a total variation minimization problem. The luminance layer is piecewise smooth containing the general object appearance, the reflectance layer is piecewise constant representing some structure information. The color layer accounts for distinct color structure in each color channel and reveals the color variation of the image. By manipulating the three layers, the proposed decomposition model can be applied to a variety of color image decomposition applications such as high dynamic range tone mapping and contrast enhancement. Our experimental results validate the highly competitive performance of the proposed method in comparison with state-of-the-arts.

Locally Adaptive Rank-Constrained Optimal Tone Mapping

High dynamic range (HDR) tone mapping is formulated as an optimization problem of maximizing perceivable spatial details given the limited dynamic range of display devices. This objective can be attained, as supported by our results, by a novel image display methodology called locally adaptive rank-constrained optimal tone mapping (LARCOTM). The scientific basis for LARCOTM is that the maximum discrimination power of human vision system can only be achieved in a relatively small locality of an image. LARCOTM is fundamentally different from existing HDR tone mapping techniques in that the former can preserve pixel value order statistics within localities in which human foveal vision retains maximum sensitivity, while the latter cannot. As a result, images enhanced by LARCOTM are free of artifacts such as halos and double edges that plague other HDR methods.