Tone-mapped Image Quality Assessment Research Articles

Quality Assessment of Tone-Mapped Images Using Fundamental Color and Structural Features

High dynamic range (HDR) images require tone-mapping to be viewed on low dynamic range (LDR) displays. The performance of tone-mapping algorithms can be evaluated through a subjective study in which participants based on their liking rank or score tone-mapped images (TMIs). Subjective evaluation can be painstakingly slow; therefore, several quantitative metrics have been proposed for objective evaluation. This paper presents a new robust metric that uses 16 features, measuring the loss of color, contrast, brightness, and structure, extracted from the test TMI and the reference HDR image. The effect of these attributes on image quality is investigated and combined into a single score in the [0, 1] range describing the quality of TMI. We validate the performance of the proposed metric by comparing it with 24 existing state-of-the-art metrics. The study uses two subjective datasets of TMIs, including one existing benchmark dataset and a new proposed dataset comprising HDR images of a variety of scenes, and a dataset of traditional images not generated through tone-mapping. In these studies, our method shows the highest correlation with subjective scores for both datasets of TMIs and remains in the second position for the dataset of traditional images.

Real-time self-supervised tone curve estimation for HDR image

High dynamic range (HDR) image acquisition technology can record real-scene information. However, most display devices only support standard dynamic range images; therefore, dynamic range compression via tone mapping operators is a key technology for HDR image visualization. Recently, deep learning has achieved significantly better results than traditional methods in tone mapping; however, some problems remain. On the one hand, the labels of datasets are usually selected from the results of existing traditional methods, which limit the quality of newly generated results. On the other hand, these algorithms require huge computational resources and cannot be practically applied. As the dynamic range can be compressed using an intuitive and simple sigmoid curve mapping, a lightweight network is designed in this study to estimate the patch-wise tone curve parameters for HDR images. Simultaneously, the differentiable approximation of tone mapped image quality assessment is introduced as a self-supervised loss term so that the method only needs HDR data as training data. Experimental results demonstrate that the proposed method achieves better results than existing methods in both objective and subjective metrics at a low computational cost.

No-reference quality index of tone-mapped images based on authenticity, preservation, and scene expressiveness

As a future-proof technology, high dynamic range (HDR) has the superiority in scene reappearance. Tone mapping operator (TMO) renders the dynamic effect of HDR images in a standard dynamic range (SDR) display at the expense of some insignificant information. However, it is difficult for existing TMOs to consistently achieve high-quality conversion of all HDR images, and the loss of converted images is inevitable. Therefore, accurate prediction of tone-mapped image (TMI) quality is of considerable significance. In this paper, we propose the reappearance effect of the TMIs index (RETI). According to the characteristics of primeval HDR images, the quality of TMIs is considered from three important elements: authenticity, the preservation of energy and information, and scene expressiveness. The feature vectors originated from three elements are combined with subjective quality to train prediction model. Experiments are executed on tow mainstream TMI quality assessment databases and the results show that the proposed method has a good prediction ability and generalization ability in comparison to some state-of-the-art methods. Our MATLAB source code will be released at https://github.com/niannianyouyu/RETI.

Blind quality assessment of tone-mapped images using multi-exposure sequences

The tone mapping operator (TMO) enables high dynamic range (HDR) images to be presented on low dynamic range (LDR) consumer electronic devices. However, the results obtained by this method are not always ideal due to the reduced number of bits. In comparison, the multi-exposure image fusion (MEF) bypasses the intermediate HDR image composition and directly produces an image presented on standard devices. Inspired by this, this paper proposes a quality assessment method for tone-mapped image (TMI) based on generating multi-exposure sequences. Specifically, the method uses a generative adversarial network (GAN) to generate a set of sequences with different exposure levels based on the TMIs. Then a two-branch convolutional neural network (CNN) is used to extract features from the tone-mapped images and the multi-exposure reference sequences, respectively. Finally, the transformer is used to mine the intrinsic connections between TMIs and multi-exposure sequences and learn the mapping relationships from feature space to quality space. We conducted extensive experiments on the ESPL-LIVE HDR database. The applicability and effectiveness of the proposed method are verified by comparing and analyzing relevant features and model configurations with existing mainstream evaluation algorithms.

Blind Quality Assessment of Tone-Mapped Images Based on Visual-Processing Features

Blind Quality Assessment of Tone-Mapped Images Based on Visual-Processing Features

No-reference quality assessment of tone-mapped image using clustering perception and salient regions detection

In order to display a high dynamic range (HDR) image on a standard display device, we need to compress an HDR image into a tone-mapped image (TMI) using the tone-mapped operator (TMO). In order to evaluate the performance of different TMOs, a blind quality evaluation method of TMI is proposed. First, the TMI is divided into bright, dark and middle regions by clustering algorithm, and the Y channel is extracted in these three regions respectively to calculate the goodness of Gaussian fitness of the histogram. Second, the TMI is decomposed by tucker decomposition, and the texture features are extracted on the first sub-band. After that, the non-negative matrix is applied to calculate the salient regions of TMI, and features in the salient regions are extracted. Then, for the sake of comprehensive evaluation the quality of TMI, the naturalness feature is extracted in the gray scale domain and colorfulness features are extracted in HSI color space. Finally, the final score quality is predicted using support vector regression model training. Experimental results on TMID and ESPL-LIVE HDR database proved the proposed method is very effective and superior to the existing blind TMI quality assessment methods.

Deep Tone-Mapping Operator Using Image Quality Assessment Inspired Semi-Supervised Learning

Tone-mapping operator (TMO) is intended to convert high dynamic range (HDR) content into a lower dynamic range so that it can be displayed on a standard dynamic range (SDR) device. The tone-mapped result of HDR content is usually stored as SDR image. For different HDR scenes, traditional TMOs are able to obtain a satisfying SDR image only under manually fine-tuned parameters. In this paper, we address this problem by proposing a learning-based TMO using deep convolutional neural network (CNN). We explore different CNN structure and adopt multi-scale and multi-branch fully convolutional design. When training deep CNN, we introduce image quality assessments (IQA), specifically, tone-mapped image quality assessment, and implement it as semi-supervised loss terms. We discuss and prove the effectiveness of semi-supervised loss terms, CNN structure, data pre-processing, etc. by several experiments. Finally, we demonstrate that our approach can produce appealing results under diversified HDR scenes.

Blind Tone-Mapped Image Quality Assessment Based on Regional Sparse Response and Aesthetics.

High dynamic range (HDR) images give a strong disposition to capture all parts of natural scene information due to their wider brightness range than traditional low dynamic range (LDR) images. However, to visualize HDR images on common LDR displays, tone mapping operations (TMOs) are extra required, which inevitably lead to visual quality degradation, especially in the bright and dark regions. To evaluate the performance of different TMOs accurately, this paper proposes a blind tone-mapped image quality assessment method based on regional sparse response and aesthetics (RSRA-BTMI) by considering the influences of detail information and color on the human visual system. Specifically, for the detail loss in a tone-mapped image (TMI), multi-dictionaries are first designed for different brightness regions and whole TMI. Then regional sparse atoms aggregated by local entropy and global reconstruction residuals are presented to characterize the regional and global detail distortion in TMI, respectively. Besides, a few efficient aesthetic features are extracted to measure the color unnaturalness of TMI. Finally, all extracted features are linked with relevant subjective scores to conduct quality regression via random forest. Experimental results on the ESPL-LIVE HDR database demonstrate that the proposed RSRA-BTMI method is superior to the existing state-of-the-art blind TMI quality assessment methods.

Tone-Mapped Image Quality Assessment for Electronics Displays by Combining Luminance Partition and Colorfulness Index

Tone mapping operators (TMOs) reproduce the high dynamic range (HDR) images on low dynamic range (LDR) consumer electronics devices such as monitors or printers. To accurately measure and compare the performance of different TMOs, this article proposes a tone-mapped images (TMIs) luminance partition model and corresponding quality measure. First, each tone-mapped (TM) image is segmented into highlight region (HR), dark region (DR) and midtone region (MR) based on luminance partition. Second, local entropies and contrast features are extracted in the HR and DR, and color-based features are captured in the MR. Meanwhile, the gray-level co-occurrence matrix (GLCM) and Canny operator are utilized to measure the microstructural distortions and halo effects, respectively. Finally, all extracted features are combined and trained together with subjective ratings to form a regression model using support vector regression (SVR). Experimental results show that the proposed method outperforms the state-of-the-art no-reference (NR) methods. Specifically, the spearman correlation coefficients (SRCC) values of our method reach 0.83 and 0.76 on the tone-mapped image database (TMID) and the ESPL-LIVE HDR database, respectively.

Blind tone mapped image quality assessment with image segmentation and visual perception

With tone mapping, high dynamic range (HDR) image contents can be displayed on low dynamic range (LDR) display devices, in which some important visual information may be distorted. Thus, the tone mapped image (TMI) quality assessment is one of important issues in HDR image/video processing fields. Considering the difference of visual distortion degrees between the flat and complex regions in TMI, and considering that high-quality TMI should preserve as much information as possible of its original HDR image especially in the high/low luminance regions, this paper proposes a new blind TMI quality assessment method with image segmentation and visual perception. First, we design different features to describe the distortion of TMI’s different regions with two kinds of TMI segmentation. Then, considering that there lacks an efficient algorithm to quantify the importance of features, a feature clustering scheme is designed to eliminate the poor effect feature components in the extracted features to improve the effectiveness of the selected features. Finally, considering the diversity of tone mapping operator (TMO), which may cause global and local distortion of TMI, some other global features are also combined. At last, a final feature vector is formed to synthetically describe the distortion in TMI and used to blindly predict the TMI’s quality. Experimental results in the public ESPL-LIVE HDR database show that the Pearson linear correlation coefficient and Spearman rank order correlation coefficient of the proposed method reach 0.8302 and 0.7887, respectively, which is superior to the state-of-the-art blind TMI quality assessment methods, and it means that the proposed method is highly consistent with human visual perception.

Blind Quality Assessment of Tone-Mapped Images Considering Colorfulness, Naturalness, and Structure

Nowadays, high-dynamic-range (HDR) imaging represents a prevailing trend and attracts much attention from both academic and industrial scholars. Since HDR images cannot be properly produced on the mainstream low-dynamic-range (LDR) displays, various tone-mapping operators or postprocessing technologies have been designed to transform HDR images into LDR images for visualization on LDR displays. However, it inevitably induces artifacts and distortions due to dynamic range compression. Besides, existing tone-mapped (TM) technologies cannot effectively handle all kinds of images with diverse contents and structures, leaving to a very challenging and urgent image quality assessment (IQA) problem. To cope with this challenge, in this paper, an effective blind quality assessment approach for TM images is proposed through a comprehensive consideration of their characteristics. More specifically, to dig out sufficient information from TM images, multiple quality-sensitive features are captured to fully represent different attributes, including colorfulness, naturalness, and structure. The connection between feature space and associated subjective ratings is established via a regression model. Extensive experiments on a recently released TM image database prove that the proposed approach is superior to the state-of-the-art no-reference IQA approaches.

Biologically Inspired Blind Quality Assessment of Tone-Mapped Images

Currently, many tone mapping operators (TMOs) have been provided to compress high dynamic range images to low dynamic range (LDR) images for visualizing them on the common displays. Since quality degradation is inevitably induced by compression, how to evaluate the obtained LDR images is indeed a headache problem. Until now, only a few full reference (FR) image quality assessment metrics have been proposed. However, they highly depend on reference image and neglect human visual system characteristics, hindering the practical applications. In this paper, we propose an effective blind quality assessment method of tone-mapped image without access to reference image. Inspired by that the performance of existing TMOs largely depend on the brightness and chromatic and structural properties of a scene, we evaluate the perceptual quality from the perspective of color information processing in the brain. Specifically, motivated by the physiological and psychological evidence, we simulate the responses of single-opponent (SO) and double-opponent (DO) cells, which play an important role in the processing of the color information. To represent the textural information, we extract three features from gray-level co-occurrence matrix (GLCM) calculated from SO responses. Meanwhile, both GLCM and local binary pattern descriptor are employed to extract texture and structure in the responses of DO cells. All these extracted features and associated subjective ratings are learned to reveal the connection between feature space and human opinion score. Extensive experiments demonstrate that the proposed method outperforms the state-of-the-art blind quality assessment methods and is comparable with the popular FR methods on two recently published tone-mapped image databases.

Full-Reference Objective Quality Assessment of Tone-Mapped Images

In this paper we present a novel method for full-reference image quality assessment (IQA) of tone-mapped images displayed on standard low dynamic range (LDR) displays. Due to the dynamic range compression caused by the tone-mapping process a mixture of several artifacts and distortions may be produced in the tone-mapped images. This makes the quality assessment of the tone-mapped images very challenging. Due to the diversity of such artifacts and distortions we propose a “bag of features” (BOF) approach to tackle this problem. Specifically in the proposed method a number of different perceptually relevant quality-related features are first extracted from a given tone-mapped image and its reference HDR image. These features are designed such that they capture different aspects and attributes of the tone-mapped image such as its structural fidelity naturalness and overall brightness. A support vector regressor is then trained based on the extracted features and it is used for measuring the visual quality of a tone-mapped image. Our experimental results indicate that the proposed method achieves high accuracy as compared to several existing methods.

Blind Tone-Mapped Image Quality Assessment Based on Brightest/Darkest Regions, Naturalness and Aesthetics

A tone-mapped image (TMI) converted from its high-dynamic-range image (HDRI) tends to appear overexposed in its brightest regions or underexposed in its darkest regions, resulting in inevitable loss of details and impaired naturalness and aesthetics. To address this issue, this paper proposes a novel blind TMI quality assessment (BTMIQA) method for HDRIs used in different dynamic range displays. The brightest and darkest regions of the TMI are first defined and segmented, and their local detail features are used in combination with the global detail feature of the TMI to evaluate the detail distortion. The natural scene statistics features of the luminance and yellow channels of the TMI are then used to evaluate the luminance naturalness and chrominance naturalness, respectively. Subsequently, to predict the aesthetics of the TMI, a series of colorfulness features are extracted and concatenated in a final feature vector, which is used to evaluate the TMI quality by random forest regression. Experiments are performed on the TMI database and ESPL-LIVE high-dynamic-range database, and the results show the effectiveness of the proposed method. Compared with the existing TMI quality assessment methods, the proposed BTMIQA method is more consistent with human visual perception.

High Dynamic Range Image Analysis through Various Tone Mapping Techniques

image quality is improved drastically with the increase of the technology. The conventional display devices may not be suitable for these High dynamic range images. The tone mapping is the process to show the good quality image in the normal LDR display devices. This paper presents a review of the tone mapping algorithms. It provides the methodology on Tone Mapped Image Quality Index (TMIQI) and the Blind Quality Assessment of Tone-Mapped Images (BTMQI). The region is basically expanded and compressed to visualize properly. Thereby the region-enhanced pseudo-exposures are fused into an HDR image. The image quality of BTMQI is comparatively higher than the TMIQI method. The low dynamic range images are suitable to both the conventional and advance display devices. Keywordsdynamic range imaging, structural preservation, tone mapping, perceptual image processing, structural similarity

A Tone Mapping Algorithm Based on Multi-scale Decomposition

A new Tone Mapping( TM) algorithm based on multi-scale decomposition was proposed to solve a High Dynamic Range( HDR) image displayed on an ordinary display device. The algorithm decomposed a HDR image into multiple scales using a Local Edge-Preserving( LEP) filter to smooth the details of the image effectively, while still retaining the salient edges. Then a dynamic range compression function with parameters was proposed according to the characteristics of the decomposed layers and the request of compression. By changing the parameters, the coarse scale layer was compressed and the fine scale layer was boosted, which resulted in compressing the dynamic range of the image and boosting the details. Finally, by restructuring the image and restoring the color, the image after mapping had a good visual quality. The experimental results demonstrate that the proposed method is better than the algorithm proposed by Gu et al.( GU B, LI W J, ZHU M Y, et al. Local edge-preserving multiscale decomposition for high dynamic range image tone mapping [J]. IEEE Transactions on Image Processing, 2013, 22( 1) : 70- 79) and Yeganeh et al.( YEGANEH H, WANG Z. Objective quality assessment of tonemapped images [J]. IEEE Transactions on Image Processing, 2013, 22( 2) : 657- 667) in naturalness, structural fidelity and quality assessment; moreover, it avoids the halo artifacts which is a common problem existing in the local tone mapping algorithms. The algorithm can be used for the tone mapping of the HDR image.

Blind Quality Assessment of Tone-Mapped Images Via Analysis of Information, Naturalness, and Structure

High dynamic range (HDR) imaging techniques have been working constantly, actively, and validly in the fault detection and disease diagnosis in the astronomical and medical fields, and currently they have also gained much more attention from digital image processing and computer vision communities. While HDR imaging devices are starting to have friendly prices, HDR display devices are still out of reach of typical consumers. Due to the limited availability of HDR display devices, in most cases tone mapping operators (TMOs) are used to convert HDR images to standard low dynamic range (LDR) images for visualization. But existing TMOs cannot work effectively for all kinds of HDR images, with their performance largely depending on brightness, contrast, and structure properties of a scene. To accurately measure and compare the performance of distinct TMOs, in this paper develop an effective and efficient no-reference objective quality metric which can automatically assess LDR images created by different TMOs without access to the original HDR images. Our model is shown to be statistically superior to recent full- and no-reference quality measures on the existing tone-mapped image database and a new relevant database built in this work.

Objective Quality Assessment of Tone-Mapped Images

Tone-mapping operators (TMOs) that convert high dynamic range (HDR) to low dynamic range (LDR) images provide practically useful tools for the visualization of HDR images on standard LDR displays. Different TMOs create different tone-mapped images, and a natural question is which one has the best quality. Without an appropriate quality measure, different TMOs cannot be compared, and further improvement is directionless. Subjective rating may be a reliable evaluation method, but it is expensive and time consuming, and more importantly, is difficult to be embedded into optimization frameworks. Here we propose an objective quality assessment algorithm for tone-mapped images by combining: 1) a multiscale signal fidelity measure on the basis of a modified structural similarity index and 2) a naturalness measure on the basis of intensity statistics of natural images. Validations using independent subject-rated image databases show good correlations between subjective ranking score and the proposed tone-mapped image quality index (TMQI). Furthermore, we demonstrate the extended applications of TMQI using two examples-parameter tuning for TMOs and adaptive fusion of multiple tone-mapped images.