Tone-mapped High Dynamic Range Images Research Articles

A lightweight exposure bracketing strategy for HDR imaging without access to camera raw

A lightweight learning-based exposure bracketing strategy is proposed in this paper for high dynamic range (HDR) imaging without access to camera RAW. Some low-cost, power-efficient cameras, such as webcams, video surveillance cameras, sport cameras, mid-tier cellphone cameras, and navigation cameras on robots, can only provide access to 8-bit low dynamic range (LDR) images. Exposure fusion is a classical approach to capture HDR scenes by fusing images taken with different exposures into a 8-bit tone-mapped HDR image. A key question is what the optimal set of exposure settings are to cover the scene dynamic range and achieve a desirable tone. The proposed lightweight neural network predicts these exposure settings for a 3-shot exposure bracketing, given the input irradiance information from 1) the histograms of an auto-exposure LDR preview image, and 2) the maximum and minimum levels of the scene irradiance. Without the processing of the preview image streams, and the circuitous route of first estimating the scene HDR irradiance and then tone-mapping to 8-bit images, the proposed method gives a more practical HDR enhancement for real-time and on-device applications. Experiments on a number of challenging images reveal the advantages of our method in comparison with other state-of-the-art methods qualitatively and quantitatively.

Image declipping: Saturation correction in single images

High dynamic range (HDR) images present fine details in a scene and are visually more appealing than low dynamic range (LDR) images, since they contain a greater dynamic range of color gamut. HDR compatible displays are currently high-cost, therefore tone-mapping algorithms have widely been used to obtain high quality images for LDR screens with a lower cost. However, tone-mapped images may contain clipped pixel regions, which should be corrected to retrieve the lost information, to acquire visually pleasing LDR images. In a single image, the recovery of color and texture information in clipped regions is challenging, yet an attractive research field in image processing. Although there are several algorithms present in literature, developing a general framework for different types of image content is hard to achieve. This study proposes a single image declipping method based on linear embeddings, difference of pixels and block-search. Experimental results carried out on a tone-mapped HDR image dataset and LDR images demonstrate that the proposed algorithm is able to successfully recover saturated pixels in various types of images. Detailed statistical and visual comparisons show that this approach produces superior results on average for both tone-mapped and LDR images when compared to existing techniques.

Blind Tone-Mapped HDR Image Quality Measurement by Analysis of Low-Level and High-Level Perceptual Characteristics

Tone-mapping technology aims to transform High Dynamic Range (HDR) images into standard dynamic range images for visualizing on standard screens. However, various visual artifacts will inevitably be induced during the tone-mapping process. To quantify the visual quality degradation for Tone-Mapped Images (TMIs) accurately and automatically, a robust blind TMI quality measurement method is developed by analysis of low-level and high-level perceptual characteristics. Specifically, for low-level visual features, considering that the tone-mapping operator is prone to destroy the local image contrast, which conveys vital structural information, we resort to the joint statistical features based on gradient map and Laplacian of Gaussian response to portray the local image contrast variation in TMIs. In addition, Chromatic Local Binary Pattern (CLBP) is leveraged to measure the colorfulness degradation over four chromatic descriptor maps. For high-level quality-aware features, deep-learned features are utilized by a pre-trained convolutional neural network to characterize the semantics variation. Finally, the extracted low-level and high-level features are combined and mapped into an overall quality score by regression function. Extensive experiments show that our metric achieves a higher performance over two publicly available TMI benchmark databases.

Blind Quality Index for Tone-mapped HDR Images and Multi-Exposure Fused Images

Tone mapping operators (TMOs) and multi-exposure fusion algorithms (MEFs) are employed to visualize the informative contents of high dynamic range (HDR) images on the standard dynamic range (SDR) devices, but also lead to visual quality degradation. Thus, it is urgent to evaluate the perceptual quality of tone-mapped images (TMIs) and multi-exposure fusion images (MEFIs). This paper proposes a blind quality index for TMIs/MEFIs by capturing exposure, colorfulness and structure characteristic features. The connection between feature space and associated subjective ratings is established via a regression model. The proposed method is tested on the public database and compared to the state-of-the-art blind image quality assessment (IQA) methods. The experimental results show that the proposed method achieves superior performance.

Exploiting Local Degradation Characteristics and Global Statistical Properties for Blind Quality Assessment of Tone-Mapped HDR Images

Tone mapping operators (TMOs) are developed to convert a high dynamic range (HDR) image into a low dynamic range (LDR) one for display with the goal of preserving as much visual information as possible. However, image quality degradation is inevitable due to the dynamic range compression during the tone-mapping process. This accordingly raises an urgent demand for effective quality evaluation methods to select a high-quality tone-mapped image (TMI) from a set of candidates generated by distinct TMOs or the same TMO with different parameter settings. A key element to the success of TMI quality evaluation is to extract effective features that are highly consistent with human perception. Towards this end, this paper proposes a novel blind TMI quality metric by exploiting both local degradation characteristics and global statistical properties for feature extraction. Several image attributes including texture, structure, colorfulness and naturalness are considered either locally or globally. The extracted local and global features are aggregated into an overall quality via regression. Experimental results on two benchmark databases demonstrate the superiority of the proposed metric over both the state-of-the-art blind quality models designed for synthetically distorted images (SDIs) and the blind quality models specifically developed for TMIs.

Tone Mapped High Dynamic Range Image Quality Assessment Techniques: Survey and Analysis

The dynamic range of luminance value obtained by natural scenes frequently extent than the images that are captured by recently developed imaging sensors. In this real world, various practical scenarios desire to attain high dynamic range (HDR) illumination. So a large number of HDR imaging techniques were developed during the past decades. But, still there exist some limitations, so we performed survey in this area to overcome these limitations. The consistent methods developed for tone mapped HDR images is found to be meaningful and momentous as this techniques are found to be more beneficial for performing the comparison process for the tone mapping operators (TMOs) and also to advance the improvements in TMO techniques. During past decades, numbers of Image Quality Assessment (IQA) methods have been emerged to estimate the tone mapped HDR images. Recent advancements that are obtained in IQA techniques of tone mapped HDR images are reviewed. These IQA methods are normally categorized into two types they are full reference, and no/blind reference (NR). This type performs the QA process based on the information’s that are obtained from the reference images. NR-IQA is highly recommended by various datasets to assess HDR images as it does not require any perfect reference image for tone mapped HDR image. In this review, we discuss some valuable and interesting IQA methods that are widely employed to assess the image quality. Yet, there exist a prospective to perform a various research works in this field.

High dynamic range image compression based on visual saliency

Abstract With wider luminance range than conventional low dynamic range (LDR) images, high dynamic range (HDR) images are more consistent with human visual system (HVS). Recently, JPEG committee releases a new HDR image compression standard JPEG XT. It decomposes an input HDR image into base layer and extension layer. The base layer code stream provides JPEG (ISO/IEC 10918) backward compatibility, while the extension layer code stream helps to reconstruct the original HDR image. However, this method does not make full use of HVS, causing waste of bits on imperceptible regions to human eyes. In this paper, a visual saliency-based HDR image compression scheme is proposed. The saliency map of tone mapped HDR image is first extracted, then it is used to guide the encoding of extension layer. The compression quality is adaptive to the saliency of the coding region of the image. Extensive experimental results show that our method outperforms JPEG XT profile A, B, C and other state-of-the-art methods. Moreover, our proposed method offers the JPEG compatibility at the same time.

Referenceless Quality Evaluation of Tone-Mapped HDR and Multiexposure Fused Images

Nowadays, the standard dynamic range (SDR) image acquired at a fixed exposure exposes weakness in portraying fine-grained details of real scenes. The high dynamic range (HDR) image and other types of SDR images generated by multiexposure fusion techniques provide us new choices for scene representation. To display on SDR screens, an HDR image must be tone-mapped to an SDR one. Since different tone-mapping/fusion algorithms produce images with varying visual quality levels, it naturally desires a quality evaluation model for comparison. This article proposes an effective model in the absence of the reference image. By analyzing the characteristics of tone-mapped HDR and multiexposure fused images, we first extract multiple quality-sensitive features from the following aspects: 1) colorfulness; 2) exposure; and 3) naturalness. Then, the model is built by bridging all extracted features and associated subjective ratings via support vector regression. Extensive experiments on publicly available databases prove the superiority of our model over the state-of-the-art referenceless quality evaluation ones.

Tone-mapped High Dynamic Range Image Restoration with a Mask-applied Neural Network

Tone-mapped High Dynamic Range Image Restoration with a Mask-applied Neural Network

Palm vein recognition using a high dynamic range approach

In this study, the authors propose a novel approach for palm vein recognition relying on high dynamic range (HDR) imaging. Specifically, the authors speculate that the exploitation of multiple-exposure vein images guarantees better recognition performance than a baseline system relying on single-exposure acquisitions. To verify the authors' assumptions, a multiple-exposure dataset is collected from 86 subjects, with 12 sets of palm vein images captured for each user. Each set is composed of five images, acquired at different exposures, which can be fused to generate a HDR representation of the actual vein pattern. Local binary pattern and local derivative pattern are employed to extract features from single-exposure images, raw HDR images, and tone-mapped HDR images. The obtained experimental results show that significant performance improvement can be achieved when discriminative features are extracted from HDR contents, with respect to the use of single-exposure images.

Combining Global and Local Feature Analyses for Quality Evaluation of Tone-Mapped HDR Images

In existing literature, many tone mapping operators have been designed to convert high dynamic range (HDR) images to low dynamic range (LDR) images for visualization on the standard LDR displays. However, it inevitably causes distortions and artifacts due to the dynamic range compression, resulting in an unpleasant visual experience. In this paper, we propose a no reference (NR) quality evaluation method for tone-mapped (TM) HDR images based on global and local feature analyses. The key strategy of the proposed method lies in measuring the abnormal exposure both globally and locally as well as the halo effects. To be specific, given that the abnormal exposure usually induces the color and brightness changes, the global colorfulness and distribution of brightness are first extracted to quantify the abnormal degree of exposure. Then, since abnormal exposure has diverse influences on local regions, the local features are further extracted from the statistical perspective on the discrete cosine transform domain. Finally, the edge strength is computed locally to measure the halo effects. All extracted quality-sensitive features are combined and trained together with subjective ratings to form a regression model using support vector regression. With extensive experiments, we have shown that the proposed method outmatches several mainstream NR image quality assessment methods designed for both natural scene images and TM HDR images.

Deep Chain HDRI: Reconstructing a High Dynamic Range Image from a Single Low Dynamic Range Image

In this paper, we propose a novel deep neural network model that reconstructs a high dynamic range (HDR) image from a single low dynamic range (LDR) image. The proposed model is based on a convolutional neural network composed of dilated convolutional layers, and infers LDR images with various exposures and illumination from a single LDR image of the same scene. Then, the final HDR image can be formed by merging these inference results. It is relatively easy for the proposed method to find the mapping between the LDR and an HDR with a different bit depth because of the chaining structure inferring the relationship between the LDR images with brighter (or darker) exposures from a given LDR image. The method not only extends the range, but also has the advantage of restoring the light information of the actual physical world. For the HDR images obtained by the proposed method, the HDR-VDP2 Q score, which is the most popular evaluation metric for HDR images, was 56.36 for a display with a 1920$\times$1200 resolution, which is an improvement of 6 compared with the scores of conventional algorithms. In addition, when comparing the peak signal-to-noise ratio values for tone mapped HDR images generated by the proposed and conventional algorithms, the average value obtained by the proposed algorithm is 30.86 dB, which is 10 dB higher than those obtained by the conventional algorithms.

Quality assessment method based on exposure condition analysis for tone-mapped high-dynamic-range images

Tone-mapping operators (TMOs) can be used to transform the data format of high-dynamic-range (HDR) images to traditional low-dynamic-range (LDR) image data format. Hence, the performance of a TMO is crucial for applying HDR images in widely-used LDR image processing systems at this stage. In this work, we propose an exposure condition analysis based quality assessment method for tone-mapped HDR images. Firstly, a purpose-designed HDR exposure segmentation model was used to divide HDR images by analyzing local exposure property. Then, we extracted two new low-complexity quality features (abnormal exposure ratio and exposure residual energy) and a color-based feature in different exposure regions. Finally, the quality assessment model was obtained by regression training. Experiments demonstrate the ability of our method to predict the quality of tone-mapped HDR images. The Pearson linear correlation coefficients are higher than 0.88; thus, the proposed method is highly consistent with human visual perception.

Naturalness index for a tone-mapped high dynamic range image.

High dynamic range (HDR) images can only be backward-compatible with existing low dynamic range (LDR) imaging systems after being processed by tone-mapping operators. Hence, the quality assessment (QA) of tone-mapped HDR images has become an important and challenging issue in HDR imaging research. In this paper, we propose a naturalness index for a tone-mapped image to predict its quality. First, we extract the statistical features of the tone-mapped image's luminance value and use it to evaluate the brightness naturalness with no reference information. Meanwhile, we use perceptive color, image contrast, and detail information to represent the image content and predict their naturalness qualities, respectively. Then, the four components of the naturalness qualities are combined to yield the overall naturalness quality of the tone-mapped image. Experimental results on a publicly available database demonstrated that, in comparison with a traditional LDR image QA method and a leading tone-mapped image QA method, the proposed method has better performance in evaluating a tone-mapped image's quality.

Rendering 3-D High Dynamic Range Images: Subjective Evaluation of Tone-Mapping Methods and Preferred 3-D Image Attributes

High dynamic range (HDR) images provide superior picture quality by allowing a larger range of brightness levels to be captured and reproduced than traditional 8-bit low dynamic range (LDR) images. Even with existing 8-bit displays, picture quality can be significantly improved if content is first captured in HDR format, and then is tone-mapped to convert it from HDR to the LDR format. Tone mapping methods have been extensively studied for 2-D images. This paper addresses the problem of presenting stereoscopic tone-mapped HDR images on 3-D LDR displays and how it is different from the 2-D scenario. We first present a subjective psychophysical experiment that evaluates existing tone-mapping operators on 3-D HDR images. The results show that 3-D content derived using tone-mapping is much preferred to that captured directly with a pair of LDR cameras. Global (spatially invariant) and local (spatially variant) tone-mapping methods have similar 3-D effects. The second part of our study focuses on how the preferred level of brightness and the preferred amount of details differ between 3-D and 2-D images by conducting another set of subjective experiments. Our results show that while people selected slightly brighter images in 3-D viewing compared to 2-D, the difference is not statistically significant. However, compared to 2-D images, the subjects consistently preferred having a greater amount of details when watching 3-D. These results suggest that 3-D content should be prepared differently (sharper and possibly slightly brighter) from the same content intended for 2-D displaying, to achieve optimal appearance in each format. The complete database of the original HDR image pairs and their LDR counterparts are available online.

.MU.-Law Based HDR Coding and Its Error Analysis

In this paper, we propose a coding algorithm for High Dynamic Range Images (HDRI). Our encoder applies a tone mapping model based on scaled µ-Law encoding, followed by a conventional Low Dynamic Range Image (LDRI) encoder. The tone mapping model is designed to minimize the difference between the tone-mapped HDRI and its LDR version. By virtue of the nature of the µ-Law model, not only the quality of the HDRI but also the one of the LDRI is improved, compared with a state of the art in conventional HDRI coding methods. Furthermore the error limit caused by our encoding is theoretically analyzed.

Tone-mapping high dynamic range images by novel histogram adjustment

In this paper, we present novel histogram adjustment methods for displaying high dynamic range image. We first present a global histogram adjustment based tone mapping operator, which well reproduces global contrast for high dynamic range images. We then segment images and carry out adaptive contrast adjustment using our global tone mapping operator in the local regions to reproduce local contrast and ensure better quality. We demonstrate that our methods are fast, easy to use and a fixed set of parameter values produce good results for a wide variety of images.

Beyond Tone Mapping: Enhanced Depiction of Tone Mapped HDR Images

Abstract High Dynamic Range (HDR) images capture the full range of luminance present in real world scenes, and unlike Low Dynamic Range (LDR) images, can simultaneously contain detailed information in the deepest of shadows and the brightest of light sources. For display or aesthetic purposes, it is often necessary to perform tone mapping, which creates LDR depictions of HDR images at the cost of contrast information loss. The purpose of this work is two‐fold: to analyze a displayed LDR image against its original HDR counterpart in terms of perceived contrast distortion, and to enhance the LDR depiction with perceptually driven colour adjustments to restore the original HDR contrast information. For analysis, we present a novel algorithm for the characterization of tone mapping distortion in terms of observed loss of global contrast, and loss of contour and texture details. We classify existing tone mapping operators accordingly. We measure both distortions with perceptual metrics that enable the automatic and meaningful enhancement of LDR depictions. For image enhancement, we identify artistic and photographic colour techniques from which we derive adjustments that create contrast with colour. The enhanced LDR image is an improved depiction of the original HDR image with restored contrast information. Categories and Subject Descriptors (according to ACM CCS): I.3.3 [Computer Graphics]: Picture/Image Generation I.4.0 [Image Processing and Computer Vision]: GeneralImage processing software