Tone Mapping Research Articles

Meta underwater camera: A smart protocol for underwater image enhancement

Underwater images captured by an underwater camera normally suffer from visual degradation issues, such as color deviations, low contrasts, and detail blurs. Existing studies tend to address these issues separately by individual techniques, such that underwater image visibility can hardly be improved in an overall, consistent manner. To address this limitation, we present a smart protocol for underwater image enhancement. The protocol comprises a comprehensive cascade of seven image enhancement techniques, i.e., attenuated channel compensation, white balance, tone mapping, hue-saturation-lightness (HSL) model-based saturation adjustment, contrast stretching, gamma correction, and high-pass fusion. The protocol is smartly configured by reinforcement learning. Specifically, a set of underwater image multi-color space features are considered as a state, a set of parameter values for the protocol as an action, and an increment of a non-reference visual preference score as a reward. By the reinforcement learning strategy, the parameter values for the seven techniques in the protocol are optimally configured as a whole, resulting in optimal underwater image enhancement results. We refer to the functionality of the smart protocol as a meta underwater camera (MUC) methodology because it operates behind an underwater camera’s operation of capturing underwater images but provides underwater images of high visibility beyond those of low visibility originally obtained from the underwater camera. Qualitative and quantitative empirical results validate that our MUC methodology outperforms state-of-the-art underwater image enhancement methods. We release the reproducible code at https://gitee.com/wanghaoupc/MUC_UnderwaterImageEnhancement for public evaluations.

Efficient tone mapping via macro and micro information enhancement and color correction

Efficient tone mapping via macro and micro information enhancement and color correction

Perceptual Tone Mapping Model for High Dynamic Range Imaging

Perceptual Tone Mapping Model for High Dynamic Range Imaging

Tone Mapping Method Based on the Least Squares Method

Tone mapping is used to compress the dynamic range of image data without distortion. To compress the dynamic range of HDR images and prevent halo artifacts, a tone mapping method is proposed based on the least squares method. Our method first uses weights for the estimation of the illumination, and the image detail layer is obtained by the Retinex model. Then, a global tone mapping function with the parameter is used to compress the dynamic range, and the parameter is obtained by fitting the function to the histogram equalization. Finally, the detail layer and the illumination layer are fused to obtain the LDR image. The experimental results show that the proposed method can efficiently restore real-world scene information while preventing halo artifacts. Therefore, tone mapping quality index and mean Weber contrast of the tone-mapped image are 8% and 12% higher than the closest competition tone mapping method.

Wavelet based tone mapping (TM) enhancement to a detection system for faint and compact sources in HDR and large FOV radio scenes

For high dynamic range (HDR) and large field-of-view (FOV) radio scenes, detection of faint sources with compact support and tiny sizes in comparatively extensive celestial areas is a challenging task. In the era of the Square Kilometer Array (SKA), it will be of particular concern, and data-targeted systematic methods are put on the agenda. This paper proposes a systematic method to solve this problem. From the perspective of image processing, a multiscale image contrast enhancement (ICE) is proposed to normalize the HDR. Source representation is realized by a starlet framework, where a two-step tone mapping (TM) including an arc-tangent function compression and an aggregated gain function enhancement are embedded. According to two bionic principles, the gain function is designed to highlight faint sources, remove artifacts (e.g., halos), and avoid feature distortions. For source detection, an integration method is proposed for projections between the celestial and the planar coordinate systems. It serves for coordinate transform and source location. A detection indicator system is built, and elaborate discrimination principles are drawn up. In the experiments, this method is dedicated to the dataset of the TIFR GMRT Sky Survey (TGSS) alternative data release 1 (ADR1) to facilitate the scientific goal of discovering new sources. Results show its advantages over the state of the art methods in visual inspections and detection ratios, and preliminarily prove that it has prospects for future HDR and large FOV radio data.

TM-Net: A Neural Net Architecture for Tone Mapping.

Tone mapping functions are applied to images to compress the dynamic range of an image, to make image details more conspicuous, and most importantly, to produce a pleasing reproduction. Contrast Limited Histogram Equalization (CLHE) is one of the simplest and most widely deployed tone mapping algorithms. CLHE works by iteratively refining an input histogram (to meet certain conditions) until convergence, then the cumulative histogram of the result is used to define the tone map that is used to enhance the image. This paper makes three contributions. First, we show that CLHE can be exactly formulated as a deep tone mapping neural network (which we call the TM-Net). The TM-Net has as many layers as there are refinements in CLHE (i.e., 60+ layers since CLHE can take up to 60 refinements to converge). Second, we show that we can train a fixed 2-layer TM-Net to compute CLHE, thereby making CLHE up to 30× faster to compute. Thirdly, we take a more complex tone-mapper (that uses quadratic programming) and show that it too can also be implemented - without loss of visual accuracy-using a bespoke trained 2-layer TM-Net. Experiments on a large corpus of 40,000+ images validate our methods.

Three-Stage Tone Mapping Algorithm

In this paper, a tone mapping algorithm is presented to map real-world luminance into displayed luminance. Our purpose is to reveal the local contrast of real-world scenes on a conventional monitor. Around this point, we propose a three-stage algorithm to visualize high dynamic range images. All pixels of high dynamic range images are classified into three groups. For the first stage, we introduce piecewise linear mapping as the global tone mapping operator to map the luminance of the first group, which provides overall impressions of luminance. For the second stage, the luminance of the second group is determined by the weighted average of its neighborhood pixels, which are derived from the first group’s pixels. For the third stage, the luminance of the third group is determined by the weighted average of its neighborhood pixels, which are derived from the second group’s pixels. Experimental results on several real-world images and the TMQI database show that our algorithm can improve the visibility of real-world scenes with about 12% and 9% higher scores of mean opinion score and tone-mapped image quality index than the closest competitive tone mapping methods. Compared to the existing tone mapping methods, our algorithm produces visually compelling results without halo artifacts and loss of detail.

DRBR-HDR: Dual-Branch recursive band reconstruction network for HDR with large motions

Ghosting artifacts due to misaligned imaging and missing content of the moving regions are major challenges of synthesizing high dynamic range (HDR) images from multiple low-dynamic range (LDR) with different exposures in dynamic scenes. Therefore, it hopes the HDR reconstruction model can align the LDRs’ features and restore the missing content without artifacts. In the paper, a new dual-branch recursive band reconstruction network for high dynamic range (DRBR-HDR) is proposed to generate credible result in missing content regions, which not only uses global features as supplementary information to help local features from different receptive fields for efficient feature alignment but also designs a series of coarse-to-fine band representation to better repair missing areas in the process of recursion. In addition, we introduce an interactive attention mechanism for local branches to alleviate ghosting artifacts. The experimental results demonstrate that DRBR-HDR achieves state-of-the-art performance compared with that of the prevailing HDR reconstruction methods in various challenging scenes.Index Terms—inverse tone mapping, band reconstruction, global features, high dynamic range images.

Text2Light

High-quality HDRIs (High Dynamic Range Images), typically HDR panoramas, are one of the most popular ways to create photorealistic lighting and 360-degree reflections of 3D scenes in graphics. Given the difficulty of capturing HDRIs, a versatile and controllable generative model is highly desired, where layman users can intuitively control the generation process. However, existing state-of-the-art methods still struggle to synthesize high-quality panoramas for complex scenes. In this work, we propose a zero-shot text-driven framework, Text2Light , to generate 4K+ resolution HDRIs without paired training data. Given a free-form text as the description of the scene, we synthesize the corresponding HDRI with two dedicated steps: 1 ) text-driven panorama generation in low dynamic range (LDR) and low resolution (LR), and 2 ) super-resolution inverse tone mapping to scale up the LDR panorama both in resolution and dynamic range. Specifically, to achieve zero-shot text-driven panorama generation, we first build dual codebooks as the discrete representation for diverse environmental textures. Then, driven by the pre-trained Contrastive Language-Image Pre-training (CLIP) model, a text-conditioned global sampler learns to sample holistic semantics from the global codebook according to the input text. Furthermore, a structure-aware local sampler learns to synthesize LDR panoramas patch-by-patch, guided by holistic semantics. To achieve super-resolution inverse tone mapping, we derive a continuous representation of 360-degree imaging from the LDR panorama as a set of structured latent codes anchored to the sphere. This continuous representation enables a versatile module to upscale the resolution and dynamic range simultaneously. Extensive experiments demonstrate the superior capability of Text2Light in generating high-quality HDR panoramas. In addition, we show the feasibility of our work in realistic rendering and immersive VR.

Constant hue loci in different color spaces for stimuli in Rec. 2020 color gamut and HDR conditions.

Hue is an important attribute for characterizing a color stimulus, which is also an output in various color spaces. The investigations on the hue linearity and constant hue loci for different color spaces were generally conducted using conventional CRT displays or surface color samples, in which the color stimuli were within small color gamuts and viewed under standard dynamic range conditions. With the development of imaging technologies, the hue linearity and constant hue loci need to be investigated for wide color gamuts and high dynamic range conditions, which is critically important for image processing (e.g., gamut mapping and tone mapping). In this study, we carefully carried out a hue matching experiment using high-power LED devices. The color stimuli almost reached Rec. 2020 color gamut with the luminance above the diffuse white luminance (i.e., a high dynamic range condition). The results suggested that the hue linearity of ICTCP color space was the best among the nine color spaces. Twenty-one constant hue loci were derived for each of these nine color spaces, which can be used for hue correction when performing image processing and to further revise the color spaces.

Nonconvex regularization for convex image smoothing

Image smoothing is a fundamental part of many computer vision and graphics. In this paper, to improve the sparsity of the gradient norm of the smoothing image, we use nonconvex penalty functions as regularization terms. A condition on nonconvex regularizers is given to ensure that the objective functions are strictly convex. We apply the powerful majorization-minimization (MM) algorithm for solving the convex image smoothing models. Proof of global convergence for the MM algorithm is provided. The superiority of the proposed method in terms of peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and visual quality is verified through comprehensive experiments in a variety of applications such as image abstraction, artifact removal, detail enhancement, texture removal, image denoising and high dynamic range (HDR) tone mapping.

INTERVAL QUALITY INDICATORS OF THE DYNAMIC RANGE COMPRESSION OF INFRARED IMAGES ON THE BASIS OF A TONE MAPPING MATRIX

The article proposes a dynamic range compression model for infrared (IR) images based on a tone mapping
 matrix, the elements of which relate the brightness levels of the original image with a wide dynamic range and
 the brightness levels of a non-linearly transformed image with a narrow dynamic range, and also indicate, depending
 on the variant of formation of this matrices on: a) loss of discrimination between adjacent pixels due
 to compression of the dynamic range; b) the level of non-linear compression distortions; c) ambiguity of tone
 mapping. Based on this model, interval indicators of the quality of compression of the dynamic range of infrared
 images are proposed, which allow estimating the potential distinguishing power, the real loss of discrimination
 between adjacent pixels after transformation, the magnitude of nonlinear compression distortions, the uniformity
 of the use of the dynamic range, and the ambiguity of tone mapping for the selected interval of the dynamic range.
 The proposed indicators improve the accuracy of assessing the quality of compression of the dynamic range of IR
 images by expanding the system of known indicators that evaluate the contrast, entropy, statistical naturalness
 of the converted images, and the structural accuracy of tone mapping.

Color Appearance Characterization of Highlight Stimuli in HDR Scenes Across a Wide Range of Diffuse White Luminance

Various uniform color spaces and color appearance models were mainly developed for characterizing stimuli under a low dynamic range condition. Real scenes in daily life, however, are commonly high dynamic range (HDR), containing highlights with luminance beyond the diffuse white, whose color appearance characterization was never investigated in the past. This study was carefully designed to investigate the color appearance characterization of highlights in HDR scenes, covering extremely wide ranges of diffuse white luminance (up to 11000 cd/m<sup>2</sup>), stimulus luminance (up to 49000 cd/m<sup>2</sup>), stimulus chromaticities (reach Rec. 2020 gamut), and scene luminance contrast (up to 72045). The observers viewed two stimuli, including one highlight and one dark stimulus, in a viewing booth, and were asked to adjust the color appearance of another stimulus, so that the color differences between the adjusted stimulus to each of the other two stimuli appeared the same. The results clearly showed that none of the existing models, including the one (i.e., IC<sub>t</sub>C<sub>p</sub>) that was recently designed for HDR scenes, has a good performance. The models using a power function to characterize the non-linear compressive responses of the human visual system (i.e., CIELAB and IPT) had a slightly better performance. The findings provided some guidance for performing tone mapping and chroma/saturation adjustments, and clearly suggest the necessity to carry out further work to develop a better model for HDR scenes.

GiTMO: An HDR10 Compatible Generic Tone-Mapping for Real-Time Video Play

Though the market of HDR displays for consumers is gaining maturity, low dynamic range (LDR) displays still hold a majority share in the consumer market. Many tone-mapping operators (TMO) have been developed to convert the HDR contents into a form suitable for LDR displays. This work proposes a generic tone-mapping operator (GiTMO) that can represent all global TMOs in a generic form of a one-dimensional array of a few coefficients. We use these coefficients and the existing HDR10 delivery pipeline to produce a tone-mapped version of the HDR content at the receiver end in real time. In our framework, coefficients are sent to the receiver as supplemental enhancement information (SEI) message of High Efficiency Video Coding (HEVC). Thus, HDR10, which is designed for HDR display, becomes compatible with LDR displays too. This can relieve the content producers from the need to release separate copies of HDR and LDR content. GiTMO, being generic and efficient, can generate LDR faithfully for all global TMOs, making the consumer end oblivious of the TMO used at the transmitter side. We perform extensive evaluations to validate the accuracy and efficiency of the GiTMO and the proposed content delivery format, and their benefits over the existing approaches.

Learning Frequency Domain Priors for Image Demoireing

Image demoireing is a multi-faceted image restoration task involving both moire pattern removal and color restoration. In this paper, we raise a general degradation model to describe an image contaminated by moire patterns, and propose a novel multi-scale bandpass convolutional neural network (MBCNN) for single image demoireing. For moire pattern removal, we propose a multi-block-size learnable bandpass filters (M-LBFs), based on a block-wise frequency domain transform, to learn the frequency domain priors of moire patterns. We also introduce a new loss function named Dilated Advanced Sobel loss (D-ASL) to better sense the frequency information. For color restoration, we propose a two-step tone mapping strategy, which first applies a global tone mapping to correct for a global color shift, and then performs local fine tuning of the color per pixel. To determine the most appropriate frequency domain transform, we investigate several transforms including DCT, DFT, DWT, learnable non-linear transform and learnable orthogonal transform. We finally adopt the DCT. Our basic model won the AIM2019 demoireing challenge. Experimental results on three public datasets show that our method outperforms state-of-the-art methods by a large margin.

P‐7.13: Enhanced High Dynamic Tone Mapping Using Guided Filter

P‐7.13: Enhanced High Dynamic Tone Mapping Using Guided Filter

6.2: Visual perception based non‐reference binocular tone mapping algorithm

Binocular tone mapping aims to improve the overall quality of the image erceived by the human eye, and has been continuously optimized since it was proposed. Based on visual perception experiment, a series of visual perception experiments are designed for the binocular perception ability of human eyes. On this basis, a visual perception based non-reference binocular tone mapping algorithm is proposed. The algorithm has a more streamlined and closer to the human visual perception framework, and is more adaptive in practice.

L0 image smoothing via iterating truncated L1 gradient regularization

Edge-preserving image smoothing is a fundamental tool in computational photography and graphics. It aims to suppress insignificant details while maintaining salient structures. The classical L0 filter provides an elegant framework for a variety of applications. However, it inclines to sharpen the salient edges, thus suffering from gradient reversals and color deviations. We propose a solution toward the objective of optimizing summed squared error regularized by the L0-norm of the gradients. The proposed solution explores an iterative strategy, where each iteration is an optimization problem based on the truncated L1 regularization, which can be solved efficiently with the combination of alternating direction method of multipliers and Fourier domain optimization. Leveraging L1-norm limits the aggressive modifications of gradients during the iterative process, which alleviates various artifacts in classical L0 filter. Experiment results indicate that the proposed method achieves superior performance in various applications, including texture removal, detail enhancement, HDR tone mapping, and compression artifact removal. The proposed filter can be conveniently implemented on GPU. It takes our filter 0.51 s to process a 720P color image on a NVIDIA GTX 1080 GPU.

No-reference quality index of tone-mapped images based on authenticity, preservation, and scene expressiveness

As a future-proof technology, high dynamic range (HDR) has the superiority in scene reappearance. Tone mapping operator (TMO) renders the dynamic effect of HDR images in a standard dynamic range (SDR) display at the expense of some insignificant information. However, it is difficult for existing TMOs to consistently achieve high-quality conversion of all HDR images, and the loss of converted images is inevitable. Therefore, accurate prediction of tone-mapped image (TMI) quality is of considerable significance. In this paper, we propose the reappearance effect of the TMIs index (RETI). According to the characteristics of primeval HDR images, the quality of TMIs is considered from three important elements: authenticity, the preservation of energy and information, and scene expressiveness. The feature vectors originated from three elements are combined with subjective quality to train prediction model. Experiments are executed on tow mainstream TMI quality assessment databases and the results show that the proposed method has a good prediction ability and generalization ability in comparison to some state-of-the-art methods. Our MATLAB source code will be released at https://github.com/niannianyouyu/RETI.

Cyclic Learning-Based Lightweight Network for Inverse Tone Mapping

Recent studies on inverse tone mapping (iTM) have moved toward indirect mapping, which generates a stack of low dynamic range (LDR) images with multiple exposure values (multi-EV stack) and then merges them. In order to generate multi-EV stack(s), several large-scale networks with more than 20 M parameters have been proposed, but their high dynamic range (HDR) reconstruction and multi-EV stack generation performance were not acceptable. Also, some previous methods using cycle consistency should even have trained additional networks that are not used for multi-EV stack generation, which results in large memory for training. Thus, this paper proposes novel cyclic learning based on cycle consistency to reduce the memory burden in training. In detail, we eliminated networks used only for training, so the proposed method enables efficient learning in terms of training-purpose memory. In addition, this paper presents a lightweight iTM network that dramatically reduces the network sizes of the existing networks. Actually, the proposed lightweight network requires only a small parameter size of 1/100 compared to the state-of-the-art (SOTA) method. The lightweight network contributes to the practical use of iTM. Therefore, the proposed method based on a lightweight network reliably generates a multi-EV stack. Experimental results show that the proposed method achieves quantitatively SOTA performance and is qualitatively comparable to conventional indirect iTM methods.