Single Low Dynamic Range Image Research Articles

Color Event Enhanced Single-Exposure HDR Imaging

Single-exposure high dynamic range (HDR) imaging aims to reconstruct the wide-range intensities of a scene by using its single low dynamic range (LDR) image, thus providing significant efficiency. Existing methods pay high attention to restoring the luminance by inversing the tone-mapping process, while the color in the over-/under-exposed area cannot be well restored due to the information loss of the single LDR image. To address this issue, we introduce color events into the imaging pipeline, which record asynchronous pixel-wise color changes in a high dynamic range, enabling edge-like scene perception under challenging lighting conditions. Specifically, we propose a joint framework that incorporates color events and a single LDR image to restore both content and color of an HDR image, where an exposureaware transformer (EaT) module is designed to propagate the informative hints, provided by the normal-exposed LDR regions and the event streams, to the missing areas. In this module, an exposure-aware mask is estimated to suppress distractive information and strengthen the restoration of the over-/under-exposed regions. To our knowledge, we are the first to use color events to enhance single-exposure HDR imaging. We also contribute corresponding datasets, consisting of synthesized datasets and a real-world dataset collected by a DAVIS346-color camera. The datasets can be found at https://www.kaggle.com/datasets/mengyaocui/ce-hdr. Extensive experiments demonstrate the effectiveness of the proposed method.

Hybrid High Dynamic Range Imaging fusing Neuromorphic and Conventional Images.

Reconstruction of high dynamic range image from a single low dynamic range image captured by a conventional RGB camera, which suffers from over- or under-exposure, is an ill-posed problem. In contrast, recent neuromorphic cameras like event camera and spike camera can record high dynamic range scenes in the form of intensity maps, but with much lower spatial resolution and no color information. In this paper, we propose a hybrid imaging system (denoted as NeurImg) that captures and fuses the visual information from a neuromorphic camera and ordinary images from an RGB camera to reconstruct high-quality high dynamic range images and videos. The proposed NeurImg-HDR+ network consists of specially designed modules, which bridges the domain gaps on resolution, dynamic range, and color representation between two types of sensors and images to reconstruct high-resolution, high dynamic range images and videos. We capture a test dataset of hybrid signals on various HDR scenes using the hybrid camera, and analyze the advantages of the proposed fusing strategy by comparing it to state-of-the-art inverse tone mapping methods and merging two low dynamic range images approaches. Quantitative and qualitative experiments on both synthetic data and real-world scenarios demonstrate the effectiveness of the proposed hybrid high dynamic range imaging system. Code and dataset can be found at: https://github.com/hjynwa/NeurImg-HDR.

Fast and flexible stack‐based inverse tone mapping

AbstractInverse tone mapping technique is widely used to restore the lost textures from a single low dynamic range image. Recently, many stack‐based deep inverse tone mapping networks have achieved impressive results by estimating a set of multi‐exposure images from a single low dynamic range input. However, there are still some limitations. On the one hand, these methods usually set a fixed length for the estimated multi‐exposure stack, which may introduce computational redundancy or cause inaccurate results. On the other hand, they neglect that the difficulties of estimating each exposure value are different and use the identical model to increase or decrease exposure value. To solve these problems, the authors design an exposure decision network to adaptively determine the number of times the exposure of low dynamic range input should be increased or decreased. Meanwhile, the authors decouple the increasing/decreasing process into two sub‐modules, exposure adjustment and optional detail recovery, based on the characteristics of different variations of exposure values. With these improvements, this method can fast and flexibly estimate the multi‐exposure stack from a single low dynamic range image. Experiments on several datasets demonstrate the advantages of the proposed method compared to state‐of‐the‐art inverse tone mapping methods.

High dynamic range imaging by a pseudo exposure fusion method based on artificial remapping

High dynamic range imaging is a promising image enhancement technology, which can reconstruct a high dynamic range (HDR) image with a real visual perception from captured one (or several) raw low dynamic range (LDR) image. However, the input LDR image only provides partial information of the scene. Besides, in traditional methods that require multiple captured images as input, it is prone to halos and/or ghosting artifacts due to camera shake or object motion. In this paper, we propose a simple yet valid method that firstly uses artificial remapping to generate a set of simulated multi-exposure image sequence from a single LDR image, then fuses them into a high-quality HDR image via multiscale pyramid framework-based exposure fusion. It was demonstrated that artifacts-free and details-preserving HDR image can be achieved. The method thus can be potentially applied to the field of high dynamic range imaging, and can be further used for astronomical exploration, biomedical imaging, and optical imaging fields in the future.

LiTMNet: A deep CNN for efficient HDR image reconstruction from a single LDR image

Existing methods can generate a high dynamic range (HDR) image from a single low dynamic range (LDR) image using convolutional neural networks (CNNs). However, they are too cumbersome to run on mobile devices with limited computational resources. In this work, we design a lightweight CNN, namely LiTMNet which takes a single LDR image as input and recovers the lost information in its saturated regions to reconstruct an HDR image. To avoid trading off the reconstruction quality for efficiency, LiTMNet does not only adapt a lightweight encoder for efficient feature extraction, but also contains newly designed upsampling blocks in the decoder to alleviate artifacts and further accelerate the reconstruction. The final HDR image is produced by nonlinearly blending the network prediction and the original LDR image. Qualitative and quantitative comparisons demonstrate that LiTMNet produces HDR images of high quality comparable with the current state of the art and is 38× faster as tested on a mobile device. Please refer to the supplementary video for additional visual results.

HSVNet: Reconstructing HDR Image from a Single Exposure LDR Image with CNN

Most photographs are low dynamic range (LDR) images that might not perfectly describe the scene as perceived by humans due to the difference in dynamic ranges between photography and natural scenes. High dynamic range (HDR) images have been used widely to depict the natural scene as accurately as possible. Even though HDR images can be generated by an exposure bracketing method or HDR-supported cameras, most photos are still taken as LDR due to annoyance. In this paper, we propose a method that can produce an HDR image from a single arbitrary exposure LDR image. The proposed method, HSVNet, is a deep learning architecture using a Convolutional Neural Networks (CNN) based U-net. Our model uses the HSV color space that enables the network to identify saturated regions and adaptively focus on crucial components. We generated a paired LDR-HDR image dataset of diverse scenes including under/oversaturated regions for training and testing. We also show the effectiveness of our method through experiments, compared to existing methods.

Deep Arbitrary HDRI: Inverse Tone Mapping With Controllable Exposure Changes

Deep convolutional neural networks (CNNs) have recently made significant advances in the inverse tone mapping technique, which generates a high dynamic range (HDR) image from a single low dynamic range (LDR) image that has lost information in over- and under-exposed regions. The end-to-end inverse tone mapping approach specifies the dynamic range in advance, thereby limiting dynamic range expansion. In contrast, the method of generating multiple exposure LDR images from a single LDR image and subsequently merging them into an HDR image enables flexible dynamic range expansion. However, existing methods for generating multiple exposure LDR images require an additional network for each exposure value to be changed or a process of recursively inferring images that have different exposure values. Therefore, the number of parameters increases significantly due to the use of additional networks, and an error accumulation problem arises due to recursive inference. To solve this problem, we propose a novel network architecture that can control arbitrary exposure values without adding networks or applying recursive inference. The training method of the auxiliary classifier-generative adversarial network structure is employed to generate the image conditioned on the specified exposure. The proposed network uses a newly designed spatially-adaptive normalization to address the limitation of existing methods that cannot sufficiently restore image detail due to the spatially equivariant nature of the convolution. Spatially-adaptive normalization facilitates restoration of the high frequency component by applying different normalization parameters to each element in the feature map according to the characteristics of the input image. Experimental results show that the proposed method outperforms state-of-the-art methods, yielding a 5.48dB higher average peak signal-to-noise ratio, a 0.05 higher average structure similarity index, a 0.28 higher average multi-scale structure similarity index, and a 7.36 higher average HDR-VDP-2 for various datasets.

Object-based illumination transferring and rendering for applications of mixed reality

In applications of augmented reality or mixed reality, rendering virtual objects in real scenes with consistent illumination is crucial for realistic visualization experiences. Prior learning-based methods reported in the literature usually attempt to reconstruct complicated high dynamic range environment maps from limited input, and rely on a separate rendering pipeline to light up the virtual object. In this paper, an object-based illumination transferring and rendering algorithm is proposed to tackle this problem within a unified framework. Given a single low dynamic range image, instead of recovering lighting environment of the entire scene, the proposed algorithm directly infers the relit virtual object. It is achieved by transferring implicit illumination features which are extracted from its nearby planar surfaces. A generative adversarial network is adopted in the proposed algorithm for implicit illumination features extraction and transferring. Compared to previous works in the literature, the proposed algorithm is more robust, as it is able to efficiently recover spatially varying illumination in both indoor and outdoor scene environments. Experiments have been conducted. It is observed that notable experiment results and comparison outcomes have been obtained quantitatively and qualitatively by the proposed algorithm in different environments. It shows the effectiveness and robustness for realistic virtual object insertion and improved realism.

Luminance Attentive Networks for HDR Image and Panorama Reconstruction

AbstractIt is very challenging to reconstruct a high dynamic range (HDR) from a low dynamic range (LDR) image as an ill‐posed problem. This paper proposes a luminance attentive network named LANet for HDR reconstruction from a single LDR image. Our method is based on two fundamental observations: (1) HDR images stored in relative luminance are scale‐invariant, which means the HDR images will hold the same information when multiplied by any positive real number. Based on this observation, we propose a novel normalization method called “HDR calibration “for HDR images stored in relative luminance, calibrating HDR images into a similar luminance scale according to the LDR images. (2) The main difference between HDR images and LDR images is in under‐/over‐exposed areas, especially those highlighted. Following this observation, we propose a luminance attention module with a two‐stream structure for LANet to pay more attention to the under‐/over‐exposed areas. In addition, we propose an extended network called panoLANet for HDR panorama reconstruction from an LDR panorama and build a dualnet structure for panoLANet to solve the distortion problem caused by the equirectangular panorama. Extensive experiments show that our proposed approach LANet can reconstruct visually convincing HDR images and demonstrate its superiority over state‐of‐the‐art approaches in terms of all metrics in inverse tone mapping. The image‐based lighting application with our proposed panoLANet also demonstrates that our method can simulate natural scene lighting using only LDR panorama. Our source code is available at https://github.com/LWT3437/LANet.

Extending and Matching a High Dynamic Range Image from a Single Image.

Extending the dynamic range can present much richer contrasts and physical information from the traditional low dynamic range (LDR) images. To tackle this, we propose a method to generate a high dynamic range image from a single LDR image. In addition, a technique for the matching between the histogram of a high dynamic range (HDR) image and the original image is introduced. To evaluate the results, we utilize the dynamic range for independent image quality assessment. It recognizes the difference in subtle brightness, which is a significant role in the assessment of novel lighting, rendering, and imaging algorithms. The results show that the picture quality is improved, and the contrast is adjusted. The performance comparison with other methods is carried out using the predicted visibility (HDR-VDP-2). Compared to the results obtained from other techniques, our extended HDR images can present a wider dynamic range with a large difference between light and dark areas.

Combining highlight removal and low‐light image enhancement technique for HDR‐like image generation

Low dynamic range (LDR) image may contain low-light and highlight areas due to the limitations of the dynamic range of conventional image sensors. Low-light and highlight phenomena limit colour richness and visibility of objects in an image. Therefore, it can cause a reduction in the quality of images and a loss in accuracy in the application of image recognition. To overcome this, high dynamic range (HDR)-like images have been developed with rich colours such as those seen by the human eye. In this study, the authors propose a method to obtain an HDR-like image from a single LDR image by removing the specular component from highlight pixels as well as strengthening the actual colour. Next, they select low-light image enhancement via illumination map estimation as a low-light enhancement technique by showing the comparison with gamma-based expansion operator. They evaluate their HDR-like output images with non-reference and full-reference metrics. They show the comparison of their proposed method with six other methods. Besides, visually, their proposed method delivers more pleasing output than the output of other competitive methods.

Dynamic Range Expansion Using Cumulative Histogram Learning for High Dynamic Range Image Generation

In modern digital photographs, most images have low dynamic range (LDR) formats, which means that the range of light intensities from the darkest to the brightest is much lower than the range that can be perceived by the human eye. Therefore, to visualize images as naturally as possible on devices that display them in high dynamic range (HDR) format, the LDR images need to be converted into HDR images. The aim of this study was to develop an adaptive inverse tone mapping operator (iTMO) that can convert a single LDR image into a realistic HDR image based on artificial neural networks. In contrast to conventional iTMO algorithms, our technique was developed by learning the complicated relationship between various LDR–HDR pair images, which enabled nearly ground-truth HDR images to be generated from various types of LDR images. The novel learning technique is called cumulative histogram-based learning and color difference learning. The superior performance of our technique over conventional methods was assessed through objective evaluations of various types of LDR and HDR images.

An optimization framework for inverse tone mapping using a single low dynamic range image

Conventional inverse tone-mapping (ITM) methods tend to produce contrast distortions such as contrast loss and contrast reversal in reconstructed high dynamic range (HDR) images. This paper proposes a novel ITM optimization framework based on the assumption that the input low dynamic range (LDR) image is similar to the LDR image obtained by tone mapping a true HDR image. In the proposed framework, an HDR image is initially reconstructed by applying a conventional tone-mapping function in a reverse manner, and then the reconstructed HDR image is iteratively modified toward the optimum HDR image by minimizing the difference between the input LDR image and a tone-mapped LDR image obtained from the reconstructed HDR image. The experimental results demonstrate that the proposed framework effectively reconstructs a high-quality HDR image and outperforms other conventional methods in terms of objective quality.

HLS Based Approach to Develop an Implementable HDR Algorithm

Hardware suitability of an algorithm can only be verified when the algorithm is actually implemented in the hardware. By hardware, we indicate system on chip (SoC) where both processor and field-programmable gate array (FPGA) are available. Our goal is to develop a simple algorithm that can be implemented on hardware where high-level synthesis (HLS) will reduce the tiresome work of manual hardware description language (HDL) optimization. We propose an algorithm to achieve high dynamic range (HDR) image from a single low dynamic range (LDR) image. We use highlight removal technique for this purpose. Our target is to develop parameter free simple algorithm that can be easily implemented on hardware. For this purpose, we use statistical information of the image. While software development is verified with state of the art, the HLS approach confirms that the proposed algorithm is implementable to hardware. The performance of the algorithm is measured using four no-reference metrics. According to the measurement of the structural similarity (SSIM) index metric and peak signal-to-noise ratio (PSNR), hardware simulated output is at least 98.87 percent and 39.90 dB similar to the software simulated output. Our approach is novel and effective in the development of hardware implementable HDR algorithm from a single LDR image using the HLS tool.

Illumination consistency based on single low dynamic range images

In this paper, a light consistency solution for generating high dynamic range (HDR) images based on a single low dynamic range image(LDR) is proposed, and the virtual object is rendered by illumination. The solution can reduce the time of image acquisition and processing, and solve the problems caused by the limitations of image acquisition equipment. The solution is divided into three stages: image preprocessing, high dynamic range image generation and virtual object relighting. Firstly, in the stage of image pretreatment, the wavelet noise reduction method based on a Gaussian mixture model is used to remove image noise and avoid image detail distortion. The inverse camera response function is utilized to linearize the image, the pixel brightness range is expanded based on the inverse tone mapping function, and the threshold segmentation method is combined with flooding Gaussian smoothing to calculate the highlight spread diagram to compensate for scene highlights lost during camera shooting. Then, the extended dynamic range image is interpolated linearly by using the specular expansion image to get the high dynamic range image. Based on the analysis and experimental simulation, compared with other methods, it is found that using a single low-dynamic-range image can greatly reduce the time of image acquisition and processing and reduce the limitations of image acquisition equipment, while maintaining good light fusion. Based on the simulation results, the efficiency of light consistency processing is improved.

Perceptual brightness-based inverse tone mapping for high dynamic range imaging

This paper proposes a novel high dynamic range imaging method based on inverse tone mapping for generating a high dynamic range image from a single low dynamic range image. With the increasing expectations for image quality, displays providing high dynamic range images have been released. However, although low dynamic range images are currently commonly used, they do not use the high representation capability of high dynamic range displays. Moreover, high dynamic range contents are scarce. Therefore, high dynamic range imaging techniques are gaining attention. The proposed method involves adaptive specular region detection to determine the bright and dark areas for high dynamic range imaging, output mapping range decision to consider perceptual brightness, final mapping-level decision to prevent degradation of perceptual image quality, and inverse pixel mapping for luminance mapping. The simulation results of the proposed method yielded an HDR-VDP-2 quality score of 59.51, which was higher than those of the conventional methods. Furthermore, the execution time of the proposed algorithm was 0.66 s, which had the lowest complexity compared to the conventional methods. For subjective performance evaluation, we analyzed the perceived image quality on the display with the DRIM, which confirmed the superiority of the proposed method over the conventional methods.

Deep Chain HDRI: Reconstructing a High Dynamic Range Image from a Single Low Dynamic Range Image

In this paper, we propose a novel deep neural network model that reconstructs a high dynamic range (HDR) image from a single low dynamic range (LDR) image. The proposed model is based on a convolutional neural network composed of dilated convolutional layers, and infers LDR images with various exposures and illumination from a single LDR image of the same scene. Then, the final HDR image can be formed by merging these inference results. It is relatively easy for the proposed method to find the mapping between the LDR and an HDR with a different bit depth because of the chaining structure inferring the relationship between the LDR images with brighter (or darker) exposures from a given LDR image. The method not only extends the range, but also has the advantage of restoring the light information of the actual physical world. For the HDR images obtained by the proposed method, the HDR-VDP2 Q score, which is the most popular evaluation metric for HDR images, was 56.36 for a display with a 1920$\times$1200 resolution, which is an improvement of 6 compared with the scores of conventional algorithms. In addition, when comparing the peak signal-to-noise ratio values for tone mapped HDR images generated by the proposed and conventional algorithms, the average value obtained by the proposed algorithm is 30.86 dB, which is 10 dB higher than those obtained by the conventional algorithms.

Generation Method of High Dynamic Range Image from a Single Low Dynamic Range Image Based on Retinex Enhancement

Generation Method of High Dynamic Range Image from a Single Low Dynamic Range Image Based on Retinex Enhancement

Interactive relighting in single low-dynamic range images

This article addresses the relighting of outdoor and large indoor scenes illuminated by nondistant lights, which has seldom been discussed in previous works. We propose a method for users to interactively edit the illumination of a scene by moving existing lights and inserting synthetic lights into the scene that requires only a small amount of user annotation and a single low-dynamic range (LDR) image. We achieve this by adopting a top-down approach that estimates the scene reflectance by fitting a diffuse illumination model to a photograph. This approach gains stability and robustness by estimating the camera, scene geometry, and light sources in sequence and by using a confidence map, which is a per-pixel weight map. The results of our evaluation demonstrates that the proposed method can estimate a scene accurately enough for realistic relighting of images. Moreover, the experimental results of our user studies show that the synthesized images are so realistic as to be almost indistinguishable from real photographs.

Generating HDR radiance maps from single LDR image

We present a method of generating high dynamic range (HDR) radiance maps from a single low dynamic range (LDR) image and its camera response function (CRF). The method first models and estimates the inverse CRF and then multiplies the inverse CRF by a weighting function to make it smooth near the maximum and minimum pixel values, and finally conducts the smooth inverse CRF on the input LDR image to generate HDR image. In the method, the inverse CRF is estimated using one single LDR image and an empirical model of CRF, based on measured RGB distributions at color edges. Unlike most existing methods, the proposed method expands image from both high and low luminance regions. Thus, the algorithm can avoid the artifacts and detail loss in dark area which results from extending image only from bright region. Extensive experimental results show that the approach induces less contrast distortion and produces high visual quality HDR image.