Input Low Dynamic Range Image Research Articles

High dynamic range imaging with short- and long-exposures based on artificial remapping using multiscale exposure fusion

High dynamic range imaging (HDRI) is an excellent high-quality image acquisition technique, which can reflect real human visual characteristics from one (or several) captured low dynamic range (LDR) image. However, the input LDR image only provides partial information of the scene. Besides, in traditional HDRI methods that require multiple captured images as input, field of view errors can be induced, which will be difficult to apply it to the emerging image acquisition systems. Here, we propose a novel HDRI method that reconstructs an HDR image from only a pair of short- and long-exposure images based on artificial remapping using multi-scale exposure fusion. Firstly, we introduce a simulated exposure model called artificial remapping to synthesize a multi-exposure image sequence from the input LDR image pairs. Then, weighting maps of the sequence for fusion can be obtained according to the evaluation factors of contrast, saturation, as well as improved exposedness. Finally, we utilize the pyramid based multiscale exposure fusion framework to integrate them into an enhanced HDR image. Comparative experiments, fully implemented on some source images, have been demonstrated that better performance can be realized compared with some competing methods in qualitative and quantitative evaluation. Note that the operation of the proposed method is simple yet effective, which is easy to popularize. The method thus can be potentially applied to the emerging image acquisition systems where two images are captured simultaneously by two image sensors or by one image sensor with a pair of short- and long-exposure setting.

High dynamic range imaging by a pseudo exposure fusion method based on artificial remapping

High dynamic range imaging is a promising image enhancement technology, which can reconstruct a high dynamic range (HDR) image with a real visual perception from captured one (or several) raw low dynamic range (LDR) image. However, the input LDR image only provides partial information of the scene. Besides, in traditional methods that require multiple captured images as input, it is prone to halos and/or ghosting artifacts due to camera shake or object motion. In this paper, we propose a simple yet valid method that firstly uses artificial remapping to generate a set of simulated multi-exposure image sequence from a single LDR image, then fuses them into a high-quality HDR image via multiscale pyramid framework-based exposure fusion. It was demonstrated that artifacts-free and details-preserving HDR image can be achieved. The method thus can be potentially applied to the field of high dynamic range imaging, and can be further used for astronomical exploration, biomedical imaging, and optical imaging fields in the future.

Use of Local Region Maps on Convolutional LSTM for Single-Image HDR Reconstruction

Low dynamic range (LDR) images captured by consumer cameras have a limited luminance range. As the conventional method for generating high dynamic range (HDR) images involves merging multiple-exposure LDR images of the same scene (assuming a stationary scene), we introduce a learning-based model for single-image HDR reconstruction. An input LDR image is sequentially segmented into the local region maps based on the cumulative histogram of the input brightness distribution. Using the local region maps, SParam-Net estimates the parameters of an inverse tone mapping function to generate a pseudo-HDR image. We process the segmented region maps as the input sequences on long short-term memory. Finally, a fast super-resolution convolutional neural network is used for HDR image reconstruction. The proposed method was trained and tested on datasets including HDR-Real, LDR-HDR-pair, and HDR-Eye. The experimental results revealed that HDR images can be generated more reliably than using contemporary end-to-end approaches.

High Dynamic Range Imaging for Dynamic Scenes With Large-Scale Motions and Severe Saturation

Two key challenges exist in high dynamic range (HDR) imaging from multiexposure low dynamic range (LDR) images for dynamic scenes: 1) aligning the input images with large-scale foreground motions and 2) recovering large saturated regions from a limited number of input LDR images. To tackle these challenges, several deep convolutional neural networks have been proposed that have made significant progress. However, these methods tend to suffer from ghosting and saturation artifacts when applied to some challenging scenes. In this article, we propose an end-to-end deformable HDR imaging network, called DHDRNet, which attempts to alleviate these problems by building an effective aligning module and adopting self-guided attention. First, we analyze the alignment process in the HDR imaging task and correspondingly design a pyramidal deformable module (PDM) that aligns LDR images at multiple scales and reconstructs aligned features in a coarse-to-fine manner. In this way, the proposed DHDRNet can handle large-scale complex motions and suppress ghosting artifacts caused by misalignments. Moreover, we adopt self-guided attention to reduce the influence of saturated regions during the aligning and merging processes, which helps suppress artifacts and retain fine details in the final HDR image. Extensive qualitative and quantitative comparisons demonstrate that the proposed model outperforms the existing start-of-the-art methods and that it is robust to challenging scenes with large-scale motions and severe saturation. The source code is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Tx000/DHDRNet</uri> .

MIEHDR CNN: Main Image Enhancement based Ghost-Free High Dynamic Range Imaging using Dual-Lens Systems

We study the High Dynamic Range (HDR) imaging problem using two Low Dynamic Range (LDR) images that are shot from dual-lens systems in a single shot time with different exposures. In most of the related HDR imaging methods, the problem is usually solved by Multiple Images Merging, i.e. the final HDR image is fused from pixels of all the input LDR images. However, ghost artifacts can be hardly avoided using this strategy. Instead of directly merging the multiple LDR inputs, we use an indirect way which enhances the main image, i.e. the short exposure image IS, using the long exposure image IL serving as guidance. In detail, we propose a new model, named MIEHDR CNN model, which consists of three subnets, i.e. Soft Warp CNN, 3D Guided Denoising CNN and Fusion CNN. The Soft Warp CNN aligns IL to get the aligned result ILA using the soft exposed result of IS as reference. The 3D Guided Denoising CNN denoises the soft exposed result of IS using ILA as guidance, whose result are fed into the Fusion CNN with IS to get the HDR result. The MIEHDR CNN model is implemented by MindSpore and experimental results show that we can outperform related methods largely and avoid ghost artifacts.

HDR Image Reconstruction Using Segmented Image Learning

Converting a low dynamic range (LDR) image into a high dynamic range (HDR) image produces an image that closely depicts the real world without requiring expensive devices. Recent deep learning developments can produce highly realistic and sophisticated HDR images. This paper proposes a deep learning method to segment the bright and dark regions from an input LDR image and reconstruct the corresponding HDR image with similar dynamic ranges in the real world. The proposed multi-stage deep learning network brightens bright regions and darkens dark regions, and features with extended brightness range are combined to form the HDR image. Dividing the LDR image into the bright and dark regions effectively implements information on lost over-exposed and under-exposed areas, reconstructing a natural HDR image with color and appearance that is similar to reality. Experimental results confirm that the proposed method achieves an 8.52% higher HDR visual difference predictor (HDR-VDP) and a 41.2% higher log exposure range than current methods. Qualitative evaluation also verifies that the proposed method generates images that are close in quality to the ground truth.

Deep Coupled Feedback Network for Joint Exposure Fusion and Image Super-Resolution.

Nowadays, people are getting used to taking photos to record their daily life, however, the photos are actually not consistent with the real natural scenes. The two main differences are that the photos tend to have low dynamic range (LDR) and low resolution (LR), due to the inherent imaging limitations of cameras. The multi-exposure image fusion (MEF) and image super-resolution (SR) are two widely-used techniques to address these two issues. However, they are usually treated as independent researches. In this paper, we propose a deep Coupled Feedback Network (CF-Net) to achieve MEF and SR simultaneously. Given a pair of extremely over-exposed and under-exposed LDR images with low-resolution, our CF-Net is able to generate an image with both high dynamic range (HDR) and high-resolution. Specifically, the CF-Net is composed of two coupled recursive sub-networks, with LR over-exposed and under-exposed images as inputs, respectively. Each sub-network consists of one feature extraction block (FEB), one super-resolution block (SRB) and several coupled feedback blocks (CFB). The FEB and SRB are to extract high-level features from the input LDR image, which are required to be helpful for resolution enhancement. The CFB is arranged after SRB, and its role is to absorb the learned features from the SRBs of the two sub-networks, so that it can produce a high-resolution HDR image. We have a series of CFBs in order to progressively refine the fused high-resolution HDR image. Extensive experimental results show that our CF-Net drastically outperforms other state-of-the-art methods in terms of both SR accuracy and fusion performance. The software code is available here https://github.com/ytZhang99/CF-Net.

Single Exposure High Dynamic Range Image Reconstruction Based on Deep Dual-Branch Network

In this article, we relate the operation of single-frame-based high dynamic range (HDR) image reconstruction to the following two tasks: 1) highlight suppression in over-exposed areas and 2) noise elimination in under-exposed areas. The common goal of both tasks is to preserve or even enhance the details and improve the visibility of scenes when generating the HDR image. These two tasks can be solved separately with fundamentally different ways. In this article, we propose a dual-branch network to process the over- and under- exposed areas respectively for single-frame-based HDR image reconstruction. First, the low dynamic range (LDR) image is normalized, linearized and inputted into both branches, and the masks of the over- and under- exposed regions are calculated to detect the improper exposed areas. Second, the over- and under- exposed areas are restored and enhanced by the two branches respectively, at the same time, the color distribution is learned to obtain more consistent color saturation between the generated HDR image and the ground truth. Third, the output of the two branches and the linearized input LDR image are combined based on the masks to obtain the reconstructed HDR image. Extensive experiments show that the proposed method can efficiently restore the texture and color of the over-exposed areas, suppress the noise of the under-exposed areas, and obtain the HDR image with good contrast, clear details and high structural fidelity of the ground truth image appearance.

An optimization framework for inverse tone mapping using a single low dynamic range image

Conventional inverse tone-mapping (ITM) methods tend to produce contrast distortions such as contrast loss and contrast reversal in reconstructed high dynamic range (HDR) images. This paper proposes a novel ITM optimization framework based on the assumption that the input low dynamic range (LDR) image is similar to the LDR image obtained by tone mapping a true HDR image. In the proposed framework, an HDR image is initially reconstructed by applying a conventional tone-mapping function in a reverse manner, and then the reconstructed HDR image is iteratively modified toward the optimum HDR image by minimizing the difference between the input LDR image and a tone-mapped LDR image obtained from the reconstructed HDR image. The experimental results demonstrate that the proposed framework effectively reconstructs a high-quality HDR image and outperforms other conventional methods in terms of objective quality.

Single image‐based HDR image generation with camera response function estimation

The high performance of high dynamic range (HDR) image and the limitations of sensors impel to acquire HDR images from low dynamic range (LDR) ones. The most popular methods use multiple LDR images with different exposures of the same scene as input. However, it is difficult to have multiple exposures for most of LDR images. The authors propose a single image-based method to generate HDR image based on camera response function (CRF) reconstruction. The method first estimates the CRF according to the empirical model and the input LDR image. Then, the empirical model of the inverse CRF is constructed according to the relationship between the derivatives of CRF and its inverse function; the optimal solution of inverse CRF is obtained depending on the imaging properties of the edge pixels. Finally, the HDR image is generated by performing the inverse CRF on the original LDR image. The imitation of imaging procedure inherently makes the final HDR image high quality. The experimental results indicate that the proposed approach expand image from the dark and bright regions simultaneously. The resulting metric images also illustrate that their proposed method causes lower total contrast error compared with other single image-based methods.

Generating HDR radiance maps from single LDR image

We present a method of generating high dynamic range (HDR) radiance maps from a single low dynamic range (LDR) image and its camera response function (CRF). The method first models and estimates the inverse CRF and then multiplies the inverse CRF by a weighting function to make it smooth near the maximum and minimum pixel values, and finally conducts the smooth inverse CRF on the input LDR image to generate HDR image. In the method, the inverse CRF is estimated using one single LDR image and an empirical model of CRF, based on measured RGB distributions at color edges. Unlike most existing methods, the proposed method expands image from both high and low luminance regions. Thus, the algorithm can avoid the artifacts and detail loss in dark area which results from extending image only from bright region. Extensive experimental results show that the approach induces less contrast distortion and produces high visual quality HDR image.

Linear mapping based inverse tone mapping

We propose an inverse tone mapping (iTM) method which can both handle the details of low dynamic range (LDR) images and expand the dynamic ranges of the LDR images. The conventional iTM algorithms often fail to precisely restore the details of the input LDR images. To deal with this problem, we take a two-layer approach where each LDR image is separated into a base layer and a detail layer by bilateral filtering. The detail layer is mapped into that of a high dynamic range (HDR) image via linear mapping while the base layer is expanded via linear stretching to the dynamic range of a target display device. Then, the two resultant base and detail layers are used to reconstruct one final HDR image. From this, the details of the reconstructed HDR image can be revived via learned linear mapping. In order to learn the mapping from the LDR detail layer to an HDR detail layer, the HDR-LDR pairs of training patches of detail layers are classified into various groups based on the features of LDR detail patches. For each group, a linear mapping is learned during a training phase, which can then be applied for HDR reconstruction in testing phases. From the experimental results, we observed that proposed method can restore much more details of HDR images than the conventional methods.

High dynamic range image acquisition using multiple images with different apertures

The authors propose a method to obtain a high dynamic range (HDR) image from multiple images with different exposure. Unlike conventional methods that use multiple images with different shutter speeds, the proposed method takes multiple images with identical and short shutter speeds but with different apertures. Consequently, the input low dynamic range image is less affected by scene change, while it has undesirable defocus blur due to different depth of field. In order to mitigate defocus blur of input images with larger apertures, we estimate the defocus map of each input image and use it as the spatially variant point spread function to deblur the image. Then, we extract the weight maps of input images, which are used to combine them to synthesize an HDR image. Our experimental results show that the proposed algorithm produces high-quality HDR images using a small number (typically three) of input images.