Low Dynamic Range Images Research Articles

A Novel De-Ghosting Image Fusion Technique for Multi-Exposure, Multi-Focus Images Using Guided Image Filtering

In this paper, a novel de-ghosting image fusion technique is presented, which enhances the quality of low dynamic range images using multi-level exposures taken from the ordinary camera and also removes the ghosting artifact. In the proposed algorithm, first, the source images, taken under different exposure settings, are decomposed into base and detail layers using two-scale decomposition. The base and detail layers contain small and large-scale variation details of the source images, respectively. The Laplacian-of-Gaussian filter is applied to the source images to get the edge information. Afterward, the saliency map of the edges is computed. To remove the ghosting artifacts, a weight matrix is calculated by applying the median filter on the histogram equalized source images. The weight matrix is combined with the saliency map to generate more accurate weights. The separate weights for the base and detail layers are calculated using guided image filters. Finally, the base and detail layers’ weights are fused with the source images to generate a vivid and enhanced image without any artifacts. The proposed technique is evaluated both qualitatively and quantitatively. The comparison of our technique in terms of Yang’s Metric ( $Q_{Y}$ ), Quality Mutual Information ( $Q_{MI}$ ), Gradient-based Fusion Metric ( $Q_{G}$ ) and Chen Blum’s Metric ( $Q_{CB}$ ) with other state-of-the-art techniques proves that the proposed technique outperforms existing techniques.

Multi-exposure high dynamic range imaging with informative content enhanced network

For High Dynamic Range (HDR) imaging systems, a new multi-exposure HDR imaging method based on Informative Content Enhanced Network (ICEN) is proposed to overcome the disadvantage that the existing deep learning based methods fails to fully exploit the differently exposed Low Dynamic Range (LDR) image contents to recover the details of the under/over-exposed regions. Specifically, the key contents in differently exposed LDR images that contribute to HDR imaging are firstly defined as Informative Contents (ICs). Then, the ICs are enhanced by the proposed Residual Channel Attention Module (RCAM), and then fused to generate HDR image. Furthermore, a generation scheme is designed for constructing HDR image labels and producing a multi-exposure HDR imaging dataset for training the proposed ICEN. The experimental results show that the proposed method is superior to the existing HDR imaging methods in quantitative and qualitative analysis, and can quickly generate high-quality HDR images.

Multiexposure Estimation and Fusion Based on a Sparsity Exposure Dictionary

Multiexposure image fusion (MEF), which is an emerging hot research topic, aims to fuse multiexposure images into a high-quality low dynamic range (LDR) image. In this article, we propose a novel exposure estimation-based MEF method with sparse decomposition and a designed sparsity exposure dictionary (SED). First, in order to represent more elementary information of multiexposure images, SED is trained by underexposed and overexposed images and constructed by the K-means-based algorithm (K-SVD) algorithm. Next, the SED is applied to construct exposure estimation maps by the number of atoms of the image patch, which is obtained by sparse decomposition. When the exposure estimation maps are obtained, a novel fusion framework that uses these maps to fuse multiexposure images is presented. In this framework, we employ the exposure estimation maps and an adaptive guided filter to construct the final fusion decision maps. The two factors of guided filter are adaptively determined without manual interference to obtain an optimal representation effect for multiexposure image sequences with different sizes and number. Finally, a fused image is generated via the pyramid merging method. The proposed fusion method is conducted on a series of multiexposure image sequences and compared to seven popular MEF methods. The experimental results clearly demonstrate that the proposed method outperforms the state-of-the-art methods in terms of subjective visual and quantitative evaluations.

No-Reference Image Quality Metric for Tone-Mapped Images

Tone-mapping operators transform high dynamic range (HDR) images into displayable low dynamic range (LDR) images. Image quality evaluation of these LDR images is not possible by comparison with their corresponding high dynamic range images. Hence, a no-reference image quality metric for tone-mapped LDR images is proposed based on the fitting to the present psychophysical results including different visual image quality attributes. Ten images, including HDR natural scenes, were tonemapped using six TMOs. They were used in the assessment and visual attributes were determined to predict the quality of these images. The visual attributes (brightness and Naturalness) were modeled using parameters derived from CAM16-UCS. Results showed that the quality prediction of the model had a reasonable degree of accuracy.

A Contribution Algorithm from LDRI to HDRI

High dynamic range image (HDRI) which is combined with low dynamic range image (LDRI) needs to be mapped to a low dynamic area to display. In the process of mapping, it is impossible to determine the contribution of low dynamic image sequences in the display images, so that it results in a problem that the low dynamic images cannot be accurately selected. In this paper, for the first time, a contribution algorithm from LDRI to HDRI according to the corresponding response curve of the camera is proposed.

Weighted Sparse Representation and Gradient Domain Guided Filter Pyramid Image Fusion Based on Low-Light-Level Dual-Channel Camera

Generally, the dynamic range of night vision scenes is large. Owing to the limited dynamic range of traditional low light imaging technology, the captured images are always partially overexposed or underexposed. Multi-exposure fusion is the most effective method of overcoming the dynamic range limitation of sensor, and multi-frame low dynamic range (LDR) image fusion is a key consideration. However, existing fusion methods have problems such as image detail blurring and image aliasing. This paper proposes an image multi-scale decomposition method based on gradient domain guided filter (GDGF), which can better extract image details. The fusion algorithm adopts different fusion strategies for different scales. The low-frequency layer of the image uses a new weighted sparse representation (wSR) method, which can eliminate the image boundary problems and more adequately retain the image edges.

A Biological Vision Inspired Framework for Image Enhancement in Poor Visibility Conditions.

Image enhancement is an important pre-processing step for many computer vision applications especially regarding the scenes in poor visibility conditions. In this work, we develop a unified two-pathway model inspired by the biological vision, especially the early visual mechanisms, which contributes to image enhancement tasks including low dynamic range (LDR) image enhancement and high dynamic range (HDR) image tone mapping. Firstly, the input image is separated and sent into two visual pathways: structure-pathway and detail-pathway, corresponding to the M-and P-pathway in the early visual system, which code the low-and high-frequency visual information, respectively. In the structure-pathway, an extended biological normalization model is used to integrate the global and local luminance adaptation, which can handle the visual scenes with varying illuminations. On the other hand, the detail enhancement and local noise suppression are achieved in the detail-pathway based on local energy weighting. Finally, the outputs of structure-and detail-pathway are integrated to achieve the low-light image enhancement. In addition, the proposed model can also be used for tone mapping of HDR images with some fine-tuning steps. Extensive experiments on three datasets (two LDR image datasets and one HDR scene dataset) show that the proposed model can handle the visual enhancement tasks mentioned above efficiently and outperform the related state-of-the-art methods.

Low-complexity open-loop coding of IDR infrared images having JPEG compatibility

In this paper, we present a low-complexity open-loop Intermediate Dynamic Range (IDR) infrared image coding algorithm which provides two scalability layers. The first layer corresponds to a low dynamic range (LDR) version of the image which can be obtained by a JPEG decoder, while the second layer corresponds to the original IDR image. To achieve bit-depth scalability, we separate an input IDR image into LDR and residual images by simple operations, so that the residual image has a bit depth not higher than 8 bits per pixel, i.e., it can be compressed by JPEG baseline or other coding algorithm, developed for 8-bit depth formats. For real-time applications, we introduce a predefined look-up table containing quality factors for both the LDR and the residual images, so that increase of the table index corresponds to reduction of bit rate and increase of distortion close to an operational rate-distortion curve. Experimental results show that the proposed algorithm is significantly less complex than JPEG-XT Profile C, part 6, and provides better coding performance than its reference software with default tone mapping operator for the vast majority of the infrared test images.

Snapshot High Dynamic Range Imaging via Sparse Representations and Feature Learning

Bracketed High Dynamic Range (HDR) imaging architectures acquire a sequence of Low Dynamic Range (LDR) images in order to either produce a HDR image or an “optimally” exposed LDR image, achieving impressive results under static camera and scene conditions. However, in real world conditions, ghost-like artifacts and noise effects limit the quality of HDR reconstruction. We address these limitations by introducing a post-acquisition snapshot HDR enhancement scheme that generates a bracketed sequence from a small set of LDR images, and in the extreme case, directly from a single exposure. We achieve this goal via a sparse-based approach where transformations between differently exposed images are encoded through a dictionary learning process, while we learn appropriate features by employing a stacked sparse autoencoder (SSAE) based framework. Via experiments with real images, we demonstrate the improved performance of our method over the state-of-the-art, while our single-shot based HDR formulation provides a novel paradigm for the enhancement of LDR imaging and video sequences.

A high dynamic range imaging algorithm: implementation and evaluation

Camera specifications have become smaller and smaller, accompanied with great strides in technology and thinner product demands, which have led to some challenges and problems. One of those problems is that the image quality is reduced at the same time. The decrement of radius lens is also a cause leading to the sensor not absorbing a sufficient amount of light, resulting in captured images which include more noise. Moreover, current image sensors cannot preserve whole dynamic range in the real world. This paper proposes a Histogram Based Exposure Time Selection (HBETS) method to automatically adjust the proper exposure time of each lens for different scenes. In order to guarantee at least two valid reference values for High Dynamic Range (HDR) image processing, we adopt the proposed weighting function that restrains random distributed noise caused by micro-lens and produces a high quality HDR image. In addition, an integrated tone mapping methodology, which keeps all details in bright and dark parts when compressing the HDR image to Low Dynamic Range (LDR) image for display on monitors, is also proposed. Eventually, we implement the entire system on Adlink MXC-6300 platform that can reach 10 fps to demonstrate the feasibility of the proposed technology.

Efficient and Adaptive Tone Mapping Algorithm Based on Guided Image Filter

The recognition rate of computer vision algorithms is highly dependent on the image quality. To enhance the visual quality of the images captured under high-dynamic range (HDR) scenes, we propose an efficient and adaptive tone mapping algorithm based on guided image filter (GIF). The HDR image is compressed adaptively according to its average luminance. Then we decompose it into a base layer and a detail layer using the guided image filter. We improve the base layer and enhance the detail layer simultaneously, and combine the two layers to get the final low-dynamic range (LDR) image. Since the parameters are linked with image statistics, they adaptively fit to various kinds of images. The objective evaluation results on HDR image sets demonstrate the superiority of our proposed algorithm. Meanwhile, the result of our algorithm can reduce the halo artifacts and preserve more detail by subjective observation.

An optimization framework for inverse tone mapping using a single low dynamic range image

Conventional inverse tone-mapping (ITM) methods tend to produce contrast distortions such as contrast loss and contrast reversal in reconstructed high dynamic range (HDR) images. This paper proposes a novel ITM optimization framework based on the assumption that the input low dynamic range (LDR) image is similar to the LDR image obtained by tone mapping a true HDR image. In the proposed framework, an HDR image is initially reconstructed by applying a conventional tone-mapping function in a reverse manner, and then the reconstructed HDR image is iteratively modified toward the optimum HDR image by minimizing the difference between the input LDR image and a tone-mapped LDR image obtained from the reconstructed HDR image. The experimental results demonstrate that the proposed framework effectively reconstructs a high-quality HDR image and outperforms other conventional methods in terms of objective quality.

HDR image retrieval by using color-based descriptor and tone mapping operator

Various methods have been performed for the purpose of Low Dynamic Range (LDR) image retrieval. However, no major work concerning the High Dynamic Range (HDR) image indexing has been widely diffused yet. We therefore propose a method that tackles the problem of efficiently and accurately retrieving HDR images. The proposed system is based on a hybrid descriptor which combines two color features. The first one is histogram based on the hue–saturation–value (HSV) color space that approaches the perception of human vision, whereas the second comprises the first- and second-order moments of the color bands. As a dissimilarity measure, we retained the Manhattan distance. In the second part of our work, we proposed an automatic tone mapping operator (TMO) to get an overview on the result images by using Standard Dynamic Range (SDR) devices. Comparisons with recent state-of-the-art TMOs have shown that our TM method produces LDR images with adequate quality while maintaining low complexity. Finally, to test our retrieval system, we have created two databases. Experimental evaluation showed that our system supports HDR images while achieving satisfying results in terms of accuracy and computational cost.

Blind Quality Assessment of Tone-Mapped Images Considering Colorfulness, Naturalness, and Structure

Nowadays, high-dynamic-range (HDR) imaging represents a prevailing trend and attracts much attention from both academic and industrial scholars. Since HDR images cannot be properly produced on the mainstream low-dynamic-range (LDR) displays, various tone-mapping operators or postprocessing technologies have been designed to transform HDR images into LDR images for visualization on LDR displays. However, it inevitably induces artifacts and distortions due to dynamic range compression. Besides, existing tone-mapped (TM) technologies cannot effectively handle all kinds of images with diverse contents and structures, leaving to a very challenging and urgent image quality assessment (IQA) problem. To cope with this challenge, in this paper, an effective blind quality assessment approach for TM images is proposed through a comprehensive consideration of their characteristics. More specifically, to dig out sufficient information from TM images, multiple quality-sensitive features are captured to fully represent different attributes, including colorfulness, naturalness, and structure. The connection between feature space and associated subjective ratings is established via a regression model. Extensive experiments on a recently released TM image database prove that the proposed approach is superior to the state-of-the-art no-reference IQA approaches.

Weighted sparse representation multi-scale transform fusion algorithm for high dynamic range imaging with a low-light dual-channel camera.

Most imaging devices lose image information during the acquisition process due to their low dynamic range (LDR). Existing high dynamic range (HDR) imaging techniques have a trade-off with time or spatial resolution, resulting in potential motion blur or image misalignment. Current HDR methods are based on the fusion of multi-frame LDR images and can suffer from blurring of fine details, image aliasing, and image boundary effects. This study developed a dual-channel camera (DCC) to achieve HDR imaging, which can eliminate image motion blur and registration problems. Considering the output characteristics of the camera, we propose a weighted sparse representation multi-scale transform fusion algorithm, which fully preserves the original image information, while eliminating image aliasing and boundary problems in the fused image, resulting in high-quality HDR imaging.

An Adaptive Method for Image Dynamic Range Adjustment

In this paper, we relate the operation of image dynamic range adjustment to the following two tasks: 1) for a high dynamic range (HDR) image, its dynamic range will be mapped to the available dynamic range of display devices and 2) for a low dynamic range (LDR) image, its distribution of intensity will be extended to adequately utilize the full dynamic range of display devices. The common goal of both tasks is to preserve or even enhance the details and improve the visibility of scenes when being matched to the available dynamic range of a display device. In this paper, we propose an efficient method for image dynamic range adjustment with three adaptive steps. First, according to the histogram of the luminance map separated from the given RGB image, two suitable Gamma functions are adaptively selected to separately adjust the luminance of the dark and bright components. Second, an adaptive fusion strategy is proposed to combine the two adjusted luminance maps in order to balance the enhancement of the details in different regions. Third, an adaptive luminance-dependent color restoration method is designed to combine the fused luminance map with the original color components to obtain more consistent color saturation between the images before and after dynamic range adjustment. Extensive experiments show that the proposed method can efficiently compress the dynamic range of HDR scenes with good contrast, clear details, and high structural fidelity of the original image appearance. In addition, the proposed method can also obtain promising performance when being used to enhance LDR nighttime images and greatly facilitates the object (car) detection in nighttime traffic scenes.

Statistics of natural images as a function of dynamic range.

The statistics of real world images have been extensively investigated, but in virtually all cases using only low dynamic range image databases. The few studies that have considered high dynamic range (HDR) images have performed statistical analyses categorizing images as HDR according to their creation technique, and not to the actual dynamic range of the underlying scene. In this study we demonstrate, using a recent HDR dataset of natural images, that the statistics of the image as received at the camera sensor change dramatically with dynamic range, with particularly strong correlations with dynamic range being observed for the median, standard deviation, skewness, and kurtosis, while the one over frequency relationship for the power spectrum breaks down for images with a very high dynamic range, in practice making HDR images not scale invariant. Effects are also noted in the derivative statistics, the single pixel histograms, and the Haar wavelet analysis. However, we also show that after some basic early transforms occurring within the eye (light scatter, nonlinear photoreceptor response, center-surround modulation) the statistics of the resulting images become virtually independent from the dynamic range, which would allow them to be processed more efficiently by the human visual system.

Sensor interpixel correlation analysis and reduction for color filter array high dynamic range image reconstruction

AbstractHigh dynamic range imaging (HDRI) by bracketing of low dynamic range (LDR) images is demanding, as the sensor is deliberately operated at saturation. This exacerbates any crosstalk, interpixel capacitance, blooming and smear, all causing interpixel correlations (IC) and a deteriorated modulation transfer function (MTF). Established HDRI algorithms exclude saturated pixels, but generally overlook IC. This work presents a calibration method to estimate the affected region from saturated pixels for a color filter array (CFA) sensor, using the native CFA as a matched filter. The method minimizes color crosstalk given a set of candidates for proximity regions, and requires no special setup. Results are shown for a 21‐bit HDR output image with improved color fidelity and reduced noise. The calibration reduces IC in the LDR images and is performed only once for a given sensor. The improvement is applicable to any HDRI algorithm based on CFA image bracketing, irrespective of sensor technology. Generalizations to subsaturated and supersaturated pixels are described, facilitating a suggested irradiance‐exposure dependent point spread function charge repatriation strategy.

Improved image selection for stack-based HDR imaging

Stack-based high dynamic range (HDR) imaging is a technique for achieving a larger dynamic range in an image by combining several low dynamic range images acquired at different exposures. Minimizing the set of images to combine, while ensuring that the resulting HDR image fully captures the scene’s irradiance, is important to avoid long image acquisition and postprocessing times. The problem of selecting the set of images has received much attention. However, existing methods either are not fully automatic, can be slow, or can fail to fully capture more challenging scenes. In this paper, we propose a fully automatic method for selecting the set of exposures to acquire that is both fast and more accurate. We show on an extensive set of benchmark scenes that our proposed method leads to improved HDR images as measured against ground truth using the mean squared error, a pixel-based metric, and a visible difference predictor and a quality score, both perception-based metrics.

Blind quality index for tone-mapped images based on luminance partition

Tone-mapping operators (TMOs), which are designed to convert high dynamic range (HDR) images to standard low dynamic range (LDR) images for displaying on conventional devices, have gained extensive attention recently. The quality of tone-mapped images generated by different TMOs varies significantly, which depend upon the image contents and the parameter settings. A quality index that can accurately evaluate the performances of TMOs is thus highly needed. With this motivation, this paper presents a blind quality index based on luminance partition for tone-mapped images. It is based on the fact that the Human Visual System (HVS) has different sensitivities to image regions with different luminance levels. Specifically, two adaptive thresholds are first employed to segment an image into the dark, bright and normal areas. Then, we calculate the quality-aware features from different luminance areas: 1) local entropy feature is extracted from the dark and bright areas to measure the information loss due to the overexposure or underexposure during the tone mapping process; 2) local colorfulness feature is extracted from the normal area to evaluate the reproduction of colors. With the consideration that the perception of image quality depends on the combined effects of the salient local distortion and global quality degradation, the global contrast feature is also calculated and integrated for better evaluation performance. Moreover, to take advantage of the hierarchical characteristic of the HVS, all features are calculated under a multi-resolution framework. Eventually, the extracted features are mapped into an objective quality score based on the random forest regression. The proposed metric is shown to outperform those state-of-the-art metrics according to extensive experiments conducted on two publicly available databases.