Low Dynamic Range Images Research Articles

Ghost-Free Deep High-Dynamic-Range Imaging Using Focus Pixels for Complex Motion Scenes.

Multi-exposure image fusion inevitably causes ghost artifacts owing to inaccurate image registration. In this study, we propose a deep learning technique for the seamless fusion of multi-exposed low dynamic range (LDR) images using a focus-pixel sensor. For auto-focusing in mobile cameras, a focus-pixel sensor originally provides left (L) and right (R) luminance images simultaneously with a full-resolution RGB image. These L/R images are less saturated than the RGB images because they are summed up to be a normal pixel value in the RGB image of the focus pixel sensor. These two features of the focus pixel image, namely, relatively short exposure and perfect alignment are utilized in this study to provide fusion cues for high dynamic range (HDR) imaging. To minimize fusion artifacts, luminance and chrominance fusions are performed separately in two sub-nets. In a luminance recovery network, two heterogeneous images, the focus pixel image and the corresponding overexposed LDR image, are first fused by joint learning to produce an HDR luminance image. Subsequently, a chrominance network fuses the color components of the misaligned underexposed LDR input to obtain a 3-channel HDR image. Existing deep-neural-network-based HDR fusion methods fuse misaligned multi-exposed inputs directly. They suffer from visual artifacts that are observed mostly in saturated regions because pixel values are clipped out. Meanwhile, the proposed method reconstructs missing luminance with aligned unsaturated focus pixel image first, and thus, the luma-recovered image provides the cues for accurate color fusion. The experimental results show that the proposed method not only accurately restores fine details in saturated areas, but also produce ghost-free high-quality HDR images without pre-alignment.

Multiview Ghost-Free Image Enhancement for In-the-Wild Images With Unknown Exposure and Geometry

The multiview low dynamic range images captured with sparse camera arrangement under ill-lighting conditions contain highlighted and shadow regions due to over-exposed and under-exposed regions. The processing of these images produces contrast distortion, and it is challenging to maintain relative brightness with color consistency. Moreover, the disparity map estimation faces the challenges of holes and artifacts due to a wide baseline and poor visibility, with a shared view of vision. In this article, we propose a multiview ghost-free image enhancement strategy for in-the-wild images with unknown exposure and geometry. We address the complex geometric alignment problem for a wide variational baseline among multiple sparsely arranged cameras. The features among multiple viewpoints are detected and matched for the image restoration. The restored image contains highlighted and shadow regions with a color imbalance problem. We synthesize virtual images following the intensity mapping function, which compensates for the relative brightness and color distortions. Finally, we fuse all the images to obtain high-quality images. The proposed method is more frequent and feasible for future multiview systems with varying baselines without relying on disparity maps. The experimental results demonstrate that the proposed method outperformed the state-of-the-art approaches.

Superpixel-Based Quality Assessment of Multi-Exposure Image Fusion for Both Static and Dynamic Scenes.

Multi-exposure image fusion (MEF) algorithms have been used to merge a stack of low dynamic range images with various exposure levels into a well-perceived image. However, little work has been dedicated to predicting the visual quality of fused images. In this work, we propose a novel and efficient objective image quality assessment (IQA) model for MEF images of both static and dynamic scenes based on superpixels and an information theory adaptive pooling strategy. First, with the help of superpixels, we divide fused images into large- and small-changed regions using the structural inconsistency map between each exposure and fused images. Then, we compute the quality maps based on the Laplacian pyramid for large- and small-changed regions separately. Finally, an information theory induced adaptive pooling strategy is proposed to compute the perceptual quality of the fused image. Experimental results on three public databases of MEF images demonstrate the proposed model achieves promising performance and yields a relatively low computational complexity. Additionally, we also demonstrate the potential application for parameter tuning of MEF algorithms.

Highlights Analysis System (HAnS) for Low Dynamic Range to High Dynamic Range Conversion of Cinematic Low Dynamic Range Content

We propose a novel and efficient algorithm for detection of specular reflections and light sources (highlights) in cinematic content. The detection of highlights is important for reconstructing them properly in the conversion of the low dynamic range (LDR) to high dynamic range (HDR) content. Highlights are often difficult to be distinguished from bright diffuse surfaces, due to their brightness being reduced in the conventional LDR content production. Moreover, the cinematic LDR content is subject to the artistic use of effects that change the apparent brightness of certain image regions (e.g. limiting depth of field, grading, complex multi-lighting setup, etc.). To ensure the robustness of highlights detection to these effects, the proposed algorithm goes beyond considering only absolute brightness and considers five different features. These features are: the size of the highlight relative to the size of the surrounding image structures, the relative contrast in the surrounding of the highlight, its absolute brightness expressed through the luminance (luma feature), through the saturation in the color space (maxRGB feature) and through the saturation in white (minRGB feature). We evaluate the algorithm on two different image data-sets. The first one is a publicly available LDR image data-set without cinematic content, which allows comparison to the broader State of the art. Additionally, for the evaluation on cinematic content, we create an image data-set consisted of manually annotated cinematic frames and real-world images. For the purpose of demonstrating the proposed highlights detection algorithm in a complete LDR-to-HDR conversion pipeline, we additionally propose a simple inverse-tone-mapping algorithm. The experimental analysis shows that the proposed approach outperforms conventional highlights detection algorithms on both image data-sets, achieves high quality reconstruction of the HDR content and is suited for use in LDR-to-HDR conversion.

Towards accurate HDR imaging with learning generator constraints

High dynamic range (HDR) image generation is a useful technology in various applications. An easy way to obtain an HDR image is the multiple exposures fusion technique which fuses a set of sequential exposures. However it suffers from the ghosting artifacts for scenes with significant motion objects, which is the key challenge in HDR imaging. Detecting and removing ghosting artifacts is a crucial issue for automatically generating HDR images of dynamic scenes. Although previous methods align the low dynamic range (LDR) images or detect motion regions to remove the ghosting artifacts in the final HDR, they still cannot generate a well-pleasing HDR image. In this paper, we propose a novel deep neural network with learning generator constraints called GHDRNet that blends information from all the exposures. Different from previous methods which only use ground truth to learn parameters of the network for HDR image reconstruction, our method is based on a novel successive network which not only estimates HDR results from dynamic scenes but also restores the static LDR images from the estimated HDR image. This special designed network restrains the estimated HDR image by restoring the corresponding LDR images. To capture deep hierarchical features and increase the receptive field for recovering the abundant details, we also propose an enhancement block (EBlock) that is a topology structure to aggregate several scale features. Furthermore, considering the cumbersome networks have a redundant parameter, we introduce a light-weight residual module to EBlock which effectively reduces the network parameters. Extensive evaluations show the advantages of our method over related state-of-the-art methods on three benchmarks.

Dark Synthetic Vision: Lightweight Active Vision to Navigate in the Dark

Overcoming illumination variance is a critical factor in vision-based navigation. Existing methods tackled this radical illumination variance issue by proposing camera control or high dynamic range (HDR) image fusion. Despite these efforts, we have found that the vision-based approaches still suffer from overcoming darkness. This letter presents real-time image synthesizing from carefully controlled seed low dynamic range (LDR) image, to enable visual simultaneous localization and mapping (SLAM) in an extremely dark environment (less than 10 lux). Unlike existing methods, we elaborately select the seed LDR image for HDR fusion to secure interframe consistency, which is important in visual navigation. After selecting the seed image by camera control, we exploit camera response function (CRF) to synthesize HDR images in real-time without requiring GPU. We validate the algorithm via two extremely dark environments, an indoor environment without light and an outdoor night. In both test scenarios, the proposed method enabled reliable visual SLAM even when the light was limited.

Method for reconstructing a high dynamic range image based on a single-shot filtered low dynamic range image.

Traditional cameras are limited by sensors and cannot directly capture single-shot high dynamic range (HDR) images. We propose an improved single-shot HDR image reconstruction method that uses a single-exposure filtered low dynamic range (FLDR) image. First, by adding an optical filter in front of the camera lens, a FLDR image with different RGB channel exposure states and luminance ranges can be captured in a single-shot, unlike the traditional LDR image. Second, a deep inverse tone mapping network (DITMnet) with multibranch features extraction and multioutput images synthesis is designed to reconstruct an HDR image from a single FLDR image. Experimentally, under different exposure states and color spaces, our method outperforms similar algorithms.

Blind Tone-Mapped Image Quality Assessment Based on Regional Sparse Response and Aesthetics.

High dynamic range (HDR) images give a strong disposition to capture all parts of natural scene information due to their wider brightness range than traditional low dynamic range (LDR) images. However, to visualize HDR images on common LDR displays, tone mapping operations (TMOs) are extra required, which inevitably lead to visual quality degradation, especially in the bright and dark regions. To evaluate the performance of different TMOs accurately, this paper proposes a blind tone-mapped image quality assessment method based on regional sparse response and aesthetics (RSRA-BTMI) by considering the influences of detail information and color on the human visual system. Specifically, for the detail loss in a tone-mapped image (TMI), multi-dictionaries are first designed for different brightness regions and whole TMI. Then regional sparse atoms aggregated by local entropy and global reconstruction residuals are presented to characterize the regional and global detail distortion in TMI, respectively. Besides, a few efficient aesthetic features are extracted to measure the color unnaturalness of TMI. Finally, all extracted features are linked with relevant subjective scores to conduct quality regression via random forest. Experimental results on the ESPL-LIVE HDR database demonstrate that the proposed RSRA-BTMI method is superior to the existing state-of-the-art blind TMI quality assessment methods.

Extending and Matching a High Dynamic Range Image from a Single Image.

Extending the dynamic range can present much richer contrasts and physical information from the traditional low dynamic range (LDR) images. To tackle this, we propose a method to generate a high dynamic range image from a single LDR image. In addition, a technique for the matching between the histogram of a high dynamic range (HDR) image and the original image is introduced. To evaluate the results, we utilize the dynamic range for independent image quality assessment. It recognizes the difference in subtle brightness, which is a significant role in the assessment of novel lighting, rendering, and imaging algorithms. The results show that the picture quality is improved, and the contrast is adjusted. The performance comparison with other methods is carried out using the predicted visibility (HDR-VDP-2). Compared to the results obtained from other techniques, our extended HDR images can present a wider dynamic range with a large difference between light and dark areas.

Combining highlight removal and low‐light image enhancement technique for HDR‐like image generation

Low dynamic range (LDR) image may contain low-light and highlight areas due to the limitations of the dynamic range of conventional image sensors. Low-light and highlight phenomena limit colour richness and visibility of objects in an image. Therefore, it can cause a reduction in the quality of images and a loss in accuracy in the application of image recognition. To overcome this, high dynamic range (HDR)-like images have been developed with rich colours such as those seen by the human eye. In this study, the authors propose a method to obtain an HDR-like image from a single LDR image by removing the specular component from highlight pixels as well as strengthening the actual colour. Next, they select low-light image enhancement via illumination map estimation as a low-light enhancement technique by showing the comparison with gamma-based expansion operator. They evaluate their HDR-like output images with non-reference and full-reference metrics. They show the comparison of their proposed method with six other methods. Besides, visually, their proposed method delivers more pleasing output than the output of other competitive methods.

Information measure based tone mapping of outdoor LDR image for maximum scale‐invariant feature transform extraction

A novel tone mapping method referred to here as information measure-based tone mapping is proposed for extracting the maximum number of scale-invariant feature transform (SIFT) feature points from outdoor low dynamic range (LDR) images. To this end, first, an image is transformed to YUV colour space in which every pixel is assigned two information scores: forward and inverse scores. The forward and inverse information scores represent the number of neighbour pixels that differ in their intensities from the pixel under consideration by a certain threshold for the respective YUV and inverse YUV images. The proposed tone mapping is then carried out in such a way as to emphasise the pixels of high information scores defined by a weighted sum of forward and inverse information scores. Experimental results show that the proposed method offers superior performance to state-of-the-art methods in terms of the number of features extracted and matched.

Reduced-Dimensional Capture of High-Dynamic Range Images with Compressive Sensing

The range of light illumination in real scenes is very large, and ordinary cameras can only record a small part of this range, which is far lower than the range of human eyes’ perception of light. High-dynamic range (HDR) imaging technology that has appeared in recent years can record a wider range of illumination than the perceptual range of the human eye. However, the current mainstream HDR imaging technology is to capture multiple low-dynamic range (LDR) images of the same scene with different exposures and then merge them into one HDR image, which greatly increases the amount of data captured. The advent of single-pixel cameras (compressive imaging system) has proved the feasibility of obtaining and restoring image data based on compressive sensing. Therefore, this paper proposes a method for reduced-dimensional capture of high dynamic range images with compressive sensing, which includes algorithms for front end (capturing) and back end (processing). At the front end, the K-SVD dictionary is used to compressive sensing the input multiple-exposure image sequence, thereby reducing the amount of data transmitted to the back end. At the back end, the Orthogonal Matching Pursuit (OMP) algorithm is used to reconstruct the input multiple-exposure image sequence. A low-rank PatchMatch algorithm is proposed to merge the reconstructed image sequence to obtain an HDR image. Simulation results show that, under the premise of reducing the complexity of the front-end equipment and the amount of communication data between the front end and the back end, the overall system achieves a good balance between the amount of calculation and the quality of the HDR image obtained.

Blind quality assessment for tone-mapped images based on local and global features

In order to show high dynamic range (HDR) images by traditional displays, various tone-mapping operators have been designed to convert HDR images into low dynamic range (LDR) images recently. However, how to estimate the visual quality of LDR images effectively is still challenging. In this paper, we propose a novel blind quality assessment method for tone-mapped images with the consideration of naturalness and the perceptual characteristics of human visual system (HVS). First, we design parametric models that describe characteristics of chromatic information in tone-mapped images and extract quality-aware features based on global statistics model to characterize the naturalness of tone-mapped images. Second, motivated by perceptual characteristics that the HVS is highly adaptive to the image texture, we employ local texture features to capture the quality degradation of tone-mapped images. Support vector regression (SVR) is used to train the quality prediction model from features to human ratings. Experimental results indicate that the proposed metric can get better performance in predicting the visual quality of tone-mapped images than the state-of-the-art methods.

High-dynamic-range X-ray CT imaging method based on energy self-adaptation between scanning angles

High-dynamic-range (HDR) X-ray CT imaging is effective in detecting some complex structures. For previous low-dynamic-range (LDR) imaging detectors, multi-energy LDR image sequence fusion can extend the dynamic range, but the efficiency is decreased. However, with the application of HDR imaging devices, traditional fixed-energy X-ray imaging can cause incongruity within energy, dynamic range, and the equivalent thickness of the workpiece at different projection angles. Then, the projection has a blurred edge, and the CT image quality is poor because of scattering and the inadequate dose. In this paper, a new HDR X-ray CT imaging method with energy self-adaptation between scanning angles for HDR imaging devices is studied. Low-energy prescanning is used to determine the initial scanning energy and obtain the edge contour information with an attenuating effect on scattering. By establishing a mathematical model between the gray level of the projection and the transmission voltage, the transmission energy at each angle is adjusted adaptively. Then, the prescanning and energy self-adaption scanning projections are fused to obtain the complete projection of the complex workpiece. Finally, a conventional reconstruction algorithm is used to reconstruct the HDR CT image. The experimental results show that the proposed imaging method can achieve HDR CT imaging of complex structures with high reconstruction quality, clear edge details, and high completeness.

Tone Mapping High Dynamic Range Images by Hessian Multiset Canonical Correlations

Tone mapping algorithms reproduce high dynamic range (HDR) images on low dynamic range images in the standard display devices such as LCD, CRT, projectors, and printers. In this paper, we propose a statistical clustering-based tone mapping technique that would be able to adapt the local content of an image as well as its color. At first, the HDR image is partitioned into many overlapped color patches and we disintegrate each color patch into three segments: patch mean, color variation and color structure. Then based on the color structure component, the extracted color patches are clustered into a number of clusters by k-means clustering technique. For each cluster, the statistical signal processing technique namely Hessian multi set canonical correlations (HesMCC) has been produced to ascertain the transform matrix. Moreover, the HesMCC are fundamentally utilized for performing the dimensionality reduction of patches and to form effective tone mapped images. Contrasting with the current strategies, the procedures in the proposed clustering-based strategy can better adapt image color and its local structures by exploiting the image in the worldwide repetition. Experimental results show that the running time of the proposed method is less about 88.32%, 92%, 68.9%, and 29.4%, while comparing with other existing tone mapping methods.

Dynamic Range Expansion Using Cumulative Histogram Learning for High Dynamic Range Image Generation

In modern digital photographs, most images have low dynamic range (LDR) formats, which means that the range of light intensities from the darkest to the brightest is much lower than the range that can be perceived by the human eye. Therefore, to visualize images as naturally as possible on devices that display them in high dynamic range (HDR) format, the LDR images need to be converted into HDR images. The aim of this study was to develop an adaptive inverse tone mapping operator (iTMO) that can convert a single LDR image into a realistic HDR image based on artificial neural networks. In contrast to conventional iTMO algorithms, our technique was developed by learning the complicated relationship between various LDR–HDR pair images, which enabled nearly ground-truth HDR images to be generated from various types of LDR images. The novel learning technique is called cumulative histogram-based learning and color difference learning. The superior performance of our technique over conventional methods was assessed through objective evaluations of various types of LDR and HDR images.

Backward Compatible Object Detection Using HDR Image Content

Convolution Neural Network (CNN)-based object detection models have achieved unprecedented accuracy in challenging detection tasks. However, existing detection models (detection heads) trained on 8-bits/pixel/channel low dynamic range (LDR) images are unable to detect relevant objects under lighting conditions where a portion of the image is either under-exposed or over-exposed. Although this issue can be addressed by introducing High Dynamic Range (HDR) content and training existing detection heads on HDR content, there are several major challenges, such as the lack of real-life annotated HDR dataset(s) and extensive computational resources required for training and the hyper-parameter search. In this paper, we introduce an alternative backwards-compatible methodology to detect objects in challenging lighting conditions using existing CNN-based detection heads. This approach facilitates the use of HDR imaging without the immediate need for creating annotated HDR datasets and the associated expensive retraining procedure. The proposed approach uses HDR imaging to capture relevant details in high contrast scenarios. Subsequently, the scene dynamic range and wider colour gamut are compressed using HDR to LDR mapping techniques such that the salient highlight, shadow, and chroma details are preserved. The mapped LDR image can then be used by existing pre-trained models to extract relevant features required to detect objects in both the under-exposed and over-exposed regions of a scene. In addition, we also conduct an evaluation to study the feasibility of using existing HDR to LDR mapping techniques with existing detection heads trained on standard detection datasets such as PASCAL VOC and MSCOCO. Results show that the images obtained from the mapping techniques are suitable for object detection, and some of them can significantly outperform traditional LDR images.

Differential Evolution-based Approach for Tone-Mapping of High Dynamic Range Images

Recently, high dynamic range (HDR) imaging has received significant attention from research community as well as the industrial companies due to valuable applications of HDR images in better visualization and analysis. However, HDR images need to be converted to low dynamic range (LDR) images for viewing on standard LDR display screens. Several tone-mapping operators have been proposed for the conversion, however, so far, no significant works have been reported employing artificial intelligence to achieve better enhancement of the output images. In this paper, we present an optimization-based approach, to enhance the quality of the tone-mapped LDR images using metaheuristics. More specifically, the optimization process is based on the differential evolution (DE) algorithm which takes tone-mapping function of an existing histogram-based method as initial guess and refines the histogram bins iteratively leading to progressive enhancement of the quality of LDR image. The final results produced by the proposed optimized histogram-based approach (OHbA) showed better performance compared to the existing state-of-the-art tone-mapping algorithms.

A Dynamic Fuzzy Histogram Equalization for High Dynamic Range Images by Using Multi-Scale Retinex Algorithm

High Dynamic Range (HDR) images are appropriate to compress as Low Dynamic Range (LDR) images to show in LDR devices. A Holo artifact, a large intensity gradient, affects the value of Low Dynamic Range image to HDR image conversion. Tone reproduction approaches, and combined tone reproduction technique and image edge protective filter are some of the techniques to exclude the Holo artifacts. LDR device users are not satisfied with the converted images displayed in the LDR devices, and this affects the device value in markets. In this paper, a Multi-Scale Retinex (MSR) based Tone Reproduction Operators (TRO) is proposed to remove the Holo artifacts present in the images during HDR image to LDR image conversion. Before tone reproduction, the divergence of an image has improved using Brightness by Maintaining Dynamic Fuzzy Histogram Equalization (BPFHE) method. After evaluating the algorithm in MATLAB, performance is measured based on input HDR image and LDR output image feature comparisons.

De-ghosting in High Dynamic Range Imaging Based on Intensity Scaling Cue

A High Dynamic Range (HDR) image produced from a sequence of low dynamic range (LDR) images can contain motion artefacts (ghosting) if the scene contains moving objects. Conventional d ...