Low Dynamic Range Images Research Articles

Predictive pixel-wise optical encoding: towards single-shot high dynamic range moving object recognition

Conventional low dynamic range (LDR) imaging devices fail to preserve much information for further vision tasks because of the saturation effect. Thus, high dynamic range (HDR) imaging is an important imaging technology in extreme illuminance conditions, which enables a wide range of applications, including photography, autonomous driving, and robotics. Mainstream approaches require multi-shot methods because the conventional camera can only control the exposure globally. Although they perform well on static HDR imaging, they face a challenge with real-time HDR imaging for motional scenes because of the artifact and time latency caused by multi-shot and post-processing. To this end, we propose a framework, termed POE-VP, which achieves single-shot HDR imaging via a pixel-wise optical encoder driven by video prediction. We use highlighted motional license plate recognition as a downstream vision task to demonstrate the performance of POE-VP. From the results of simulation and real scene experiments, we validate that POE-VP outperforms conventional LDR cameras by more than 5 times in recognition accuracy and by more than 200% in information entropy. The dynamic range could reach 120 dB, and the captured data size is verified to be lower than mainstream multi-shot methods by 67%. The running time of POE-VP is also validated to satisfy the needs of high-speed HDR imaging.

Reconstructing High Dynamic Range Image from a Single Low Dynamic Range Image Using Histogram Learning

High dynamic range imaging is an important field in computer vision. Compared with general low dynamic range (LDR) images, high dynamic range (HDR) images represent a larger luminance range, making the images closer to the real scene. In this paper, we propose an approach for HDR image reconstruction from a single LDR image based on histogram learning. First, the dynamic range of an LDR image is expanded to an extended dynamic range (EDR) image. Then, histogram learning is established to predict the intensity distribution of an HDR image of the EDR image. Next, we use histogram matching to reallocate pixel intensities. The final HDR image is generated through regional adjustment using reinforcement learning. By decomposing low-frequency and high-frequency information, the proposed network can predict the lost high-frequency details while expanding the intensity ranges. We conduct the experiments based on HDR-Real and HDR-EYE datasets. The quantitative and qualitative evaluations have demonstrated the effectiveness of the proposed approach compared to the previous methods.

Multi exposure fusion for high dynamic range imaging via multi-channel gradient tensor

Multi-exposure fusion (MEF) is an effective technique for directly fusing a sequence of low dynamic range (LDR) images from a high dynamic range (HDR) natural scene. The goal is to generate an information enriched LDR image. Despite its effectiveness, current MEF methods often encounter issues such as detail loss and color degradation. Additionally, existing algorithms often struggle to balance image quality and computation time, particularly for large-sized images. This paper introduces an innovative MEF algorithm that address these challenges, offering improved performance and computational time across all image sizes. The algorithm employs a multi-channel gradient tensor on RGB images to effectively capture the contrast information among the three channels. This mechanism allows an edge-preserving image filter to maintain edges while smoothing weight maps. To enhance computational efficiency, the algorithm uses a fast approximation method suitable for large sized images. Our comprehensive experimental results demonstrate that the proposed method outperforms existing MEF techniques both quantitatively and qualitatively. Furthermore, our method reduces computational time by approximately 30% compared to the most recent state-of-the-art techniques.

Multiexposed Image-Fusion Strategy Using Mutual Image Translation Learning with Multiscale Surround Switching Maps

The dynamic range of an image represents the difference between its darkest and brightest areas, a crucial concept in digital image processing and computer vision. Despite display technology advancements, replicating the broad dynamic range of the human visual system remains challenging, necessitating high dynamic range (HDR) synthesis, combining multiple low dynamic range images captured at contrasting exposure levels to generate a single HDR image that integrates the optimal exposure regions. Recent deep learning advancements have introduced innovative approaches to HDR generation, with the cycle-consistent generative adversarial network (CycleGAN) gaining attention due to its robustness against domain shifts and ability to preserve content style while enhancing image quality. However, traditional CycleGAN methods often rely on unpaired datasets, limiting their capacity for detail preservation. This study proposes an improved model by incorporating a switching map (SMap) as an additional channel in the CycleGAN generator using paired datasets. The SMap focuses on essential regions, guiding weighted learning to minimize the loss of detail during synthesis. Using translated images to estimate the middle exposure integrates these images into HDR synthesis, reducing unnatural transitions and halo artifacts that could occur at boundaries between various exposures. The multilayered application of the retinex algorithm captures exposure variations, achieving natural and detailed tone mapping. The proposed mutual image translation module extends CycleGAN, demonstrating superior performance in multiexposure fusion and image translation, significantly enhancing HDR image quality. The image quality evaluation indices used are CPBDM, JNBM, LPC-SI, S3, JPEG_2000, and SSEQ, and the proposed model exhibits superior performance compared to existing methods, recording average scores of 0.6196, 15.4142, 0.9642, 0.2838, 80.239, and 25.054, respectively. Therefore, based on qualitative and quantitative results, this study demonstrates the superiority of the proposed model.

Deep indoor illumination estimation based on spherical gaussian representation with scene prior knowledge

High dynamic range illumination estimation from a single low dynamic range image is a critical task in the fields of computer vision, graphics and augmented reality. However, directly learning the full HDR environment map or parametric lighting information from a single image is extremely difficult and inaccurate. As a result, we propose a two-stage network approach for illumination estimation that integrates spherical gaussian (SG) representation with scene prior knowledge. In the first stage, a convolutional neural network is utilized to generate material and geometric information about the scene, which serves as prior knowledge for lighting prediction. In the second stage, we model indoor environment illumination using 128 SG functions with fixed center direction and bandwidth, allowing only the amplitude to vary. Subsequently, a Transformer-based lighting parameter regressor is employed to capture the complex relationship between the input images with scene prior information and its SG illumination. Additionally, we introduce a hybrid loss function, which combines a masked loss for high-frequency illumination with a rendering loss for improving the visual quality. By training and evaluating the lighting model on the created SG illumination dataset, the proposed method achieves competitive results in both quantitative metrics and visual quality, outperforming state-of-the-art methods.

MetaHDR: single shot high-dynamic range imaging and sensing using a multifunctional metasurface

We present MetaHDR, which is a single-shot high-dynamic range (HDR) imaging and sensing system using a multifunctional metasurface. The metasurface is capable of splitting an incident beam into multiple focusing beams with different amounts of power, simultaneously forming multiple low dynamic range (LDR) images with distinct irradiance on a photosensor. Then, the LDR images are jointly processed using a gradient-based HDR fusion algorithm, which is shown to be effective in attenuating the residual light artifacts incurred by the metasurface and the lens flare. MetaHDR achieves single-shot HDR photography and videography that increases the dynamic range by at least 50 dB compared to the original dynamic range of the photosensor. It can also perform single-shot HDR sensing, including reflectance calibration and surface curvature estimation of reflective materials. MetaHDR’s demonstrated functionalities could be broadly applied in surveillance and security, microscopic imaging, advanced manufacturing, etc.

Deep progressive feature aggregation network for multi-frame high dynamic range imaging

High dynamic range (HDR) imaging is an important task in image processing that aims to generate well-exposed images in scenes with varying illumination. Although existing multi-exposure fusion methods have achieved impressive results, generating high-quality HDR images in dynamic scenes remains difficult. The primary challenges are ghosting artifacts caused by object motion between low dynamic range images and distorted content in underexposure and overexposed regions. In this paper, we propose a deep progressive feature aggregation network for improving HDR imaging quality in dynamic scenes. To address the issues of object motion, our method implicitly samples high-correspondence features and aggregates them in a coarse-to-fine manner for alignment. In addition, our method adopts a densely connected network structure based on the discrete wavelet transform, which aims to decompose the input features into multiple frequency subbands and adaptively restore corrupted contents. Experiments show that our proposed method can achieve state-of-the-art performance under different scenes, compared to other promising HDR imaging methods. Specifically, the HDR images generated by our method contain cleaner and more detailed content, with fewer distortions, leading to better visual quality.

High Dynamic Range Image Reconstruction from Saturated Images of Metallic Objects.

This study considers a method for reconstructing a high dynamic range (HDR) original image from a single saturated low dynamic range (LDR) image of metallic objects. A deep neural network approach was adopted for the direct mapping of an 8-bit LDR image to HDR. An HDR image database was first constructed using a large number of various metallic objects with different shapes. Each captured HDR image was clipped to create a set of 8-bit LDR images. All pairs of HDR and LDR images were used to train and test the network. Subsequently, a convolutional neural network (CNN) was designed in the form of a deep U-Net-like architecture. The network consisted of an encoder, a decoder, and a skip connection to maintain high image resolution. The CNN algorithm was constructed using the learning functions in MATLAB. The entire network consisted of 32 layers and 85,900 learnable parameters. The performance of the proposed method was examined in experiments using a test image set. The proposed method was also compared with other methods and confirmed to be significantly superior in terms of reconstruction accuracy, histogram fitting, and psychological evaluation.

Color Event Enhanced Single-Exposure HDR Imaging

Single-exposure high dynamic range (HDR) imaging aims to reconstruct the wide-range intensities of a scene by using its single low dynamic range (LDR) image, thus providing significant efficiency. Existing methods pay high attention to restoring the luminance by inversing the tone-mapping process, while the color in the over-/under-exposed area cannot be well restored due to the information loss of the single LDR image. To address this issue, we introduce color events into the imaging pipeline, which record asynchronous pixel-wise color changes in a high dynamic range, enabling edge-like scene perception under challenging lighting conditions. Specifically, we propose a joint framework that incorporates color events and a single LDR image to restore both content and color of an HDR image, where an exposureaware transformer (EaT) module is designed to propagate the informative hints, provided by the normal-exposed LDR regions and the event streams, to the missing areas. In this module, an exposure-aware mask is estimated to suppress distractive information and strengthen the restoration of the over-/under-exposed regions. To our knowledge, we are the first to use color events to enhance single-exposure HDR imaging. We also contribute corresponding datasets, consisting of synthesized datasets and a real-world dataset collected by a DAVIS346-color camera. The datasets can be found at https://www.kaggle.com/datasets/mengyaocui/ce-hdr. Extensive experiments demonstrate the effectiveness of the proposed method.

A Low-Latency Noise-Aware Tone Mapping Operator for Hardware Implementation with a Locally Weighted Guided Filter

A tone mapping operator (TMO) is a module in the image signal processing pipeline that is used to convert high dynamic range images to low dynamic range images for display. Currently, state-of-the-art TMOs typically take complex algorithms and are implemented on graphics processing units, making it difficult to run with low latency on edge devices, and TMOs implemented in hardware circuits often lack additional noise suppression because of latency and hardware resource constraints. To address these issues, we proposed a low-latency noise-aware TMO for hardware implementation. Firstly, a locally weighted guided filter is proposed to decompose the luminance image into a base layer and a detail layer, with the weight function symmetric concerning the central pixel value of a window. Secondly, the mean and standard deviation of the basic layer and the detail layer are used to estimate the noise visibility according to the human visual characteristics. Finally, the gain for the detail layer is calculated to achieve adaptive noise suppression. In this process, luminance is first processed by the log2 function before being filtered and then symmetrically converted back to the linear domain by the exp2 function after compression. Meanwhile, the algorithms within the proposed TMO were optimized for hardware implementation to minimize latency and cache, achieving a low latency of 60.32 μs under video specification of 1080 P at 60 frames per second and objective metric smoothness in dark flat regions could be improved by more than 10% compared to similar methods.

DEUNet: Dual-encoder UNet for simultaneous denoising and reconstruction of single HDR image

High Dynamic Range (HDR) images have more powerful image expression capabilities and better visual quality than Low Dynamic Range (LDR) images, which can sufficiently represent real-world scenes, and will be widely used in the field of film and television in future. However, it is a very challenging task to generate an HDR image from a single-exposure LDR image. In this work, we propose a novel learning-based network, DEUNet, to reconstruct single-frame HDR image with simultaneous denoising and detail reconstruction. The proposed framework consists of two feature extraction branches, which can learn the brightness information and texture information separately for HDR image reconstruction. Each network branch is based on the UNet network structure and the two branches are interacted via spatial feature transformation. As a result, the proposed network can make full use of the multi-scale information at different levels of the image. In addition to the two encoding branches for feature extraction, the proposed network consists of another decoding network for fusing image brightness information and texture information, and a weighting network that selectively preserves most useful information. Compared with state-of-the-art methods, DEUNet can better reduce image noise while reconstructing the details in both the high and the low exposure areas. Experiments have shown that the proposed method achieves state-of-the-art performance on public datasets, indicating the effectiveness of the proposed method in this study.

ERS-HDRI: Event-Based Remote Sensing HDR Imaging

High dynamic range imaging (HDRI) is an essential task in remote sensing, enhancing low dynamic range (LDR) remote sensing images and benefiting downstream tasks, such as object detection and image segmentation. However, conventional frame-based HDRI methods may encounter challenges in real-world scenarios due to the limited information inherent in a single image captured by conventional cameras. In this paper, an event-based remote sensing HDR imaging framework is proposed to address this problem, denoted as ERS-HDRI, which reconstructs the remote sensing HDR image from a single-exposure LDR image and its concurrent event streams. The proposed ERS-HDRI leverages a coarse-to-fine framework, incorporating the event-based dynamic range enhancement (E-DRE) network and the gradient-enhanced HDR reconstruction (G-HDRR) network. Specifically, to efficiently achieve dynamic range fusion from different domains, the E-DRE network is designed to extract the dynamic range features from LDR frames and events and perform intra- and cross-attention operations to adaptively fuse multi-modal data. A denoise network and a dense feature fusion network are then employed for the generation of the coarse, clean HDR image. Then, the G-HDRR network, with its gradient enhancement module and multiscale fusion module, performs structure enforcement on the coarse HDR image and generates a fine informative HDR image. In addition, this work introduces a specialized hybrid imaging system and a novel, real-world event-based remote sensing HDRI dataset that contains aligned remote sensing LDR images, remote sensing HDR images, and concurrent event streams for evaluation. Comprehensive experiments have demonstrated the effectiveness of the proposed method. Specifically, it improves state-of-the-art PSNR by about 30% and the SSIM score by about 9% on the real-world dataset.

Fast and Accurate Illumination Estimation Using LDR Panoramic Images for Realistic Rendering.

A high dynamic range (HDR) image is commonly used to reveal stereo illumination, which is crucial for generating high-quality realistic rendering effects. Compared to the high-cost HDR imaging technique, low dynamic range (LDR) imaging provides a low-cost alternative and is preferable for interactive graphics applications. However, the limited LDR pixel bit depth significantly bothers accurate illumination estimation using LDR images. The conflict between the realism and promptness of illumination estimation for realistic rendering is yet to be resolved. In this paper, an efficient method that accurately infers illuminations of real-world scenes using LDR panoramic images is proposed. It estimates multiple lighting parameters, including locations, types and intensities of light sources. In our approach, a new algorithm that extracts illuminant characteristics during the exposure attenuation process is developed to locate light sources and outline their boundaries. To better predict realistic illuminations, a new deep learning model is designed to efficiently parse complex LDR panoramas and classify detected light sources. Finally, realistic illumination intensities are calculated by recovering the inverse camera response function and extending the dynamic range of pixel values based on previously estimated parameters of light sources. The reconstructed radiance map can be used to compute high-quality image-based lighting of virtual models. Experimental results demonstrate that the proposed method is capable of efficiently and accurately computing comprehensive illuminations using LDR images. Our method can be used to produce better realistic rendering results than existing approaches.

Wavelet-based network for high dynamic range imaging

High dynamic range (HDR) imaging from multiple low dynamic range (LDR) images has been suffering from ghosting artifacts caused by scene and objects motion. Existing methods, such as optical flow based and end-to-end deep learning based solutions, are error-prone either in detail restoration or ghosting artifacts removal. Comprehensive empirical evidence shows that ghosting artifacts caused by large foreground motion are mainly low-frequency signals and the details are mainly high-frequency signals. In this work, we propose a novel frequency-guided end-to-end deep neural network (FHDRNet) to conduct HDR fusion in the frequency domain, and Discrete Wavelet Transform (DWT) is used to decompose inputs into different frequency bands. The low-frequency signals are used to avoid specific ghosting artifacts, while the high-frequency signals are used for preserving details. Using a U-Net as the backbone, we propose two novel modules: merging module and frequency-guided upsampling module. The merging module applies the attention mechanism to the low-frequency components to deal with the ghost caused by large foreground motion. The frequency-guided upsampling module reconstructs details from multiple frequency-specific components with rich details. In addition, a new RAW dataset is created for training and evaluating multi-frame HDR imaging algorithms in the RAW domain. Extensive experiments are conducted on public datasets and our RAW dataset, showing that the proposed FHDRNet achieves state-of-the-art performance.

Unsupervised Optical Flow Estimation for Differently Exposed Images in LDR Domain

Differently exposed low dynamic range (LDR) images are often captured sequentially using a smart phone or a digital camera with movements. Optical flow thus plays an important role in ghost removal for high dynamic range (HDR) imaging. The optical flow estimation is based on the theory of photometric consistency, which assumes that the corresponding pixels between two images have the same intensity. However, the assumption is no longer valid for the differently exposed LDR images since a pixel’s intensity changes significantly inter images. To address the problem, an unsupervised optical flow estimation framework, is presented in this study. Intensity mapping functions (IMFs) are first adopted to alleviate the intensity changes between the LDR images. Then a novel IMF-based unsupervised learning objective is proposed to circumvent the need for ground truth optical flows when training the deep network. Experimental results and ablation studies on publicly available datasets show that our framework outperforms the state-of-the-art unsupervised optical flow methods, demonstrating the effectiveness of the IMF and the learning objective. Our code is available at https://github.com/liuziyang123/LDRFlow.

Microlens array camera with variable apertures for single-shot high dynamic range (HDR) imaging.

We report a microlens array camera with variable apertures (MACVA) for high dynamic range (HDR) imaging by using microlens arrays with various sizes of apertures. The MACVA comprises variable apertures, microlens arrays, gap spacers, and a CMOS image sensor. The microlenses with variable apertures capture low dynamic range (LDR) images with different f-stops under single-shot exposure. The reconstructed HDR images clearly exhibit expanded dynamic ranges surpassing LDR images as well as high resolution without motion artifacts, comparable to the maximum MTF50 value observed among the LDR images. This compact camera provides, what we believe to be, a new perspective for various machine vision or mobile devices applications.

Hardware-friendly tone-mapping operator design and implementation for real-time embedded vision applications

Tone-mapping is required to convert High Dynamic Range (HDR) images to Low Dynamic Range (LDR) images for viewing on standard displays. Most recent works in the domain include the efficient implementation of tone-mapping algorithms on Graphics Processing Units (GPUs) for throughput improvements. However, their downside is significantly high power consumption, making such solutions unsuitable for displays on embedded and general consumer electronic devices. In contrast, Field Programmable Gate Arrays (FPGAs) have demonstrated significant speedup on data-intensive image and video applications with a much lower power consumption than GPUs. This paper proposes a modular, scalable and resource-efficient FPGA implementation of a global Tone-Mapping Operator (TMO) based on histogram of the scene luminance and characteristics of the Human Visual System (HVS). Compared to GPU based implementation of a state-of-the-art TMO, our FPGA implementation achieved a higher throughput in the range of 2.5× to 4.5× with power savings of more than two orders of magnitude. Moreover, tonal quality of the proposed solution is better than the baseline TMO in approximately 80% of a dataset of 288 HDR images. The proposed solution also outperformed the state-of-the-art FPGA implementations of other TMOs in throughput, resource usage and scalability.

Multi exposure image fusion based on exposure correction and input refinement using limited low dynamic range images

Multi-Exposure Image Fusion (MEF) is an image processing technique that combines several images taken at various exposure levels into one high-quality image. The generation of artifacts and diminished perceptual quality of the fused image under few input images is an inevitable problem in MEF approaches. Although numerous MEF techniques have been proposed in the literature, little effort has been devoted to generate high quality fused images with minimal input images. The proposed work unveils a method for producing a fused image that exhibits superior visual quality with a minimum of two input images. The proposed work begins with refining the input images using an exposure correction strategy and a recursive filter. The refined images along with a selected exposure corrected image form an intermediate exposure stack. A weight map is constructed from the intermediate exposure stack using a set of weighting terms and a fast-guided filter is used to prevent the weight maps from being distorted by external anomalies. Finally, the fusion is performed using the pyramidal decomposition of weight maps and intermediate images. The resultant fused images obtained have been tested and proven to hold superior performance over the state-of-the-art methods in perceptual and empirical analysis. Benefiting from this approach is a framework that seeks fewer input images, an intermediate exposure stack, and a detail-enriched fused image.

Enhanced LDR Detail Rendering for HDR Fusion by TransU- Fusion Network

High Dynamic Range (HDR) images are widely used in automotive, aerospace, AI, and other fields but are limited by the maximum dynamic range of a single data acquisition using CMOS image sensors. High dynamic range images are usually synthesized through multiple exposure techniques and image processing techniques. One of the most challenging task in multiframe Low Dynamic Range (LDR) images fusion for HDR is to eliminate ghosting artifacts caused by motion.In traditional algorithms, optical flow is generally used to align dynamic scenes before image fusion, which can achieve good results in cases of small-scale motion scenes but causes obvious ghosting artifacts when motion magnitude is large. Recently, attention mechanisms have been introduced during the alignment stage to enhance the network’s ability to remove ghosts. However, significant ghosting artifacts still occur in some scenarios with large-scale motion or oversaturated areas. We proposea novel Distilled Feature TransformerBlock (DFTB) structure to distill and re-extract information from deep image features obtained after U-Net downsampling, achieving ghost removal at the semantic level for HDR fusion. We introduce a Feature Distillation Transformer Block (FDTB), based on the Swin-Transformer and RFDB structure. FDTB uses multiple distillation connections to learn more discriminative feature representations. For the multiexposure moving scene image fusion HDR ghost removal task, in the previous method, the use of deep learning to remove the ghost effect in the composite image has been perfect, and it is almost difficult to observe the ghost residue of moving objects in the composite HDR image. The method in this paper focuses more on how to save the details of LDR image more completely after removing the ghost to synthesize high-quality HDR image. After using the proposed FDTB, the edge texture details of the synthesized HDR image are saved more perfectly, which shows that FDTB has a better effect in saving the details of image fusion. Futhermore, we propose a new depth framework based on DFTB for fusing and removing ghosts from deep image features, called TransU-Fusion. First of all, we use the encoder in U-Net to extract image features of different exposures and map them to different dimensional feature spaces. By utilizing the symmetry of the U-Net structure, we can ultimately output these feature images as original size HDR images. Then, we further fuse high-dimensional space features using Dilated Residual Dense Block (DRDB) to expand the receptive field, which is beneficial for repairing over-saturated regions. We use the transformer in DFTB to perform low-pass filtering on low-dimensional space features and interact with global information to remove ghosts. Finally, the processed features are merged and output as an HDR image without ghosting artifacts through the decoder. After testing on datasets and comparing with benchmark and state-of-the-art models, the results demonstrate our model’s excellent information fusion ability and stronger ghost removal capability.

Hybrid High Dynamic Range Imaging fusing Neuromorphic and Conventional Images.

Reconstruction of high dynamic range image from a single low dynamic range image captured by a conventional RGB camera, which suffers from over- or under-exposure, is an ill-posed problem. In contrast, recent neuromorphic cameras like event camera and spike camera can record high dynamic range scenes in the form of intensity maps, but with much lower spatial resolution and no color information. In this paper, we propose a hybrid imaging system (denoted as NeurImg) that captures and fuses the visual information from a neuromorphic camera and ordinary images from an RGB camera to reconstruct high-quality high dynamic range images and videos. The proposed NeurImg-HDR+ network consists of specially designed modules, which bridges the domain gaps on resolution, dynamic range, and color representation between two types of sensors and images to reconstruct high-resolution, high dynamic range images and videos. We capture a test dataset of hybrid signals on various HDR scenes using the hybrid camera, and analyze the advantages of the proposed fusing strategy by comparing it to state-of-the-art inverse tone mapping methods and merging two low dynamic range images approaches. Quantitative and qualitative experiments on both synthetic data and real-world scenarios demonstrate the effectiveness of the proposed hybrid high dynamic range imaging system. Code and dataset can be found at: https://github.com/hjynwa/NeurImg-HDR.