High Dynamic Range Image Synthesis Research Articles

Deep Unrolled Low-Rank Tensor Completion for High Dynamic Range Imaging.

The major challenge in high dynamic range (HDR) imaging for dynamic scenes is suppressing ghosting artifacts caused by large object motions or poor exposures. Whereas recent deep learning-based approaches have shown significant synthesis performance, interpretation and analysis of their behaviors are difficult and their performance is affected by the diversity of training data. In contrast, traditional model-based approaches yield inferior synthesis performance to learning-based algorithms despite their theoretical thoroughness. In this paper, we propose an algorithm unrolling approach to ghost-free HDR image synthesis algorithm that unrolls an iterative low-rank tensor completion algorithm into deep neural networks to take advantage of the merits of both learning- and model-based approaches while overcoming their weaknesses. First, we formulate ghost-free HDR image synthesis as a low-rank tensor completion problem by assuming the low-rank structure of the tensor constructed from low dynamic range (LDR) images and linear dependency among LDR images. We also define two regularization functions to compensate for modeling inaccuracy by extracting hidden model information. Then, we solve the problem efficiently using an iterative optimization algorithm by reformulating it into a series of subproblems. Finally, we unroll the iterative algorithm into a series of blocks corresponding to each iteration, in which the optimization variables are updated by rigorous closed-form solutions and the regularizers are updated by learned deep neural networks. Experimental results on different datasets show that the proposed algorithm provides better HDR image synthesis performance with superior robustness compared with state-of-the-art algorithms, while using significantly fewer training samples.

End-to-End Differentiable Learning to HDR Image Synthesis for Multi-exposure Images

Recently, high dynamic range (HDR) image reconstruction based on the multiple exposure stack from a given single exposure utilizes a deep learning framework to generate high-quality HDR images. These conventional networks focus on the exposure transfer task to reconstruct the multi-exposure stack. Therefore, they often fail to fuse the multi-exposure stack into a perceptually pleasant HDR image as the inversion artifacts occur. We tackle the problem in stack reconstruction-based methods by proposing a novel framework with a fully differentiable high dynamic range imaging (HDRI) process. By explicitly using the loss, which compares the network's output with the ground truth HDR image, our framework enables a neural network that generates the multiple exposure stack for HDRI to train stably. In other words, our differentiable HDR synthesis layer helps the deep neural network to train to create multi-exposure stacks while reflecting the precise correlations between multi-exposure images in the HDRI process. In addition, our network uses the image decomposition and the recursive process to facilitate the exposure transfer task and to adaptively respond to recursion frequency. The experimental results show that the proposed network outperforms the state-of-the-art quantitative and qualitative results in terms of both the exposure transfer tasks and the whole HDRI process.

Multiview High Dynamic Range Image Synthesis Using Fuzzy Broad Learning System.

Compared with the normal low dynamic range (LDR) images, the high dynamic range (HDR) images provide more dynamic range and image details. Although the existing techniques for generating the HDR images have a good effect for static scenes, they usually produce artifacts on the HDR images for dynamic scenes. In recent years, some learning-based approaches are used to synthesize the HDR images and obtain good results. However, there are also many problems, including the deficiency of explaining and the time-consuming training process. In this article, we propose a novel approach to synthesize multiview HDR images through fuzzy broad learning system (FBLS). We use a set of multiview LDR images with different exposure as input and transfer corresponding Takagi-Sugeno (TS) fuzzy subsystems; then, the structure is expanded in a wide sense in the "enhancement groups" which transfer from the TS fuzzy rules with nonlinear transformation. After integrating fuzzy subsystems and enhancement groups with the trained-well weight, the HDR image is generated. In FBLS, applying the incremental learning algorithm and the pseudoinverse method to compute the weights can greatly reduce the training time. In addition, the fuzzy system has better interpretability. In the learning process, IF-THEN fuzzy rules can effectively help the model to detect the artifacts and reject them in the final HDR result. These advantages solve the problem of existing deep-learning methods. Furthermore, we set up a new dataset of multiview LDR images with corresponding HDR ground truth to train our system. Our experimental results show that our system can synthesize high-quality multiview HDR images, which has a higher training speed than other learning methods.

面向空间目标观测的高动态范围图像合成

面向空间目标观测的高动态范围图像合成

High-Dynamic-Range Spectral Imaging System for Omnidirectional Scene Capture

Omnidirectional imaging technology has been widely used for scene archiving. It has been a crucial technology in many fields including computer vision, image analysis and virtual reality. It should be noted that the dynamic range of luminance values in a natural scene is quite large, and the scenes containing various objects and light sources consist of various spectral power distributions. Therefore, this paper proposes a system for acquiring high dynamic range (HDR) spectral images for capturing omnidirectional scenes. The system is constructed using two programmable high-speed video cameras with specific lenses and a programmable rotating table. Two different types of color filters are mounted on the two-color video cameras for six-band image acquisition. We present several algorithms for HDR image synthesis, lens distortion correction, image registration, and omnidirectional image synthesis. Spectral power distributions of illuminants (color signals) are recovered from the captured six-band images based on the Wiener estimation algorithm. In this paper, we present two types of applications based on our imaging system: time-lapse imaging and gigapixel imaging. The performance of the proposed system is discussed in detail in terms of the system configurations, acquisition time, artifacts, and spectral estimation accuracy. Experimental results in actual scenes demonstrate that the proposed system is feasible and powerful for acquiring HDR spectral scenes through time-lapse or gigapixel omnidirectional imaging approaches. Finally, we apply the captured omnidirectional images to time-lapse spectral Computer Graphics (CG) renderings and spectral-based relighting of an indoor gigapixel image.

High dynamic range imaging by sparse representation

High dynamic range (HDR) imaging technology is becoming increasingly popular in various applications. A common approach to get an HDR image is the multiple exposed images fusion. However, the phenomenon of ghosting artifacts is brought in for the scene with non-static objects. This paper proposes a ghost-free HDR image synthesis algorithm that utilizes a sparse representation framework. Based on the dependency among adjacent low dynamic range (LDR) images and the sparsity of the moving object that leads to the ghost artifacts, we formulate the problem into two steps: moving object detection and ghost free HDR generation. In the moving object detection step, we formulate the problem as sparse representation due to the sparsity and instantaneous of the moving objects. In the HDR generation step, joint weighting is proposed to generate a ghost-free HDR image from the reference image. Experiments show that the proposed algorithm outperforms the state-of-the-art methods favorably on the textures and colors.

Multi-image high dynamic range algorithm using a hybrid camera

This paper proposes a multi-image high dynamic range (HDR) algorithm using a hybrid camera, plus a tone mapping algorithm based on sub-band decomposed multi-scale retinex (SD-MSR). Conventional multi-image HDR algorithms, which produce a single HDR image from multiple low dynamic range (LDR) images, usually perform registration by exploiting homography derived from scale-invariant feature transform matching between adjacent images to overcome severe differences in luminance. However, such a registration can cause ghost artifacts, because motion information for moving objects can be inaccurate. In order to solve this problem, we employ a hybrid camera that simultaneously produces low-resolution (LR) video sequences and high-resolution (HR) still images. So, we can estimate accurate motion from the LR video because of homogeneous luminance. Then, we apply the estimated motion information to the registration between the corresponding HR images prior to HDR image synthesis. Finally, we present a tone mapping algorithm based on SD-MSR to enhance details in the synthesized HDR image. Experimental results show that the proposed algorithms outperform state-of-the-art HDR algorithms.

Robust patch-based hdr reconstruction of dynamic scenes

High dynamic range (HDR) imaging from a set of sequential exposures is an easy way to capture high-quality images of static scenes, but suffers from artifacts for scenes with significant motion. In this paper, we propose a new approach to HDR reconstruction that draws information from all the exposures but is more robust to camera/scene motion than previous techniques. Our algorithm is based on a novel patch-based energy-minimization formulation that integrates alignment and reconstruction in a joint optimization through an equation we call the HDR image synthesis equation. This allows us to produce an HDR result that is aligned to one of the exposures yet contains information from all of them. We present results that show considerable improvement over previous approaches.