Ghost-free High Dynamic Range Image Research Articles

EdiTor: Edge-guided Transformer for Ghost-free High Dynamic Range Imaging

Synthesizing the high dynamic range (HDR) image from multi-exposure images has been extensively studied by exploiting convolutional neural networks (CNNs) recently. Despite the remarkable progress, existing CNN-based methods have the intrinsic limitation of local receptive field, which hinders the model’s capability of capturing long-range correspondence and large motions across under/over-exposure images, resulting in ghosting artifacts of dynamic scenes. To address the above challenge, we propose a novel Ed ge-gu i ded T ransf or mer framework (EdiTor) customized for ghost-free HDR reconstruction, where the long-range motions across different exposures can be delicately modeled by incorporating the edge prior. Specifically, EdiTor calculates patch-wise correlation maps on both image and edge domains, enabling the network to effectively model the global movements and the fine-grained shifts across multiple exposures. Based on this framework, we further propose an exposure-masked loss to adaptively compensate for the severely distorted regions ( e.g. , highlights and shadows). Experiments demonstrate that EdiTor outperforms state-of-the-art methods both quantitatively and qualitatively, achieving appealing HDR visualization with unified textures and colors.

Deep Unrolled Low-Rank Tensor Completion for High Dynamic Range Imaging.

The major challenge in high dynamic range (HDR) imaging for dynamic scenes is suppressing ghosting artifacts caused by large object motions or poor exposures. Whereas recent deep learning-based approaches have shown significant synthesis performance, interpretation and analysis of their behaviors are difficult and their performance is affected by the diversity of training data. In contrast, traditional model-based approaches yield inferior synthesis performance to learning-based algorithms despite their theoretical thoroughness. In this paper, we propose an algorithm unrolling approach to ghost-free HDR image synthesis algorithm that unrolls an iterative low-rank tensor completion algorithm into deep neural networks to take advantage of the merits of both learning- and model-based approaches while overcoming their weaknesses. First, we formulate ghost-free HDR image synthesis as a low-rank tensor completion problem by assuming the low-rank structure of the tensor constructed from low dynamic range (LDR) images and linear dependency among LDR images. We also define two regularization functions to compensate for modeling inaccuracy by extracting hidden model information. Then, we solve the problem efficiently using an iterative optimization algorithm by reformulating it into a series of subproblems. Finally, we unroll the iterative algorithm into a series of blocks corresponding to each iteration, in which the optimization variables are updated by rigorous closed-form solutions and the regularizers are updated by learned deep neural networks. Experimental results on different datasets show that the proposed algorithm provides better HDR image synthesis performance with superior robustness compared with state-of-the-art algorithms, while using significantly fewer training samples.

Dual-Attention-Guided Network for Ghost-Free High Dynamic Range Imaging

Dual-Attention-Guided Network for Ghost-Free High Dynamic Range Imaging

MIEHDR CNN: Main Image Enhancement based Ghost-Free High Dynamic Range Imaging using Dual-Lens Systems

We study the High Dynamic Range (HDR) imaging problem using two Low Dynamic Range (LDR) images that are shot from dual-lens systems in a single shot time with different exposures. In most of the related HDR imaging methods, the problem is usually solved by Multiple Images Merging, i.e. the final HDR image is fused from pixels of all the input LDR images. However, ghost artifacts can be hardly avoided using this strategy. Instead of directly merging the multiple LDR inputs, we use an indirect way which enhances the main image, i.e. the short exposure image IS, using the long exposure image IL serving as guidance. In detail, we propose a new model, named MIEHDR CNN model, which consists of three subnets, i.e. Soft Warp CNN, 3D Guided Denoising CNN and Fusion CNN. The Soft Warp CNN aligns IL to get the aligned result ILA using the soft exposed result of IS as reference. The 3D Guided Denoising CNN denoises the soft exposed result of IS using ILA as guidance, whose result are fed into the Fusion CNN with IS to get the HDR result. The MIEHDR CNN model is implemented by MindSpore and experimental results show that we can outperform related methods largely and avoid ghost artifacts.

High dynamic range imaging by sparse representation

High dynamic range (HDR) imaging technology is becoming increasingly popular in various applications. A common approach to get an HDR image is the multiple exposed images fusion. However, the phenomenon of ghosting artifacts is brought in for the scene with non-static objects. This paper proposes a ghost-free HDR image synthesis algorithm that utilizes a sparse representation framework. Based on the dependency among adjacent low dynamic range (LDR) images and the sparsity of the moving object that leads to the ghost artifacts, we formulate the problem into two steps: moving object detection and ghost free HDR generation. In the moving object detection step, we formulate the problem as sparse representation due to the sparsity and instantaneous of the moving objects. In the HDR generation step, joint weighting is proposed to generate a ghost-free HDR image from the reference image. Experiments show that the proposed algorithm outperforms the state-of-the-art methods favorably on the textures and colors.

Computationally Efficient Truncated Nuclear Norm Minimization for High Dynamic Range Imaging.

Matrix completion is a rank minimization problem to recover a low-rank data matrix from a small subset of its entries. Since the matrix rank is nonconvex and discrete, many existing approaches approximate the matrix rank as the nuclear norm. However, the truncated nuclear norm is known to be a better approximation to the matrix rank than the nuclear norm, exploiting a priori target rank information about the problem in rank minimization. In this paper, we propose a computationally efficient truncated nuclear norm minimization algorithm for matrix completion, which we call TNNM-ALM. We reformulate the original optimization problem by introducing slack variables and considering noise in the observation. The central contribution of this paper is to solve it efficiently via the augmented Lagrange multiplier (ALM) method, where the optimization variables are updated by closed-form solutions. We apply the proposed TNNM-ALM algorithm to ghost-free high dynamic range imaging by exploiting the low-rank structure of irradiance maps from low dynamic range images. Experimental results on both synthetic and real visual data show that the proposed algorithm achieves significantly lower reconstruction errors and superior robustness against noise than the conventional approaches, while providing substantial improvement in speed, thereby applicable to a wide range of imaging applications.

Ghost-Free High Dynamic Range Imaging via Rank Minimization

We propose a ghost-free high dynamic range (HDR) image synthesis algorithm using a low-rank matrix completion framework, which we call RM-HDR. Based on the assumption that irradiance maps are linearly related to low dynamic range (LDR) image exposures, we formulate ghost region detection as a rank minimization problem. We incorporate constraints on moving objects, i.e., sparsity, connectivity, and priors on under- and over-exposed regions into the framework. Experiments on real image collections show that the RM-HDR can often provide significant gains in synthesized HDR image quality over state-of-the-art approaches. Additionally, a complexity analysis is performed which reveals computational merits of RM-HDR over recent advances in deghosting for HDR.

Spatially Adaptive Histogram Equalization for Single Image-Based Ghost-free High Dynamic Range Imaging

In this paper, we present a spatially adaptive histogram equalization method for generating a ghost-free high dynamic range (HDR) image using a single input image. Existing multiple input-based HDR methods fuse multiple low dynamic range (LDR) images, acquired using different exposures. However, these methods work only under the assumption that neither global nor local motions exists in between the LDR images. To overcome such an unrealistic constraint, we generate two LDR images from a single input image. To generate LDR images with appropriate exposures, we divide the entire intensity range into multiple sub-ranges using histogram quantization and separately perform histogram equalization in each sub-range. Thus, we can acquire a set of differently exposed LDR images of the same scene, which are then fused to generate a ghost-free HDR image. The major contribution of this work is twofold: (i) a novel estimation method for providing optimal sub-ranges of intensity using histogram quantization, which (ii) requires no additional hardware for generating multiple LDR images. Because the proposed method uses a set of optimally self-generated LDR images, it is inherently free of ghost artifacts and can provide a ghost-free HDR function for low-cost, lightweight imaging devices, such as mobile phones and compact cameras.

Spatially Adaptive Histogram Equalization for Single Image-Based Ghost-free High Dynamic Range Imaging

Spatially Adaptive Histogram Equalization for Single Image-Based Ghost-free High Dynamic Range Imaging

Ghost-Free High Dynamic Range Imaging Using Histogram Separation and Edge Preserving Denoising

In this paper, we introduce a ghost-free High Dynamic Range imaging algorithm for obtaining ghost-free high dynamicrange (HDR) images. The multiple image fusion based HDR method work only on condition that there is no movement ofcamera and object when capturing multiple, differently exposed low dynamic range (LDR) images. The proposed algorithmmakes three LDR images from a single input image to remove such an unrealistic condition. For this purpose a histogramseparation method is proposed in the algorithm for generating three LDR images by stretching each separated histogram.An edge-preserving denoising technique is also proposed in the algorithm to suppress the noise that is amplified in thestretching process. In the proposed algorithm final HDR image free from ghost artifacts in dynamic environment because itself-generates three LDR images from a single input image. Therefore, the proposed algorithm can be use in mobilephone camera and a consumer compact camera to provide the ghost artifacts free HDR images in the form of either inbuiltor post-processing software application.

Ghost detection and removal for high dynamic range images: Recent advances

High dynamic range (HDR) image generation and display technologies are becoming increasingly popular in various applications. A standard and commonly used approach to obtain an HDR image is the multiple exposures' fusion technique which consists of combining multiple images of the same scene with varying exposure times. However, if the scene is not static during the sequence acquisition, moving objects manifest themselves as ghosting artefacts in the final HDR image. Detecting and removing ghosting artefacts is an important issue for automatically generating HDR images of dynamic scenes. The aim of this paper is to provide an up-to-date review of the recently proposed methods for ghost-free HDR image generation. Moreover, a classification and comparison of the reviewed methods is reported to serve as a useful guide for future research on this topic.

Single image-based ghost-free high dynamic range imaging using local histogram stretching and spatially-adaptive denoising

In this paper, we present a novel high dynamic range (HDR) imaging method using a single image. The existing multiple image-based HDR methods work only on condition that there is no camera and object movement when acquiring multiple, differently exposed low dynamic range (LDR) images. To overcome such an unrealistic restriction, we make three LDR images from a single input image using local histogram stretching. An edge-preserving spatially adaptive denoising method is also proposed to suppress the noise that is amplified in the histogram stretching process. Because the proposed method self-generates three histogram-stretched LDR images from a single input image, ghost artifacts that are the result of the relative motion between the camera and objects during exposure time, are inherently removed. Therefore, the proposed method can be applied to mobile imaging devices such as a mobile phone camera and a consumer compact camera to provide the ghost artifacts free HDR function in the form of either embedded or post-processing software.

Improved elastic registration for removing ghost artifacts in high dynamic imaging

In this paper, we present a ghost artifacts removing method for obtaining artifact-free high dynamic range (HDR) images in the presence of camera movement. The existing HDR methods work on condition that there is no camera movement when acquiring multiple low dynamic range (LDR) images. For overcoming such unrealistic restriction, we register the multiple LDR images by using a novel target frame finding method. Ghost artifacts are removed in the resulting HDR images since the registration process of LDR images compensates undesired camera motion. Since the proposed method can embed the self compensating function into an imaging device, Ghost-free HDR imaging application can be extended to most consumer cameras, such as mobile phone cameras, point-and-shoot cameras, and digital single lens reflected (DSLR) cameras without using additional stabilizing equipments.