Multi-exposure Image Research Articles

Calibration Method of Camera Response Function Based on Multi-Exposure Image Sequence

Calibration Method of Camera Response Function Based on Multi-Exposure Image Sequence

Single image defogging via multi-exposure image fusion and detail enhancement

Outdoor cameras play an important role in monitoring security and social governance. As a common weather phenomenon, haze can easily affect the quality of camera shooting, resulting in loss and distortion of image details. This paper proposes an improved multi-exposure image fusion defogging technique based on the artificial multi-exposure image fusion (AMEF) algorithm. First, the foggy image is adaptively exposed, and the fused image is subsequently obtained via multiple exposures. The fusion weight is determined by the saturation, contrast, and brightness. Finally, the image fused by a multi-scale Laplacian algorithm is enhanced with simple adaptive details to obtain a clearer defogging image. It is subjectively and objectively verified that this algorithm can obtain more image details and distinct picture colors without a priori information, effectively improving the defogging ability.

DDFusion: An efficient multi-exposure fusion network with dense pyramidal convolution and de-correlation fusion

In this work, we propose DDFusion, a novel multi-exposure image fusion network. DDFusion addresses the limitations of existing methods by effectively recovering details near extremely bright regions and learning associations between non-contiguous regions. To achieve this, our network incorporates a dense pyramidal (DensePy) convolution block in the encoder for multi-scale feature extraction, and a de-correlation fusion (DF) block for enabling structurally coherent and edge-preserving multi-scale feature fusion. It facilitates a smoother transition from highlighted areas to adjacent regions in the fused image. Experimental results demonstrate the superiority of DDFusion over state-of-the-art deep methods in terms of both visual quality and quantitative evaluation. Moreover, DDFusion achieves stronger multi-scale feature extraction capability with smaller computational complexity.

Gaussian of Differences: A Simple and Efficient General Image Fusion Method.

The separate analysis of images obtained from a single source using different camera settings or spectral bands, whether from one or more than one sensor, is quite difficult. To solve this problem, a single image containing all of the distinctive pieces of information in each source image is generally created by combining the images, a process called image fusion. In this paper, a simple and efficient, pixel-based image fusion method is proposed that relies on weighting the edge information associated with each pixel of all of the source images proportional to the distance from their neighbors by employing a Gaussian filter. The proposed method, Gaussian of differences (GD), was evaluated using multi-modal medical images, multi-sensor visible and infrared images, multi-focus images, and multi-exposure images, and was compared to existing state-of-the-art fusion methods by utilizing objective fusion quality metrics. The parameters of the GD method are further enhanced by employing the pattern search (PS) algorithm, resulting in an adaptive optimization strategy. Extensive experiments illustrated that the proposed GD fusion method ranked better on average than others in terms of objective quality metrics and CPU time consumption.

Multi exposure image fusion based on exposure correction and input refinement using limited low dynamic range images

Multi-Exposure Image Fusion (MEF) is an image processing technique that combines several images taken at various exposure levels into one high-quality image. The generation of artifacts and diminished perceptual quality of the fused image under few input images is an inevitable problem in MEF approaches. Although numerous MEF techniques have been proposed in the literature, little effort has been devoted to generate high quality fused images with minimal input images. The proposed work unveils a method for producing a fused image that exhibits superior visual quality with a minimum of two input images. The proposed work begins with refining the input images using an exposure correction strategy and a recursive filter. The refined images along with a selected exposure corrected image form an intermediate exposure stack. A weight map is constructed from the intermediate exposure stack using a set of weighting terms and a fast-guided filter is used to prevent the weight maps from being distorted by external anomalies. Finally, the fusion is performed using the pyramidal decomposition of weight maps and intermediate images. The resultant fused images obtained have been tested and proven to hold superior performance over the state-of-the-art methods in perceptual and empirical analysis. Benefiting from this approach is a framework that seeks fewer input images, an intermediate exposure stack, and a detail-enriched fused image.

Low-light image enhancement based on virtual exposure

Under poor illumination, the image information captured by a camera is partially lost, which seriously affects the visual perception of the human. Inspired by the idea that the fusion of multiexposure images can yield one high-quality image, an adaptive enhancement framework for a single low-light image is proposed based on the strategy of virtual exposure. In this framework, the exposure control parameters are adaptively generated through a statistical analysis of the low-light image, and a virtual exposure enhancer constructed by a quadratic function is applied to generate several image frames from a single input image. Then, on the basis of generating weight maps by three factors, i.e., contrast, saturation and saliency, the image sequences and weight images are transformed by a Laplacian pyramid and Gaussian pyramid, respectively, and multiscale fusion is implemented layer by layer. Finally, the enhanced result is obtained by pyramid reconstruction rule. Compared with the experimental results of several state-of-the-art methods on five datasets, the proposed method shows its superiority on several image quality evaluation metrics. This method requires neither image calibration nor camera response function estimation and has a more flexible application range. It can weaken the possibility of overenhancement, effectively avoid the appearance of a halo in the enhancement results, and adaptively improve the visual information fidelity.

Unsupervised Multi-Exposure Image Fusion Breaking Exposure Limits via Contrastive Learning

This paper proposes an unsupervised multi-exposure image fusion (MEF) method via contrastive learning, termed as MEF-CL. It breaks exposure limits and performance bottleneck faced by existing methods. MEF-CL firstly designs similarity constraints to preserve contents in source images. It eliminates the need for ground truth (actually not exist and created artificially) and thus avoids negative impacts of inappropriate ground truth on performance and generalization. Moreover, we explore a latent feature space and apply contrastive learning in this space to guide fused image to approximate normal-light samples and stay away from inappropriately exposed ones. In this way, characteristics of fused images (e.g., illumination, colors) can be further improved without being subject to source images. Therefore, MEF-CL is applicable to image pairs of any multiple exposures rather than a pair of under-exposed and over-exposed images mandated by existing methods. By alleviating dependence on source images, MEF-CL shows better generalization for various scenes. Consequently, our results exhibit appropriate illumination, detailed textures, and saturated colors. Qualitative, quantitative, and ablation experiments validate the superiority and generalization of MEF-CL. Our code is publicly available at https://github.com/hanna-xu/MEF-CL.

EMEF: Ensemble Multi-Exposure Image Fusion

Although remarkable progress has been made in recent years, current multi-exposure image fusion (MEF) research is still bounded by the lack of real ground truth, objective evaluation function, and robust fusion strategy. In this paper, we study the MEF problem from a new perspective. We don’t utilize any synthesized ground truth, design any loss function, or develop any fusion strategy. Our proposed method EMEF takes advantage of the wisdom of multiple imperfect MEF contributors including both conventional and deep learning-based methods. Specifically, EMEF consists of two main stages: pre-train an imitator network and tune the imitator in the runtime. In the first stage, we make a unified network imitate different MEF targets in a style modulation way. In the second stage, we tune the imitator network by optimizing the style code, in order to find an optimal fusion result for each input pair. In the experiment, we construct EMEF from four state-of-the-art MEF methods and then make comparisons with the individuals and several other competitive methods on the latest released MEF benchmark dataset. The promising experimental results demonstrate that our ensemble framework can “get the best of all worlds”. The code is available at https://github.com/medalwill/EMEF.

Multi-exposure image fusion via perception enhanced structural patch decomposition

Multi-exposure image fusion (MEF) is an affordable and convenient option for high-dynamic-range imaging. Current MEF methods are prone to visually unrealistic results since they take no account of perceptual factors. To address this problem, a multi-exposure image fusion method is proposed based on perception enhanced structural patch decomposition, namely PESPD-MEF. An image patch is first decomposed into four components: perceptual gain, signal strength, signal structure, and mean intensity. Then, the enhancement rule is designed for perceptual gain, and the latter three elements are fused independently in different ways. Finally, the fused components are aggregated to generate informative and perception-realistic results. Moreover, the multi-scale framework is adopted to boost the fused performance. The proposed method is also extended to address single low-light image enhancement issue. Experimental results demonstrate that the proposed approach outperforms the state-of-the-art methods by a large margin in terms of perceptual realism. The source code is available at https://github.com/Junchao2018/PESPD-MEF.

Study on a Low-Illumination Enhancement Method for Online Monitoring Images Considering Multiple-Exposure Image Sequence Fusion

In order to improve the problem of low image quality caused by insufficient illumination, a low-light image enhancement method with robustness is proposed, which can effectively handle extremely dark images while achieving good results for scenes with insufficient local illumination. First, we expose the images to different degrees to form a multi-exposure image sequence; then, we introduce global-based luminance weights and contrast-based gradient weights to fuse the multi-exposure image sequence; finally, we use a bootstrap filter to suppress the noise that may occur during the image processing. We employ pertinent assessment criteria, such as the Peak Signal to Noise Ratio (PSNR), Structural Similarity (SSIM), the Average Gradient (AG), and the Figure Definition (FD), to assess how well the method enhances. Experimental results show that PSNR (31.32) and SSIM (0.74) are the highest in pretty dark scenes compared to most conventional algorithms such as MF, BIMEF, LECARM, etc. Similarly, in processing uneven illumination such as “moonlit night” images, the AG (10.21) and the FD (14.54) are at maximum. In addition, other evaluation metrics such as Shannon (SH) are optimal in the above scenarios. In addition, we apply the algorithm in this paper to the online monitoring images of electric power equipment, which can improve the image lightness while recovering the detail information. The algorithm has strong robustness in extremely dark images and natural low-light images, and the enhanced images have minimal distortion and best appearance in different low-light scenes.

Coarse-to-fine multi-scale attention-guided network for multi-exposure image fusion

Coarse-to-fine multi-scale attention-guided network for multi-exposure image fusion

Low-light image enhancement based on simulated multi-exposure fusion

We propose an efficient and novel framework for low-light image enhancement, which aims to reveal information hidden in the darkness and improve overall brightness and local contrast. Inspired by exposure fusion technique, we employ simulated multi-exposure images fusion to derive bright, natural and satisfactory results, while images are taken under poor conditions such as insufficient or uneven illumination, back-lit and limited exposure time. Specifically, we first design a novel method to generate synthesized images with varying exposure time from a single image. Thus, each image of these artificial sequences contains necessary information for the final desired enhanced result. We then introduce a flexible multi-exposure fusion framework to achieve fused images, which comprises a weight map prediction module and a multi-scale fusion module. Extensive experiments show that our approach can achieve similar or better performance compared to serval state-of-the-art methods.

Image Dehazing Based on Improved Color Channel Transfer and Multiexposure Fusion

Image dehazing is one of the problems that need to be solved urgently in the field of computer vision. In recent years, more and more algorithms have been applied to image dehazing and achieved good results. However, the image after dehazing still has color distortion, contrast and saturation disorder, and other challenges; in order to solve these problems, in this paper, an effective image dehazing method is proposed, which is based on improved color channel transfer and multiexposure image fusion to achieve image dehazing. First, the image is preprocessed using a color channel transfer method based on k-means. Second, gamma correction is introduced on the basis of guided filtering to obtain a series of multiexposure images, and the obtained multiexposure images are fused into a dehazed image through a Laplacian pyramid fusion scheme based on local similarity of adaptive weights. Finally, contrast and saturation corrections are performed on the dehazed image. Experimental verification is carried out on synthetic dehazed images and natural dehazed images, and it is verified that the method proposed is superior to existing dehazed algorithms from both subjective and objective aspects.

Adaptive fusion of multi-exposure images based on perceptron model

Abstract Multi-exposure image fusion as a technical means to bridge the dynamic range gap between real scenes and image acquisition devices, which makes the fused images better quality and more realistic and vivid simulation of real scenes, has been widely concerned by scholars from various countries. In order to improve the adaptive fusion effect of multi-exposure images, this paper proposes a fusion algorithm based on multilayer perceptron (MLP) based on the perceptron model and verifies the feasibility of the algorithm by the peak signal-to-noise ratio (PSNR), correlation coefficient (PCC), structural similarity (SSMI) and HDR-VDR-2, an evaluation index of HDR image quality. Comparison with other algorithms revealed that the average PSNR of the MLP algorithm improved by 4.43% over the Ma algorithm, 7.88% over the Vanmail algorithm, 10.30% over the FMMR algorithm, 11.19% over the PMF algorithm, and 11.19% over the PMF algorithm. For PCC, the MLP algorithm improves by 20.14%, 17.46%, 2.31%, 11.24%, and 15.36% over the other algorithms in that order. For SSMI, the MLP algorithm improved by 16.99%, 8.96%, 17.17%, 14.41%, and 4.85% over the other algorithms, in that order. For HDR-VDR-2, the MLP algorithm improved by 3.02%, 2.79%, 6.84%, 4.90%, and 6.55% over the other algorithms, in that order. The results show that the MLP algorithm can avoid image artifacts while retaining more details. The MLP-based adaptive fusion method is a step further in the theoretical study of multi-exposure image fusion, which is of great significance for subsequent research and practical application by related technology vendors.

The ultra-short-term photovoltaic power prediction based on multi-exposure high-resolution total sky images using deep learning

With the increasing installed capacity of renewable energy sources such as photovoltaic (PV) generation in the power system, ultra-short-term PV power prediction is more important for the stability. However, the prediction accuracy in the traditional methods cannot be guaranteed. Based on multi-exposure high-resolution total sky images (TSIs), this paper presents an advanced ultra-short-term PV prediction method using deep learning. To fully utilize the high-resolution images, an overlapping sliding window cutting and concatenating strategy are described to capture the global and local features of an image. The multi-exposure images are fused to provide more details about edge information and brightness. To better extract the image features and sequential features for PV prediction, a convolutional long–short-term memory model (CLSTM) with a multi-head self-attention mechanism is presented. The experiments use real-world datasets in the Zero Carbon Emission laboratory at Zhejiang University. The simulation results show that the proposed model can utilize TSIs to achieve the desired accuracy consistently. Under different weather conditions, the prediction accuracy of this model is improved by 49.1%–66% compared with that of other models.

Blind quality assessment of multi-exposure fused images considering the detail, structure and color characteristics.

In the process of multi-exposure image fusion (MEF), the appearance of various distortions will inevitably cause the deterioration of visual quality. It is essential to predict the visual quality of MEF images. In this work, a novel blind image quality assessment (IQA) method is proposed for MEF images considering the detail, structure, and color characteristics. Specifically, to better perceive the detail and structure distortion, based on the joint bilateral filtering, the MEF image is decomposed into two layers (i.e., the energy layer and the structure layer). Obviously, this is a symmetric process that the two decomposition results can independently and almost completely describe the information of MEF images. As the former layer contains rich intensity information and the latter captures some image structures, some energy-related and structure-related features are extracted from these two layers to perceive the detail and structure distortion phenomena. Besides, some color-related features are also obtained to present the color degradation which are combined with the above energy-related and structure-related features for quality regression. Experimental results on the public MEF image database demonstrate that the proposed method achieves higher performance than the state-of-the-art quality assessment ones.

Image dehazing algorithm based on artificial multi-exposure image fusion

Image dehazing algorithm based on artificial multi-exposure image fusion

Effective edge-aware weighting filter-based structural patch decomposition multi-exposure image fusion for single image dehazing

Effective edge-aware weighting filter-based structural patch decomposition multi-exposure image fusion for single image dehazing

AIM-MEF: Multi-exposure image fusion based on adaptive information mining in both spatial and frequency domains

Multi-Exposure Image Fusion (MEF) is a simple and effective solution to obtain high dynamic range images by integrating important information from a series of low dynamic range source images with different exposure levels. Despite promising results have been achieved, two issues remain unresolved in existing deep learning-based MEF methods. One is that the information in the frequency domain is underutilized, and the other is that the useful information of hard-to-learn pixels is not fully exploited. To address the above issues, we propose AIM-MEF: a multi-exposure image fusion framework based on adaptive information mining in both spatial and frequency domains. Concretely, we propose an image fusion loss in the frequency domain to mine the frequency domain information for the first time in the MEF field. In order to adaptively exploit the hard-to-learn but useful information in the source images, we propose a hard information mining strategy in both the spatial and the frequency domains, and perform information mining in pixel and frequency levels of the source images. In addition, we propose an adaptive fusion strategy based on information abundance to perform information mining at the image level. We compared AIM-MEF to 10 traditional and deep learning-based methods on two public datasets with eight objective metrics, and AIM-MEF outperformed these baselines in both subjective and objective evaluations. Codes will be publicly available.

An Improved Multiexposure Image Fusion Technique.

Multiexposure image fusion (MEF) is an effective approach to generate high dynamic range images from multilevel exposures taken from ordinary cameras. In this article, a novel MEF algorithm is proposed to gain maximum visual details as well as vivid colors from the captured scene. This algorithm first decomposes the input images with multiple exposures into the base and detail layer. The weights for the base and detail layers are computed by using exposedness function and then both the layers are combined to generate the final fused image. The proposed multiexposure technique requires fewer computational operations, preserves edges, and also reduces spatial artifacts. The proposed technique has been evaluated quantitatively using image quality assessment model based on structure similarity index measure for MEF. By the extensive experimental results, it has been illustrated that in addition to significantly outperforming other state-of-the-art techniques, the proposed technique is much faster and can achieve better image quality.