Multi-exposure Image Fusion Research Articles

An ultra-high-definition multi-exposure image fusion method based on multi-scale feature extraction

Multiple exposure image fusion is a technique used to obtain high dynamic range images. Due to its low cost and high efficiency, it has received a lot of attention from researchers in recent years. Currently, most deep learning-based multiple exposure image fusion methods extract features from different exposure images using a single feature extraction method. Some methods simply rely on two different modules to directly extract features. However, this approach inevitably leads to the loss of some feature information during the feature extraction process, thus further affecting the performance of the model. To minimize the loss of feature information as much as possible, we propose an ultra-high-definition (UHD) multiple exposure image fusion method based on multi-scale feature extraction. The method adopts a U-shaped structure to construct the overall network model, which can fully exploit the feature information at different levels. Additionally, we construct a novel hybrid stacking paradigm to combine convolutional neural networks and Transformer modules. This combined module can extract both local texture features and global color features simultaneously. To more efficiently fuse and extract features, we also design a cross-layer feature fusion module, which can adaptively learn the correlation between features at different layers. Numerous quantitative and qualitative results demonstrate that our proposed method performs well in UHD multiple exposure image fusion.

Research on metal surface specular removal algorithm based on unsupervised learning

The highlights on the surface of metal materials can seriously destroy the continuity of the image, produce certain false edges, and cause the texture details in the highlights area to weaken or even disappear, which interferes with the subsequent operations such as surface region segmentation and defect detection. Aiming at the low efficiency, high loss, easy distortion and difficult calibration of metal highlight data, an unsupervised perceptual enhancement network model based on the convolutional neural network (CNN) is proposed. Firstly, the method of generating antagonism is used to generate a large number of metal images with high-light feature information, which is used to increase the number of high-light metal image data sets in the training set. Secondly, a detail enhance model(DEM) and a color enhance model(CEM) are introduced into the context aggregation network to improve the feature detail retention rate in the low resolution weight graphs. Finally, the multi-scale structural similarity function is used to replace the original structural similarity function to solve the insensitive detail when the image size is too large. Experiments show that comparing with other multi-exposure image fusion models, the present model can improve the evaluation index of mutual information and average gradient of fused images by about 10%, and can retain more texture feature information.

Bayesian multi-exposure image fusion for robust high dynamic range ptychography

The limited dynamic range of the detector can impede coherent diffractive imaging (CDI) schemes from achieving diffraction-limited resolution. To overcome this limitation, a straightforward approach is to utilize high dynamic range (HDR) imaging through multi-exposure image fusion (MEF). This method involves capturing measurements at different exposure times, spanning from under to overexposure and fusing them into a single HDR image. The conventional MEF technique in ptychography typically involves subtracting the background noise, ignoring the saturated pixels and then merging the acquisitions. However, this approach is inadequate under conditions of low signal-to-noise ratio (SNR). Additionally, variations in illumination intensity significantly affect the phase retrieval process. To address these issues, we propose a Bayesian MEF modeling approach based on a modified Poisson distribution that takes the background and saturation into account. The expectation-maximization (EM) algorithm is employed to infer the model parameters. As demonstrated with synthetic and experimental data, our approach outperforms the conventional MEF method, offering superior phase retrieval under challenging experimental conditions. This work underscores the significance of robust multi-exposure image fusion for ptychography, particularly in imaging shot-noise-dominated weakly scattering specimens or in cases where access to HDR detectors with high SNR is limited. Furthermore, the applicability of the Bayesian MEF approach extends beyond CDI to any imaging scheme that requires HDR treatment. Given this versatility, we provide the implementation of our algorithm as a Python package.

A dual domain multi-exposure image fusion network based on spatial-frequency integration

Multi-exposure image fusion aims to generate a single high-dynamic image by integrating images with different exposures. Existing deep learning-based multi-exposure image fusion methods primarily focus on spatial domain fusion, neglecting the global modeling ability of the frequency domain. To effectively leverage the global illumination modeling ability of the frequency domain, we propose a novelty perspective on multi-exposure image fusion via the Spatial-Frequency Integration Framework, named MEF-SFI. Initially, we revisit the properties of the Fourier transform on the 2D image, and verify the feasibility of multi-exposure image fusion in the frequency domain where the amplitude and phase component is able to guide the integration of the illumination information. Subsequently, we present the deep Fourier-based multi-exposure image fusion framework, which consists of a spatial path and frequency path for local and global modeling separately. Specifically, we introduce a Spatial-Frequency Fusion Module to facilitate efficient interaction between dual domains and capture complementary information from input images with different exposures. Finally, we combine a dual domain loss function to ensure the retention of complementary information in both the spatial and frequency domains. Extensive experiments on the PQA-MEF dataset demonstrate that our method achieves visual-appealing fusion results against state-of-the-art multi-exposure image fusion approaches. Our code is available at https://github.com/SSyangguang/MEF-freq.

Leveraging a self-adaptive mean teacher model for semi-supervised multi-exposure image fusion

Deep learning-based methods have recently shown remarkable advancements in multi-exposure image fusion (MEF), demonstrating significant achievements in improving the fusion quality. Despite their success, the majority of reference images in MEF are artificially generated, inevitably introducing a portion of low-quality ones. Existing methods either utilize these mixed-quality reference images for supervised learning or heavily depend on source images for unsupervised learning, making the fusion results challenging to accurately reflect real-world illumination conditions. To overcome the impact of unreliable factors in references, we propose a self-adaptive mean teacher-based semi-supervised learning framework tailored for MEF, termed SAMT-MEF. Its self-adaptiveness is reflected from two perspectives. Firstly, we establish a self-adaptive set to retain the best-ever outputs from the teacher as pseudo labels, employing a well-crafted hybrid metric for its updates. Secondly, we employ contrastive learning to assist the self-adaptive set further in alleviating overfitting to inferior pseudo labels. Our proposed method, backed by abundant empirical evidence, outperforms state-of-the-art methods quantitatively and qualitatively on both reference and non-reference datasets. Furthermore, in some scenarios, the fusion results surpass the reference images, showcasing superior performance in practical applications. Source code are publicly available at https://github.com/hqj9994ever/SAMT-MEF.

Halo reduction multi-exposure image fusion technique

Halo reduction multi-exposure image fusion technique

CurveMEF: Multi-exposure fusion via curve embedding network

Multi-exposure image fusion (MEF) aims to combine multiple images with different exposures into a single image to improve visual quality and preserve details. This paper proposes a curve embedding network for MEF (CurveMEF), which formulates the MEF task as estimating optimal fusion curves based on state feedback using pixel intensities as state variables. The fusion curve is embedded into CurveNet, a lightweight deep network, as a physical prior constraint. Leveraging the proposed physical informed MEF method, the fusion curve can adaptively adjust the pixel distribution of overexposed and underexposed regions based on the pixel intensities, and distinguish the importance of source images. CurveMEF supports both RGB and luminance channel inputs, demonstrating its flexibility. Experimental results show that CurveMEF achieves competitive performance compared to state-of-the-art methods in both qualitative and quantitative analysis. Moreover, CurveNet requires significantly fewer parameters and training data, enabling fast training and inference. The proposed method delivers high-speed and high-quality fusion while significantly reducing computational, providing an efficient and cost-effective solution. The code is publicly available at: https://github.com/PiratePai/CurveMEF.

Multi-exposure image fusion using structural weights and visual saliency map

Multi-exposure image fusion using structural weights and visual saliency map

RDGMEF: a multi-exposure image fusion framework based on Retinex decompostion and guided filter

RDGMEF: a multi-exposure image fusion framework based on Retinex decompostion and guided filter

LGABL: UHD Multi-Exposure Image Fusion via Local and Global Aware Bilateral Learning

LGABL: UHD Multi-Exposure Image Fusion via Local and Global Aware Bilateral Learning

Hybrid-Supervised Dual-Search: Leveraging Automatic Learning for Loss-Free Multi-Exposure Image Fusion

Multi-exposure image fusion (MEF) has emerged as a prominent solution to address the limitations of digital imaging in representing varied exposure levels. Despite its advancements, the field grapples with challenges, notably the reliance on manual designs for network structures and loss functions, and the constraints of utilizing simulated reference images as ground truths. Consequently, current methodologies often suffer from color distortions and exposure artifacts, further complicating the quest for authentic image representation. In addressing these challenges, this paper presents a Hybrid-Supervised Dual-Search approach for MEF, dubbed HSDS-MEF, which introduces a bi-level optimization search scheme for automatic design of both network structures and loss functions. More specifically, we harness a unique dual research mechanism rooted in a novel weighted structure refinement architecture search. Besides, a hybrid supervised contrast constraint seamlessly guides and integrates with searching process, facilitating a more adaptive and comprehensive search for optimal loss functions. We realize the state-of-the-art performance in comparison to various competitive schemes, yielding a 10.61% and 4.38% improvement in Visual Information Fidelity (VIF) for general and no-reference scenarios, respectively, while providing results with high contrast, rich details and colors. The code is available at https://github.com/RollingPlain/HSDS_MEF.

Ref-MEF: Reference-Guided Flexible Gated Image Reconstruction Network for Multi-Exposure Image Fusion.

Multi-exposure image fusion (MEF) is a computational approach that amalgamates multiple images, each captured at varying exposure levels, into a singular, high-quality image that faithfully encapsulates the visual information from all the contributing images. Deep learning-based MEF methodologies often confront obstacles due to the inherent inflexibilities of neural network structures, presenting difficulties in dynamically handling an unpredictable amount of exposure inputs. In response to this challenge, we introduce Ref-MEF, a method for color image multi-exposure fusion guided by a reference image designed to deal with an uncertain amount of inputs. We establish a reference-guided exposure correction (REC) module based on channel attention and spatial attention, which can correct input features and enhance pre-extraction features. The exposure-guided feature fusion (EGFF) module combines original image information and uses Gaussian filter weights for feature fusion while keeping the feature dimensions constant. The image reconstruction is completed through a gated context aggregation network (GCAN) and global residual learning GRL. Our refined loss function incorporates gradient fidelity, producing high dynamic range images that are rich in detail and demonstrate superior visual quality. In evaluation metrics focused on image features, our method exhibits significant superiority and leads in holistic assessments as well. It is worth emphasizing that as the number of input images increases, our algorithm exhibits notable computational efficiency.

Robust HDR reconstruction using 3D patch based on two-scale decomposition

In this paper, we propose an effective 3D patch-based match and fusion method by taking account of dynamic or multi-view scenes in a multi-exposure image sequence using two-scale decomposition. As opposed to most multi-exposure image fusion methods, the proposed method does not require a pre-alignment step to reduce ghosting artifacts. Considering that pixel values are affected by multi-view or multi-exposure scenes, we use a uniform matching approach to match and find similar patches in different exposure images and then fuse them at each scale. By searching all similar patches (instead of the optimal patch) in the searching window to form the 3D patch when facing limited image information, we can fully use the complementary information of the multi-exposure images to preserve information about moving objects and scene details. The experimental results show that the proposed method not only performs well on dynamic scenes but also consistently generates high-quality fused images in multi-view scenes.

MIN-MEF: Multi-scale Interaction Network for Multi-exposure Image Fusion

MIN-MEF: Multi-scale Interaction Network for Multi-exposure Image Fusion

MERF: A Practical HDR-Like Image Generator via Mutual-Guided Learning Between Multi-Exposure Registration and Fusion.

In this paper, we present a novel high dynamic range (HDR)-like image generator that utilizes mutual-guided learning between multi-exposure registration and fusion, leading to promising dynamic multi-exposure image fusion. The method consists of three main components: the registration network, the fusion network, and the dual attention network which seamlessly integrates registration and fusion processes. Initially, within the registration network, the estimation of deformation fields among multi-exposure image sequences is conducted following an exposure-invariant feature extraction phase. This leads to enhanced accuracy by mitigating discrepancies across domains. Subsequently, the fusion network utilizes a progressive frequency fusion module in two distinct stages, addressing color correction and detail preservation within low and high-frequency domains, respectively. To facilitate the mutual enhancement of the registration and fusion networks, we undertake a mutual-guided learning strategy encompassing their physical connection and constraint paradigm. Firstly, a dual attention network bridges the registration and fusion networks, addressing ghosting, which is beyond the scope of registration and facilitates information exchange between input images. Secondly, a meticulously designed generative adversarial network-like iterative training schema guides the overall network framework, thereby yielding high-quality HDR-like images through mutual enhancement. Comprehensive experiments on publicly available datasets validate the superiority of our method over existing state-of-the-art approaches.

OFPF-MEF: An Optical Flow Guided Dynamic Multi-Exposure Image Fusion Network With Progressive Frequencies Learning

OFPF-MEF: An Optical Flow Guided Dynamic Multi-Exposure Image Fusion Network With Progressive Frequencies Learning

Single image defogging via multi-exposure image fusion and detail enhancement

Outdoor cameras play an important role in monitoring security and social governance. As a common weather phenomenon, haze can easily affect the quality of camera shooting, resulting in loss and distortion of image details. This paper proposes an improved multi-exposure image fusion defogging technique based on the artificial multi-exposure image fusion (AMEF) algorithm. First, the foggy image is adaptively exposed, and the fused image is subsequently obtained via multiple exposures. The fusion weight is determined by the saturation, contrast, and brightness. Finally, the image fused by a multi-scale Laplacian algorithm is enhanced with simple adaptive details to obtain a clearer defogging image. It is subjectively and objectively verified that this algorithm can obtain more image details and distinct picture colors without a priori information, effectively improving the defogging ability.

DDFusion: An efficient multi-exposure fusion network with dense pyramidal convolution and de-correlation fusion

In this work, we propose DDFusion, a novel multi-exposure image fusion network. DDFusion addresses the limitations of existing methods by effectively recovering details near extremely bright regions and learning associations between non-contiguous regions. To achieve this, our network incorporates a dense pyramidal (DensePy) convolution block in the encoder for multi-scale feature extraction, and a de-correlation fusion (DF) block for enabling structurally coherent and edge-preserving multi-scale feature fusion. It facilitates a smoother transition from highlighted areas to adjacent regions in the fused image. Experimental results demonstrate the superiority of DDFusion over state-of-the-art deep methods in terms of both visual quality and quantitative evaluation. Moreover, DDFusion achieves stronger multi-scale feature extraction capability with smaller computational complexity.

Multi exposure image fusion based on exposure correction and input refinement using limited low dynamic range images

Multi-Exposure Image Fusion (MEF) is an image processing technique that combines several images taken at various exposure levels into one high-quality image. The generation of artifacts and diminished perceptual quality of the fused image under few input images is an inevitable problem in MEF approaches. Although numerous MEF techniques have been proposed in the literature, little effort has been devoted to generate high quality fused images with minimal input images. The proposed work unveils a method for producing a fused image that exhibits superior visual quality with a minimum of two input images. The proposed work begins with refining the input images using an exposure correction strategy and a recursive filter. The refined images along with a selected exposure corrected image form an intermediate exposure stack. A weight map is constructed from the intermediate exposure stack using a set of weighting terms and a fast-guided filter is used to prevent the weight maps from being distorted by external anomalies. Finally, the fusion is performed using the pyramidal decomposition of weight maps and intermediate images. The resultant fused images obtained have been tested and proven to hold superior performance over the state-of-the-art methods in perceptual and empirical analysis. Benefiting from this approach is a framework that seeks fewer input images, an intermediate exposure stack, and a detail-enriched fused image.

Low-light image enhancement based on virtual exposure

Under poor illumination, the image information captured by a camera is partially lost, which seriously affects the visual perception of the human. Inspired by the idea that the fusion of multiexposure images can yield one high-quality image, an adaptive enhancement framework for a single low-light image is proposed based on the strategy of virtual exposure. In this framework, the exposure control parameters are adaptively generated through a statistical analysis of the low-light image, and a virtual exposure enhancer constructed by a quadratic function is applied to generate several image frames from a single input image. Then, on the basis of generating weight maps by three factors, i.e., contrast, saturation and saliency, the image sequences and weight images are transformed by a Laplacian pyramid and Gaussian pyramid, respectively, and multiscale fusion is implemented layer by layer. Finally, the enhanced result is obtained by pyramid reconstruction rule. Compared with the experimental results of several state-of-the-art methods on five datasets, the proposed method shows its superiority on several image quality evaluation metrics. This method requires neither image calibration nor camera response function estimation and has a more flexible application range. It can weaken the possibility of overenhancement, effectively avoid the appearance of a halo in the enhancement results, and adaptively improve the visual information fidelity.