Multi-exposure Image Research Articles

Evaluation of YOLO Efficiency in Automatic Orange Detection in Multi-Exposure Images

Abstract. Brazil is the largest producer of oranges in the world and the automatic detection of fruits has been a challenging task in the context of remote sensing, due to variations in fruit appearance, changes in lighting and occlusions of foliage and neighboring fruits. In this sense, this paper focus on the detection of oranges in multispectral images, with different spectral bands and exposures, using a convolutional neural network (CNN) known as YOU ONLY LOOK ONCE (YOLO). The results indicate that, after 300 epochs, the model demonstrated an accuracy of 81.5% and an approximate recovery rate of 85%. Shutter speeds 1/640s and 1/250s are not suitable for detection due to low light and overexposure, respectively. Intermediate values may be more suitable for identifying a larger number of fruits.

UUD-Fusion: An unsupervised universal image fusion approach via generative diffusion model

Image fusion is a classical problem in the field of image processing whose solutions are usually not unique. The common image fusion methods can only generate a fixed fusion result based on the source image pairs. They tend to be applicable only to a specific task and have high computational costs. Hence, in this paper, a two-stage unsupervised universal image fusion with generative diffusion model is proposed, termed as UUD-Fusion. For the first stage, a strategy based on the initial fusion results is devised to offload the computational effort. For the second stage, two novel sampling algorithms based on generative diffusion model are designed. The fusion sequence generation algorithm (FSGA) searches for a series of solutions in the solution space by iterative sampling. The fusion image enhancement algorithm (FIEA) greatly improves the quality of the fused images. Qualitative and quantitative evaluations of multiple datasets with different modalities demonstrate the great versatility and effectiveness of UUD-Fusion. It is capable of solving different fusion problems, including multi-focus image fusion task, multi-exposure image fusion task, infrared and visible fusion task, and medical image fusion task. The proposed approach is superior to current state-of-the-art methods. Our code is publicly available at https://github.com/xiangxiang-wang/UUD-Fusion.

An ultra-high-definition multi-exposure image fusion method based on multi-scale feature extraction

Multiple exposure image fusion is a technique used to obtain high dynamic range images. Due to its low cost and high efficiency, it has received a lot of attention from researchers in recent years. Currently, most deep learning-based multiple exposure image fusion methods extract features from different exposure images using a single feature extraction method. Some methods simply rely on two different modules to directly extract features. However, this approach inevitably leads to the loss of some feature information during the feature extraction process, thus further affecting the performance of the model. To minimize the loss of feature information as much as possible, we propose an ultra-high-definition (UHD) multiple exposure image fusion method based on multi-scale feature extraction. The method adopts a U-shaped structure to construct the overall network model, which can fully exploit the feature information at different levels. Additionally, we construct a novel hybrid stacking paradigm to combine convolutional neural networks and Transformer modules. This combined module can extract both local texture features and global color features simultaneously. To more efficiently fuse and extract features, we also design a cross-layer feature fusion module, which can adaptively learn the correlation between features at different layers. Numerous quantitative and qualitative results demonstrate that our proposed method performs well in UHD multiple exposure image fusion.

Research on metal surface specular removal algorithm based on unsupervised learning

The highlights on the surface of metal materials can seriously destroy the continuity of the image, produce certain false edges, and cause the texture details in the highlights area to weaken or even disappear, which interferes with the subsequent operations such as surface region segmentation and defect detection. Aiming at the low efficiency, high loss, easy distortion and difficult calibration of metal highlight data, an unsupervised perceptual enhancement network model based on the convolutional neural network (CNN) is proposed. Firstly, the method of generating antagonism is used to generate a large number of metal images with high-light feature information, which is used to increase the number of high-light metal image data sets in the training set. Secondly, a detail enhance model(DEM) and a color enhance model(CEM) are introduced into the context aggregation network to improve the feature detail retention rate in the low resolution weight graphs. Finally, the multi-scale structural similarity function is used to replace the original structural similarity function to solve the insensitive detail when the image size is too large. Experiments show that comparing with other multi-exposure image fusion models, the present model can improve the evaluation index of mutual information and average gradient of fused images by about 10%, and can retain more texture feature information.

Bayesian multi-exposure image fusion for robust high dynamic range ptychography

The limited dynamic range of the detector can impede coherent diffractive imaging (CDI) schemes from achieving diffraction-limited resolution. To overcome this limitation, a straightforward approach is to utilize high dynamic range (HDR) imaging through multi-exposure image fusion (MEF). This method involves capturing measurements at different exposure times, spanning from under to overexposure and fusing them into a single HDR image. The conventional MEF technique in ptychography typically involves subtracting the background noise, ignoring the saturated pixels and then merging the acquisitions. However, this approach is inadequate under conditions of low signal-to-noise ratio (SNR). Additionally, variations in illumination intensity significantly affect the phase retrieval process. To address these issues, we propose a Bayesian MEF modeling approach based on a modified Poisson distribution that takes the background and saturation into account. The expectation-maximization (EM) algorithm is employed to infer the model parameters. As demonstrated with synthetic and experimental data, our approach outperforms the conventional MEF method, offering superior phase retrieval under challenging experimental conditions. This work underscores the significance of robust multi-exposure image fusion for ptychography, particularly in imaging shot-noise-dominated weakly scattering specimens or in cases where access to HDR detectors with high SNR is limited. Furthermore, the applicability of the Bayesian MEF approach extends beyond CDI to any imaging scheme that requires HDR treatment. Given this versatility, we provide the implementation of our algorithm as a Python package.

A dual domain multi-exposure image fusion network based on spatial-frequency integration

Multi-exposure image fusion aims to generate a single high-dynamic image by integrating images with different exposures. Existing deep learning-based multi-exposure image fusion methods primarily focus on spatial domain fusion, neglecting the global modeling ability of the frequency domain. To effectively leverage the global illumination modeling ability of the frequency domain, we propose a novelty perspective on multi-exposure image fusion via the Spatial-Frequency Integration Framework, named MEF-SFI. Initially, we revisit the properties of the Fourier transform on the 2D image, and verify the feasibility of multi-exposure image fusion in the frequency domain where the amplitude and phase component is able to guide the integration of the illumination information. Subsequently, we present the deep Fourier-based multi-exposure image fusion framework, which consists of a spatial path and frequency path for local and global modeling separately. Specifically, we introduce a Spatial-Frequency Fusion Module to facilitate efficient interaction between dual domains and capture complementary information from input images with different exposures. Finally, we combine a dual domain loss function to ensure the retention of complementary information in both the spatial and frequency domains. Extensive experiments on the PQA-MEF dataset demonstrate that our method achieves visual-appealing fusion results against state-of-the-art multi-exposure image fusion approaches. Our code is available at https://github.com/SSyangguang/MEF-freq.

Leveraging a self-adaptive mean teacher model for semi-supervised multi-exposure image fusion

Deep learning-based methods have recently shown remarkable advancements in multi-exposure image fusion (MEF), demonstrating significant achievements in improving the fusion quality. Despite their success, the majority of reference images in MEF are artificially generated, inevitably introducing a portion of low-quality ones. Existing methods either utilize these mixed-quality reference images for supervised learning or heavily depend on source images for unsupervised learning, making the fusion results challenging to accurately reflect real-world illumination conditions. To overcome the impact of unreliable factors in references, we propose a self-adaptive mean teacher-based semi-supervised learning framework tailored for MEF, termed SAMT-MEF. Its self-adaptiveness is reflected from two perspectives. Firstly, we establish a self-adaptive set to retain the best-ever outputs from the teacher as pseudo labels, employing a well-crafted hybrid metric for its updates. Secondly, we employ contrastive learning to assist the self-adaptive set further in alleviating overfitting to inferior pseudo labels. Our proposed method, backed by abundant empirical evidence, outperforms state-of-the-art methods quantitatively and qualitatively on both reference and non-reference datasets. Furthermore, in some scenarios, the fusion results surpass the reference images, showcasing superior performance in practical applications. Source code are publicly available at https://github.com/hqj9994ever/SAMT-MEF.

Halo reduction multi-exposure image fusion technique

Halo reduction multi-exposure image fusion technique

CurveMEF: Multi-exposure fusion via curve embedding network

Multi-exposure image fusion (MEF) aims to combine multiple images with different exposures into a single image to improve visual quality and preserve details. This paper proposes a curve embedding network for MEF (CurveMEF), which formulates the MEF task as estimating optimal fusion curves based on state feedback using pixel intensities as state variables. The fusion curve is embedded into CurveNet, a lightweight deep network, as a physical prior constraint. Leveraging the proposed physical informed MEF method, the fusion curve can adaptively adjust the pixel distribution of overexposed and underexposed regions based on the pixel intensities, and distinguish the importance of source images. CurveMEF supports both RGB and luminance channel inputs, demonstrating its flexibility. Experimental results show that CurveMEF achieves competitive performance compared to state-of-the-art methods in both qualitative and quantitative analysis. Moreover, CurveNet requires significantly fewer parameters and training data, enabling fast training and inference. The proposed method delivers high-speed and high-quality fusion while significantly reducing computational, providing an efficient and cost-effective solution. The code is publicly available at: https://github.com/PiratePai/CurveMEF.

Multi-exposure image fusion using structural weights and visual saliency map

Multi-exposure image fusion using structural weights and visual saliency map

EdiTor: Edge-guided Transformer for Ghost-free High Dynamic Range Imaging

Synthesizing the high dynamic range (HDR) image from multi-exposure images has been extensively studied by exploiting convolutional neural networks (CNNs) recently. Despite the remarkable progress, existing CNN-based methods have the intrinsic limitation of local receptive field, which hinders the model’s capability of capturing long-range correspondence and large motions across under/over-exposure images, resulting in ghosting artifacts of dynamic scenes. To address the above challenge, we propose a novel Ed ge-gu i ded T ransf or mer framework (EdiTor) customized for ghost-free HDR reconstruction, where the long-range motions across different exposures can be delicately modeled by incorporating the edge prior. Specifically, EdiTor calculates patch-wise correlation maps on both image and edge domains, enabling the network to effectively model the global movements and the fine-grained shifts across multiple exposures. Based on this framework, we further propose an exposure-masked loss to adaptively compensate for the severely distorted regions ( e.g. , highlights and shadows). Experiments demonstrate that EdiTor outperforms state-of-the-art methods both quantitatively and qualitatively, achieving appealing HDR visualization with unified textures and colors.

RDGMEF: a multi-exposure image fusion framework based on Retinex decompostion and guided filter

RDGMEF: a multi-exposure image fusion framework based on Retinex decompostion and guided filter

LGABL: UHD Multi-Exposure Image Fusion via Local and Global Aware Bilateral Learning

LGABL: UHD Multi-Exposure Image Fusion via Local and Global Aware Bilateral Learning

Hybrid-Supervised Dual-Search: Leveraging Automatic Learning for Loss-Free Multi-Exposure Image Fusion

Multi-exposure image fusion (MEF) has emerged as a prominent solution to address the limitations of digital imaging in representing varied exposure levels. Despite its advancements, the field grapples with challenges, notably the reliance on manual designs for network structures and loss functions, and the constraints of utilizing simulated reference images as ground truths. Consequently, current methodologies often suffer from color distortions and exposure artifacts, further complicating the quest for authentic image representation. In addressing these challenges, this paper presents a Hybrid-Supervised Dual-Search approach for MEF, dubbed HSDS-MEF, which introduces a bi-level optimization search scheme for automatic design of both network structures and loss functions. More specifically, we harness a unique dual research mechanism rooted in a novel weighted structure refinement architecture search. Besides, a hybrid supervised contrast constraint seamlessly guides and integrates with searching process, facilitating a more adaptive and comprehensive search for optimal loss functions. We realize the state-of-the-art performance in comparison to various competitive schemes, yielding a 10.61% and 4.38% improvement in Visual Information Fidelity (VIF) for general and no-reference scenarios, respectively, while providing results with high contrast, rich details and colors. The code is available at https://github.com/RollingPlain/HSDS_MEF.

Learn to enhance the low-light image via a multi-exposure generation and fusion method

In low-light image enhancement, single-exposure images contain a limited dynamic range, which hinders the restoration of contrast and texture. To address these problems, we propose a multi-exposure generation and fusion method (MEGF), which simulates multi exposure images and performs feature fusion for low light image enhancement. First, we propose a Multi-Exposure Generation (MEG) block, which generates images with different exposure levels based on the input low-light images. The MEG block employs information entropy as an evaluation measure to prevent the underexposed or overexposed image generation. Then, the Perceptual Importance based Multi-Exposure Feature Enhancement (PIMEFE) module has been developed to fuse the multi-exposure features using the Perceptual Importance-based Feature Fusion (PIFF) module. The PIFF module selects the well-exposed features from the multi-exposure features processed by the Multi Scale Recursive Feature Enhancement (MSRFE) block. Finally, the fused features are input to the Curve Adjustment (CA) block for fine-tuning and provide color enhancement to the fused features. Moreover, we propose the Multiple Exposure Recursive Fusion (MERF) module which estimates the adjustment factors for the CA block with the guidance of multi-exposure features. Experimental results demonstrate that our method outperforms other techniques in terms of image signal-to-noise ratio, structural similarity, and color accuracy on both real and synthetic datasets.

Ref-MEF: Reference-Guided Flexible Gated Image Reconstruction Network for Multi-Exposure Image Fusion.

Multi-exposure image fusion (MEF) is a computational approach that amalgamates multiple images, each captured at varying exposure levels, into a singular, high-quality image that faithfully encapsulates the visual information from all the contributing images. Deep learning-based MEF methodologies often confront obstacles due to the inherent inflexibilities of neural network structures, presenting difficulties in dynamically handling an unpredictable amount of exposure inputs. In response to this challenge, we introduce Ref-MEF, a method for color image multi-exposure fusion guided by a reference image designed to deal with an uncertain amount of inputs. We establish a reference-guided exposure correction (REC) module based on channel attention and spatial attention, which can correct input features and enhance pre-extraction features. The exposure-guided feature fusion (EGFF) module combines original image information and uses Gaussian filter weights for feature fusion while keeping the feature dimensions constant. The image reconstruction is completed through a gated context aggregation network (GCAN) and global residual learning GRL. Our refined loss function incorporates gradient fidelity, producing high dynamic range images that are rich in detail and demonstrate superior visual quality. In evaluation metrics focused on image features, our method exhibits significant superiority and leads in holistic assessments as well. It is worth emphasizing that as the number of input images increases, our algorithm exhibits notable computational efficiency.

Improved target detection method for space-based optoelectronic systems

The detection of faint and small targets by space-based surveillance systems is difficult owing to the long distances, low energies, high speeds, high false alarm rates, and low algorithmic efficiencies involved in the process. To improve space object detection and help prevent collisions with critical facilities such as satellites, this study proposes an improved method for the detection of faint and small space-based targets. The proposed method consists of two components: star atlas preprocessing and space-based target detection. The star atlas preprocessing step applies multi-exposure image pyramidal weighted fusion to the original image containing the faint and small space-based target. After obtaining the image pyramidal weighted fusion result atlas, the algorithm employs threshold segmentation to improve the overall image clarity, highlight image details, and provide additional information for target detection. The detection of targets partially relies on the local symmetry of the image. Accordingly, a diffusion function describing the local symmetry is established to precisely locate stars by measuring the symmetry factor in a small area surrounding each pixel in the star atlas. This effectively removes the background stars while retaining high-definition and high-contrast images. The efficacy of the algorithm is validated using simulated datasets consisting of space-based and real images. The results demonstrate that the proposed technique improves the applicability of the multistage hypothesis testing (MHT) method in the context of a complex space environment, thus improving the performance of the space-based electro-optical detection system to better catalogue, identify, and track space targets.

Robust HDR reconstruction using 3D patch based on two-scale decomposition

In this paper, we propose an effective 3D patch-based match and fusion method by taking account of dynamic or multi-view scenes in a multi-exposure image sequence using two-scale decomposition. As opposed to most multi-exposure image fusion methods, the proposed method does not require a pre-alignment step to reduce ghosting artifacts. Considering that pixel values are affected by multi-view or multi-exposure scenes, we use a uniform matching approach to match and find similar patches in different exposure images and then fuse them at each scale. By searching all similar patches (instead of the optimal patch) in the searching window to form the 3D patch when facing limited image information, we can fully use the complementary information of the multi-exposure images to preserve information about moving objects and scene details. The experimental results show that the proposed method not only performs well on dynamic scenes but also consistently generates high-quality fused images in multi-view scenes.

MERF: A Practical HDR-Like Image Generator via Mutual-Guided Learning Between Multi-Exposure Registration and Fusion.

In this paper, we present a novel high dynamic range (HDR)-like image generator that utilizes mutual-guided learning between multi-exposure registration and fusion, leading to promising dynamic multi-exposure image fusion. The method consists of three main components: the registration network, the fusion network, and the dual attention network which seamlessly integrates registration and fusion processes. Initially, within the registration network, the estimation of deformation fields among multi-exposure image sequences is conducted following an exposure-invariant feature extraction phase. This leads to enhanced accuracy by mitigating discrepancies across domains. Subsequently, the fusion network utilizes a progressive frequency fusion module in two distinct stages, addressing color correction and detail preservation within low and high-frequency domains, respectively. To facilitate the mutual enhancement of the registration and fusion networks, we undertake a mutual-guided learning strategy encompassing their physical connection and constraint paradigm. Firstly, a dual attention network bridges the registration and fusion networks, addressing ghosting, which is beyond the scope of registration and facilitates information exchange between input images. Secondly, a meticulously designed generative adversarial network-like iterative training schema guides the overall network framework, thereby yielding high-quality HDR-like images through mutual enhancement. Comprehensive experiments on publicly available datasets validate the superiority of our method over existing state-of-the-art approaches.

MIN-MEF: Multi-scale Interaction Network for Multi-exposure Image Fusion

MIN-MEF: Multi-scale Interaction Network for Multi-exposure Image Fusion