Multi-exposure Fusion Research Articles

Trans2Fuse: Empowering image fusion through self-supervised learning and multi-modal transformations via transformer networks

Image fusion enhances a single image by integrating information from multiple sources with complementary data. Present end-to-end fusion methods often face overfitting or intricate parameter tuning due to inadequate task-specific training data. To address this, two-stage approaches utilize encoder–decoder networks trained on extensive natural image datasets, yet suffer from limited performance due to domain disparities. In this work, we devise a novel encoder–decoder fusion framework and introduce a self-supervised scheme based on destruction–reconstruction. This approach facilitates task-specific feature learning by proposing three auxiliary tasks: pixel intensity non-linear transformation for multi-modal fusion, brightness transformation for multi-exposure fusion, and noise transformation for multi-focus fusion. By randomly selecting one task during model training, we mutually reinforce different fusion tasks, enhancing network generalizability. We innovate an encoder combining Convolutional Neural Network (CNN) and Transformer to extract both local and global features. Rigorous evaluations against 11 traditional and deep learning-based methods span four benchmark datasets: infrared-visible fusion, medical fusion, multi-exposure fusion, and multi-focus fusion. Comprehensive assessments, encompassing nine metrics from diverse viewpoints, consistently demonstrate the superior performance of our approach in all scenarios. We will make our code, datasets, and fused images publicly available.

Low-light image enhancement based on simulated multi-exposure fusion

We propose an efficient and novel framework for low-light image enhancement, which aims to reveal information hidden in the darkness and improve overall brightness and local contrast. Inspired by exposure fusion technique, we employ simulated multi-exposure images fusion to derive bright, natural and satisfactory results, while images are taken under poor conditions such as insufficient or uneven illumination, back-lit and limited exposure time. Specifically, we first design a novel method to generate synthesized images with varying exposure time from a single image. Thus, each image of these artificial sequences contains necessary information for the final desired enhanced result. We then introduce a flexible multi-exposure fusion framework to achieve fused images, which comprises a weight map prediction module and a multi-scale fusion module. Extensive experiments show that our approach can achieve similar or better performance compared to serval state-of-the-art methods.

A modified multi-exposure fusion method for laser measurement of specular surfaces

Line laser is widely used in structural light three-dimensional (3D) measurement of metal parts; however, the laser stripes are severely disturbed when measuring complex surfaces. In the process of laser plaSignal sent for page signal is not returning for this article, hence not possible to proceed. Please do the needful, also return the task only after solving the issue. ne calibration, the laser stripe will be damaged by the pattern on the calibration board; when measuring metal surfaces, the laser stripe will be exposed unevenly. Overexposed parts show highlights and artifacts, while underexposed stripes appear burrs, high noise, or even undetectable due to material absorption, which will greatly affect the accuracy of stripe center extraction. In this paper, a modified multi-exposure fusion (MMEF) method is proposed to improve the dynamic range of laser stripe images by enhancing underexposed regions and reducing saturation. The MMEF method uses a “smoothness” index and a “validity” index to give more weight to smooth, Gaussian-like laser stripe patterns and extract reliable modulation information. To evaluate the effectiveness of our method, the performance of the MMEF method is tested for laser plane calibration and metal surface reconstruction, and the high dynamic range (HDR) method, retrieval fusion (RF) method, and exposure fusion (EF) method are introduced for comparison. Measurement experimental results show that the MMEF method proposed in this paper improves the laser plane calibration accuracy by 86.0% and the 3D measurement accuracy of metal surface by 12.5%.

Image Dehazing Based on Improved Color Channel Transfer and Multiexposure Fusion

Image dehazing is one of the problems that need to be solved urgently in the field of computer vision. In recent years, more and more algorithms have been applied to image dehazing and achieved good results. However, the image after dehazing still has color distortion, contrast and saturation disorder, and other challenges; in order to solve these problems, in this paper, an effective image dehazing method is proposed, which is based on improved color channel transfer and multiexposure image fusion to achieve image dehazing. First, the image is preprocessed using a color channel transfer method based on k-means. Second, gamma correction is introduced on the basis of guided filtering to obtain a series of multiexposure images, and the obtained multiexposure images are fused into a dehazed image through a Laplacian pyramid fusion scheme based on local similarity of adaptive weights. Finally, contrast and saturation corrections are performed on the dehazed image. Experimental verification is carried out on synthetic dehazed images and natural dehazed images, and it is verified that the method proposed is superior to existing dehazed algorithms from both subjective and objective aspects.

Multi-Task Learning Approach Using Dynamic Hyperparameter for Multi-Exposure Fusion

High-dynamic-range (HDR) image synthesis is a technology developed to accurately reproduce the actual scene of an image on a display by extending the dynamic range of an image. Multi-exposure fusion (MEF) technology, which synthesizes multiple low-dynamic-range (LDR) images to create an HDR image, has been developed in various ways including pixel-based, patch-based, and deep learning-based methods. Recently, methods to improve the synthesis quality of images using deep-learning-based algorithms have mainly been studied in the field of MEF. Despite the various advantages of deep learning, deep-learning-based methods have a problem in that numerous multi-exposed and ground-truth images are required for training. In this study, we propose a self-supervised learning method that generates and learns reference images based on input images during the training process. In addition, we propose a method to train a deep learning model for an MEF with multiple tasks using dynamic hyperparameters on the loss functions. It enables effective network optimization across multiple tasks and high-quality image synthesis while preserving a simple network architecture. Our learning method applied to the deep learning model shows superior synthesis results compared to other existing deep-learning-based image synthesis algorithms.

An effective and robust underwater image enhancement method based on color correction and artificial multi-exposure fusion

An effective and robust underwater image enhancement method based on color correction and artificial multi-exposure fusion

Tuning-free and self-supervised image enhancement against ill exposure.

Complex lighting conditions and the limited dynamic range of imaging devices result in captured images with ill exposure and information loss. Existing image enhancement methods based on histogram equalization, Retinex-inspired decomposition model, and deep learning suffer from manual tuning or poor generalization. In this work, we report an image enhancement method against ill exposure with self-supervised learning, enabling tuning-free correction. First, a dual illumination estimation network is constructed to estimate the illumination for under- and over-exposed areas. Thus, we get the corresponding intermediate corrected images. Second, given the intermediate corrected images with different best-exposed areas, Mertens' multi-exposure fusion strategy is utilized to fuse the intermediate corrected images to acquire a well-exposed image. The correction-fusion manner allows adaptive dealing with various types of ill-exposed images. Finally, the self-supervised learning strategy is studied which learns global histogram adjustment for better generalization. Compared to training on paired datasets, we only need ill-exposed images. This is crucial in cases where paired data is inaccessible or less than perfect. Experiments show that our method can reveal more details with better visual perception than other state-of-the-art methods. Furthermore, the weighted average scores of image naturalness matrics NIQE and BRISQUE, and contrast matrics CEIQ and NSS on five real-world image datasets are boosted by 7%, 15%, 4%, and 2%, respectively, over the recent exposure correction method.

IID-MEF: A multi-exposure fusion network based on intrinsic image decomposition

This paper follows the idea of divide and conquer to propose a multi-exposure fusion network for the unsupervised generation of high dynamic range-like images. We develop a new intrinsic image decomposition (IID) network based on our modified IID model to produce the reflectance, shading, and color components from source images, which respectively represent the texture structure, lighting condition, and visible color distribution of the imaging scene. Three fusion sub-networks are then designed to process these different components, producing pleasing images with rich structures, reasonable lighting, and suitable color. Specifically, we first develop a reflectance fusion network to integrate the reflectance components, in which an adaptive pixel-scale selection strategy is adopted to dynamically guide the network to preserve rich scene textures. Then, a shading fusion network with the adaptive global image-scale weighting strategy is designed to fulfill lighting adjustment, which can generate suitable illumination by defining the lighting adjustment as a game between the illumination of source images. Finally, we propose a color fusion network that embeds both pixel-scale and image-scale strategies, which can guarantee a pleasing scene color distribution with a specifically designed color fidelity loss. Extensive experiments demonstrate the superiority of our method over state-of-the-art methods in terms of texture integration, illumination adjustment, and color fidelity. Moreover, our method can be applied to fulfill tasks of the single low-light image enhancement and the single overexposed image correction with promising performance. The code is publicly available at https://github.com/HaoZhang1018/IID-MEF.

Multi-exposure electric power monitoring image fusion method without ghosting based on exposure fusion framework and color dissimilarity feature.

To solve the ghosting artifacts problem in dynamic scene multi-scale exposure fusion, an improved multi-exposure fusion method has been proposed without ghosting based on the exposure fusion framework and the color dissimilarity feature of this study. This fusion method can be further applied to power system monitoring and unmanned aerial vehicle monitoring. In this study, first, an improved exposure fusion framework based on the camera response model was applied to preprocess the input image sequence. Second, the initial weight map was estimated by multiplying four weight items. In removing the ghosting weight term, an improved color dissimilarity feature was used to detect the object motion features in dynamic scenes. Finally, the improved pyramid model as adopted to retain detailed information about the poor exposure areas. Experimental results indicated that the proposed method improves the performance of images in terms of sharpness, detail processing, and ghosting artifacts removal and is superior to the five existing multi-exposure image fusion (MEF) methods in quality evaluation.

Deep Multi-Exposure Image Fusion for Dynamic Scenes.

Recently, learning-based multi-exposure fusion (MEF) methods have made significant improvements. However, these methods mainly focus on static scenes and are prone to generate ghosting artifacts when tackling a more common scenario, i.e., the input images include motion, due to the lack of a benchmark dataset and solution for dynamic scenes. In this paper, we fill this gap by creating an MEF dataset of dynamic scenes, which contains multi-exposure image sequences and their corresponding high-quality reference images. To construct such a dataset, we propose a 'static-for-dynamic' strategy to obtain multi-exposure sequences with motions and their corresponding reference images. To the best of our knowledge, this is the first MEF dataset of dynamic scenes. Correspondingly, we propose a deep dynamic MEF (DDMEF) framework to reconstruct a ghost-free high-quality image from only two differently exposed images of a dynamic scene. DDMEF is achieved through two steps: pre-enhancement-based alignment and privilege-information-guided fusion. The former pre-enhances the input images before alignment, which helps to address the misalignments caused by the significant exposure difference. The latter introduces a privilege distillation scheme with an information attention transfer loss, which effectively improves the deghosting ability of the fusion network. Extensive qualitative and quantitative experimental results show that the proposed method outperforms state-of-the-art dynamic MEF methods. The source code and dataset are released at https://github.com/Tx000/Deep_dynamicMEF.

Cancellable Multi-Biometric Feature Veins Template Generation Based on SHA-3 Hashing

In this paper, a novel cancellable biometrics technique called Multi-Biometric-Feature-Hashing (MBFH) is proposed. The MBFH strategy is utilized to actualize a single direction (non-invertibility) biometric shape. MBFH is a typical model security conspire that is distinguished in the utilization of this protection insurance framework in numerous sorts of biometric feature strategies (retina, palm print, Hand Dorsum, fingerprint). A more robust and accurate multilingual biological structure in expressing human loneliness requires a different format to record clients with inseparable comparisons from individual biographical sources. This may raise worries about their utilization and security when these spread out designs are subverted as everybody is acknowledged for another biometric attribute.The proposed structure comprises of four sections: input multi-biometric acquisition, feature extraction, Multi-Exposure Fusion (MEF) and secure hashing calculation (SHA-3). Multimodal biometrics systems that are more powerful and precise in human-unmistakable evidence require various configurations to store a comparative customer that can be contrasted with biometric wellsprings of people. Disparate top words, biometrics graphs can’t be denied and change to another request for positive Identifications (IDs) while settling. Cancellable biometrics is may be the special procedure used to recognize this issue.

Multi-Exposure Fusion With Guidance Information: Night Color Image Enhancement for Roadside Units

Multi-Exposure Fusion With Guidance Information: Night Color Image Enhancement for Roadside Units

GALFusion: Multi-Exposure Image Fusion via a Global–Local Aggregation Learning Network

The goal of multi-exposure image fusion is to generate synthetic results with abundant details and balanced exposure from low dynamic range(LDR) images. The existing multi-exposure fusion methods often use convolution operations to extract features. However, these methods only consider the pixel values in local view field and ignore the long-range dependencies between pixels. To solve the aforementioned problem, we propose a global-local aggregation network for fusing extreme exposure images in an unsupervised way. Firstly, we design a collaborative aggregation module, composed of two sub-modules covering a non-local attention inference module and a local adaptive learning module, to mine the relevant features from source images. So that we successfully formulate a feature extraction mechanism with aggregating global and local information. Secondly, we provide a special fusion module to reconstruct fused images, which effectively avoids artifacts and suppresses information decay. Moreover, we further fine-tune the fusion results by a recursive refinement module to capture more textural details from source images. The results of both comparative and ablation analyses on two datasets demonstrate that GALFusion achieves the best marks in terms of MEF-SSIM and PSNR, outperforming the existing 12 state-of-the-art fusion methods.

Single low-light image brightening using learning-based intensity mapping

Inspired by image-to-curve transformation and multi-exposure fusion, in this paper, we have developed a new method to treat the low light image enhancement tasks as an extended problem with multiple virtual exposures by a non-linear intensity mapping function. Considering that existing image-to-curve methods have difficulty in obtaining the desired detail and brightness recovery in any one iteration without relying on any ground truth, we propose a virtual multi-exposure fusion strategy to merge the outputs from these different iterations. Specifically, a simple CNN is trained to learn a pixel-wise intensity mapping function and accordingly adjust a given image multiple times. Then the results of all iterations are retained together with the original input image for fusion via a WGIF-based Multi-scale pyramid to obtain a final enhanced output. We present experimental results to demonstrate the effectiveness of the new technique and its state-of-the-art performances.

Enhancement and Noise Suppression of Single Low-Light Grayscale Images

Low-light images have low contrast and high noise, making them not easily readable. Most existing image-enhancement methods focus on color images. In the present study, an enhancement and denoising algorithm for single low-light grayscale images is proposed. The algorithm is based on the multi-exposure fusion framework. First, on the basis of the low-light tone-mapping operators, the optimal virtual exposure image is constructed according to the information entropy criterion. Then, the latent low-rank representation is applied to two images to generate low-ranking parts and saliency parts to reduce noise after fusion. Next, the initial weight map is constructed based on the information contained in the decomposed images, and an adaptive weight refined algorithm is proposed to restore as much structural information as possible and keep the details while avoiding halo artifacts. When solving the weight maps, the decomposition and optimization of the nonlinear problem is converted into a total variation model, and an iterative method is used to reduce the computational complexity. Last, the normalized weight map is used for image fusion to obtain the enhanced image. The experimental results showed that the proposed method performed well both in the subjective and objective evaluation of state-of-the-art enhancement methods for low-light grayscale images.

Retracted: Contrast and Synthetic Multiexposure Fusion for Image Enhancement.

[This retracts the article DOI: 10.1155/2021/2030142.].

Multi-Exposure Image Fusion Based on Weighted Average Adaptive Factor and Local Detail Enhancement

In order to adapt to the local brightness and contrast of input image sequences, we propose a new weighted average adaptive factor well-exposure weight evaluation scheme. The exposure weights of brighter and darker pixels are determined according to the local average brightness and expected brightness. We find that in the traditional multi-exposure image fusion scheme, the brighter and darker regions of the scene lose many details. To solve this problem, we first propose a standard to determine the brighter and darker regions and then use a fast local Laplacian filter to enhance the image in the region. This paper selects 16 multi-exposure images of different scenes for subjective and objective analysis and compares them with eight existing multi-exposure fusion schemes. The experimental results show that the proposed method can enhance the details appropriately while preserving the details in static scenes and adapting to the input image brightness.

A novel multi-exposure fusion approach for enhancing visual semantic segmentation of autonomous driving

Visual semantic segmentation is a key technology to realize scene understanding for autonomous driving and its accuracy is affected by the light changes in images. This paper proposes a novel multi-exposure fusion approach to visual semantic enhancement of autonomous driving. Firstly, a multi-exposure image sequence is aligned to construct a stable image input. Secondly, high contrast regions of multi-exposure image sequences are evaluated by context aggregation network (CAN) to predict image weight map. Finally, the high-quality image is generated by weighted fusion of multi-exposure image sequences. The proposed approach is validated by using Cityscapes’ HDR dataset and real environment data. The experimental results show that the proposed method effectively restores lost features in the light changing images and enhances accuracy of subsequent semantic segmentation.

Multi-exposure image fusion for dynamic scenes with ghosting removal

Multi-exposure image fusion algorithms in dynamic scenes usually have ghosts. Most of the existing ghost-free methods have complex calculations and the details will be lost. To solve these problems, we present a multi-exposure fusion method, which can remove ghosts and preserve more details of the image. The method detects ghost images by the improved frame difference method and calculates the early weight maps, combining the ghost detection binary image and initial weight maps to redefine weight maps. Then with the improved Laplacian pyramid algorithm, a high-quality image with more details is fused quickly. This is a significant improvement over existing techniques by objective and subjective comparisons.

Smoke removal and image enhancement of laparoscopic images by an artificial multi-exposure image fusion method

In laparoscopic surgery, image quality is often degraded by surgical smoke or by side effects of the illumination system, such as reflections, specularities, and non-uniform illumination. The degraded images complicate the work of the surgeons and may lead to errors in image-guided surgery. Existing enhancement algorithms mainly focus on enhancing global image contrast, overlooking local contrast. Here, we propose a new Patch Adaptive Structure Decomposition utilizing the Multi-Exposure Fusion technique to enhance the local contrast of laparoscopic images for better visualization. The set of under-exposure level images is obtained from a single input blurred image by using gamma correction. Spatial linear saturation is applied to enhance image contrast and to adjust the image saturation. The Multi-Exposure Fusion (MEF) is used on a series of multi-exposure images to obtain a single clear and smoke-free fused image. MEF is applied by using adaptive structure decomposition on all image patches. Image entropy based on the texture energy is used to calculate image energy strength. The texture entropy energy determined the patch size that is useful in the decomposition of image structure. The proposed method effectively eliminate smoke and enhance the degraded laparoscopic images. The qualitative results showed that the visual quality of the resultant images is improved and smoke-free. Furthermore, the quantitative scores computed of the metrics: FADE, Blur, JNBM, and Edge Intensity are significantly improved as compared to other existing methods.