Multi-exposure Image Fusion Algorithm Research Articles

An Improved Multiexposure Image Fusion Technique.

Multiexposure image fusion (MEF) is an effective approach to generate high dynamic range images from multilevel exposures taken from ordinary cameras. In this article, a novel MEF algorithm is proposed to gain maximum visual details as well as vivid colors from the captured scene. This algorithm first decomposes the input images with multiple exposures into the base and detail layer. The weights for the base and detail layers are computed by using exposedness function and then both the layers are combined to generate the final fused image. The proposed multiexposure technique requires fewer computational operations, preserves edges, and also reduces spatial artifacts. The proposed technique has been evaluated quantitatively using image quality assessment model based on structure similarity index measure for MEF. By the extensive experimental results, it has been illustrated that in addition to significantly outperforming other state-of-the-art techniques, the proposed technique is much faster and can achieve better image quality.

Rethinking multi-exposure image fusion with extreme and diverse exposure levels: A robust framework based on Fourier transform and contrastive learning

Multi-exposure image fusion (MEF) is an important technique for generating high dynamic range images. However, most existing MEF studies focus on fusing a moderately over-exposed image and a moderately under-exposed image, and they are not robust in fusing images with extreme and diverse exposure levels. In this paper, we propose a robust MEF framework based on Fourier transform and contrastive learning. Specifically, we develop a Fourier transform-based pixel intensity transfer strategy to synthesize images with diverse exposure levels from normally exposed natural images and train an encoder–decoder network to reconstruct the original natural image. In this way, the encoder and decoder can learn to extract features from images with diverse exposure levels and generate fused images with normal exposure. We propose a contrastive regularization loss to further enhance the capability of the network in recovering normal exposure levels. In addition, we construct an extreme MEF benchmark dataset and a random MEF benchmark dataset for a more comprehensive evaluation of MEF algorithms. We extensively compare our method with fifteen competitive traditional and deep learning-based MEF algorithms on three benchmark datasets, and our method outperforms the other methods in both subjective visual effects and objective evaluation metrics. Our code, datasets and all fused images will be released.

Multi-exposure image fusion for dynamic scenes with ghosting removal

Multi-exposure image fusion algorithms in dynamic scenes usually have ghosts. Most of the existing ghost-free methods have complex calculations and the details will be lost. To solve these problems, we present a multi-exposure fusion method, which can remove ghosts and preserve more details of the image. The method detects ghost images by the improved frame difference method and calculates the early weight maps, combining the ghost detection binary image and initial weight maps to redefine weight maps. Then with the improved Laplacian pyramid algorithm, a high-quality image with more details is fused quickly. This is a significant improvement over existing techniques by objective and subjective comparisons.

Multi-Exposure Image Fusion Algorithm Based on Improved Weight Function.

High-dynamic-range (HDR) image has a wide range of applications, but its access is limited. Multi-exposure image fusion techniques have been widely concerned because they can obtain images similar to HDR images. In order to solve the detail loss of multi-exposure image fusion (MEF) in image reconstruction process, exposure moderate evaluation and relative brightness are used as joint weight functions. On the basis of the existing Laplacian pyramid fusion algorithm, the improved weight function can capture the more accurate image details, thereby making the fused image more detailed. In 20 sets of multi-exposure image sequences, six multi-exposure image fusion methods are compared in both subjective and objective aspects. Both qualitative and quantitative performance analysis of experimental results confirm that the proposed multi-scale decomposition image fusion method can produce high-quality HDR images.

Multi-Exposure Image Fusion based on Window Segmentation and a Laplacian Pyramid for Chip Package Appearance Quality Detection

A heterogeneous material image enhancement method based on multi-exposure image fusion is proposed to address the problem of obtaining high-quality images from the single imaging of chips containing two extremely different reflectivity materials. First, a multi-exposure image fusion algorithm based on window segmentation and Laplacian pyramid fusion is proposed. Then, orthogonal experiments are used to optimize the parameters of the imaging system. Next, a method based on information entropy and average gray intensity is utilized to calculate the imaging exposure times of two heterogeneous materials, and two exposure time ranges are obtained that are appropriate for regions with high and low reflectivity. Finally, the subjective and objective experimental evaluations are conducted after the multi-exposure image set has been established. The results show that the fused image has a good visual effect, the information entropy is 6.29, and the average gray intensity is 131.56. In addition, time consumption is reduced by an average of 20.3% compared to the Laplace pyramid strategy. The heterogeneous material enhancement method based on multi-exposure image fusion proposed in this paper is effective and deserving of further research and application.

An Efficient Extreme-Exposure Image Fusion Method

Since the existing commercial imaging equipment cannot meet the requirements of high dynamic range, multi-exposure image fusion is an economical and fast method to implement HDR. However, the existing multi-exposure image fusion algorithms have the problems of long fusion time and large data storage. We propose an extreme exposure image fusion method based on deep learning. In this method, two extreme exposure image sequences are sent to the network, channel and spatial attention mechanisms are introduced to automatically learn and optimize the weights, and the optimal fusion weights are output. In addition, the model in this paper adopts real-value training and makes the output closer to the real value through a new custom loss function. Experimental results show that this method is superior to existing methods in both objective and subjective aspects.

Benchmarking and comparing multi-exposure image fusion algorithms

Multi-exposure image fusion (MEF) is an important area in computer vision and has attracted increasing interests in recent years. Apart from conventional algorithms, deep learning techniques have also been applied to MEF. However, although many efforts have been made on developing MEF algorithms, the lack of benchmarking studies makes it difficult to perform fair and comprehensive performance comparison among MEF algorithms, thus hindering the development of this field significantly. In this paper, we fill this gap by proposing a benchmark of multi-exposure image fusion (MEFB), which consists of a test set of 100 image pairs, a code library of 21 algorithms, 20 evaluation metrics, 2100 fused images, and a software toolkit. To the best of our knowledge, this is the first benchmarking study in the field of MEF. This paper also gives a literature review on MEF methods with a focus on deep learning-based algorithms. Extensive experiments have been conducted using MEFB for comprehensive performance evaluation and for identifying effective algorithms. We expect that MEFB will serve as an effective platform for researchers to compare the performance of MEF algorithms.

An Improved Multi-Exposure Image Fusion Method for Intelligent Transportation System

In this paper, an improved multi-exposure image fusion method for intelligent transportation systems (ITS) is proposed. Further, a new multi-exposure image dataset for traffic signs, TrafficSign, is presented to verify the method. In the intelligent transportation system, as a type of important road information, traffic signs are fused by this method to obtain a fused image with moderate brightness and intact information. By estimating the degree of retention of different features in the source image, the fusion results have adaptive characteristics similar to that of the source image. Considering the weather factor and environmental noise, the source image is preprocessed by bilateral filtering and dehazing algorithm. Further, this paper uses adaptive optimization to improve the quality of the output image of the fusion model. The qualitative and quantitative experiments on the new dataset show that the multi-exposure image fusion algorithm proposed in this paper is effective and practical in the ITS.

Superpixel-Based Quality Assessment of Multi-Exposure Image Fusion for Both Static and Dynamic Scenes.

Multi-exposure image fusion (MEF) algorithms have been used to merge a stack of low dynamic range images with various exposure levels into a well-perceived image. However, little work has been dedicated to predicting the visual quality of fused images. In this work, we propose a novel and efficient objective image quality assessment (IQA) model for MEF images of both static and dynamic scenes based on superpixels and an information theory adaptive pooling strategy. First, with the help of superpixels, we divide fused images into large- and small-changed regions using the structural inconsistency map between each exposure and fused images. Then, we compute the quality maps based on the Laplacian pyramid for large- and small-changed regions separately. Finally, an information theory induced adaptive pooling strategy is proposed to compute the perceptual quality of the fused image. Experimental results on three public databases of MEF images demonstrate the proposed model achieves promising performance and yields a relatively low computational complexity. Additionally, we also demonstrate the potential application for parameter tuning of MEF algorithms.

An improved algorithm of multi-exposure image fusion by detail enhancement

Multi-exposure image fusion is an effective method for depicting high dynamic range of the target scene in a single image. However, there are still some problems remaining: the preserving of global contrast, the preserving of the local details in saturated regions, and the existence of halo artifacts. To solve these problems, this paper proposes a new multi-exposure image fusion algorithm with detail enhancement. Firstly, the well-exposedness evaluation function, the chromatic information evaluation function and the local detail preserved function are used to measure weight maps. Then, an improved multi-exposure fusion framework based on pyramid decomposition is proposed to further enhance the details. The experimental results demonstrate that the proposed algorithm can preserve more details than the state-of-the-art. In the view of appreciation, our approach could produce a more realistic brightness distribution of target scene as well as avoid halo artifacts.

Color-compensated multi-scale exposure fusion based on physical features

The dynamic range captured by ordinary digital cameras is much smaller than that in natural scenes, so a single image cannot fully capture the real scene details when shooting high dynamic scenes. Traditional multi-exposure image fusion algorithms often cause inconsistencies in local brightness and corresponding scenes, which easily leads to problems such as color distortion and loss of details. Based on this, a novel multi-scale exposure fusion approach based on physical features is presented in this paper. Specifically, we first employ a novel Retinex model to get the illumination maps of original images and adopt the obtained illuminations map to construct the exposure maps. Then, a feature descriptor named patterns of oriented edge magnitudes (POEM) is introduced to extract local contrast from source images. Moreover, the extracted feature is combined to construct a weight map, and the weight term is integrated with the Laplacian pyramid algorithm to obtain an initial fusion image. Finally, simple detail enhancement and color compensation operations are performed to get the final fusion result. The numerous experiments have revealed that the proposed approach yields comparative and even better results in comparison with some state-of-the-art techniques both in subjective and objective evaluation.

Image Dehazing by an Artificial Image Fusion Method Based on Adaptive Structure Decomposition

Haze can seriously affect the visible and visual quality of outdoor images. As a challenge in practice, image dehazing techniques are always used to remove haze from the captured images. Existing image dehazing algorithms focus on enhancing both global image contrast and saturation, but ignore the local enhancement. So the dehazed images do not often have good performance in the visual quality of local details. This paper proposes a new single-image dehazing solution based on the adaptive structure decomposition integrated multi-exposure image fusion (PADMEF). A set of underexposed image sequences are extracted from a single blurred image first by a series of gamma correction and the spatial linear adjustment of saturation. Then different exposure-level images are fused into a haze-free image by applying a multi-exposure image fusion (MEF) scheme based adaptive structure decomposition to each image patch. The proposed image dehazing scheme can effectively eliminate the visual degradation caused by haze without the physical model inversion of haze formation. Both apriori estimation of scene depth and the expensive refinement process of depth mapping can be avoided. The entropy of image texture named as texture energy is used to measure the image energy and obtain the information size contained in an image. Meanwhile, a texture energy based method is presented to adaptively select the corresponding patch size for the decomposition of image structure. In addition, this paper verifies that the dehazed images obtained by the patch based MEF algorithm always meet the requirements of intensity decrease. The comparative experiment results are evaluated in both qualitative and quantitative aspects, which confirm the effectiveness of the proposed solution in haze removal.

A Fast Image Dehazing Algorithm for Highway Tunnel Based on Artificial Multi-exposure Image Fusion

Single image dehazing has been a challenging problem due to its ill-posed nature. This paper proposed a fast image dehazing algorithm for highway tunnel based on artificial multi-exposure image fusion. The original blurred images are transformed into under-exposed images using a series of gamma-correction operations. Meanwhile, a multi-exposure image fusion algorithm based on two-scale decomposition is proposed. It exploits the guided filter to separate each multi-exposure image into base and detail layers. The fused base and detail layers are integrated into the fused images. Then, a linear saturation adjustment method is utilized to enhance the saturation in spatial domain. Through the above image processing steps, the final haze-free images can be obtained. In this paper, the superiority of the proposed image dehazing algorithm based on artificial multi-exposure image fusion is demonstrated through theoretical analysis and experimental verification. It can quickly improve the visibility of foggy images, and make the processed images colorful and clear.

Perceptual Evaluation for Multi-Exposure Image Fusion of Dynamic Scenes.

A common approach to high dynamic range (HDR) imaging is to capture multiple images of different exposures followed by multi-exposure image fusion (MEF) in either radiance or intensity domain. A predominant problem of this approach is the introduction of the ghosting artifacts in dynamic scenes with camera and object motion. While many MEF methods (often referred to as deghosting algorithms) have been proposed for reduced ghosting artifacts and improved visual quality, little work has been dedicated to perceptual evaluation of their deghosting results. Here we first construct a database that contains 20 multiexposure sequences of dynamic scenes and their corresponding fused images by nine MEF algorithms. We then carry out a subjective experiment to evaluate fused image quality, and find that none of existing objective quality models for MEF provides accurate quality predictions. Motivated by this, we develop an objective quality model for MEF of dynamic scenes. Specifically, we divide the test image into static and dynamic regions, measure structural similarity between the image and the corresponding sequence in the two regions separately, and combine quality measurements of the two regions into an overall quality score. Experimental results show that the proposed method significantly outperforms the state-of-the-art. In addition, we demonstrate the promise of the proposed model in parameter tuning of MEF methods.1.

Algorithm of multiexposure image fusion with detail enhancement and ghosting removal

Multiexposure image fusion is an effective method for depicting a high dynamic range display of target scenes. However, the loss of local details and ghosting artifacts in fusion images has not yet been well resolved in the existing research. To solve these problems, a multiexposure image fusion algorithm with detail enhancement and ghosting removal is proposed. First, the exposure brightness and chromaticity information in YIQ color space are used to measure weight maps. When the captured scene is dynamic with moving objects, the weight maps are refined to remove ghosting artifacts based on the image difference and superpixel segmentation. Finally, an improved Laplacian pyramid fusion framework is proposed to achieve image fusion with detail enhancement. The experimental results demonstrate the effectiveness of the proposed method in terms of both visual quality and objective evaluation and show that the proposed algorithm can preserve more details and can remove ghosting artifacts effectively.

Multi exposure image fusion algorithm based on YCbCr space

To solve the problem that scene details and visual effects are difficult to be optimized in high dynamic image synthesis, we proposes a multi exposure image fusion algorithm for processing low dynamic range images in YCbCr space, and weighted blending of luminance and chromatic aberration components respectively. The experimental results show that the method can retain color effect of the fused image while balancing details of the bright and dark regions of the high dynamic image.

Multi-Exposure Image Fusion through Structural Patch Decomposition with Gaussian Filter

It propose a simple yet effective Structural Patch Decomposition with Gaussian Filter (SPDGF) approach for multi-exposure image fusion (MEF) that is robust to ghosting effect. It decomposes an image patch into three conceptually independent components: signal strength, signal structure, and mean intensity. Upon fusing these three components separately then reconstruct a desired patch and place it back into the fused image. This novel approach benefits MEF in many aspects. First, as opposed to most pixel-wise MEF methods, the proposed algorithm does not require post-processing steps to improve visual quality or to reduce spatial artifacts. Second, it handles RGB color channels jointly and thus produces fused images with more vivid color appearance. Third and most importantly, the direction of the signal structure component in the patch vector space provides ideal information for ghost removal. It allows us to reliably and efficiently reject inconsistent object motions then a chosen reference image without performing computationally expensive motion estimation. Now compare the proposed algorithm with SPD-MEF methods on different images (with camera and object motion). Extensive experimental results demonstrate that the proposed algorithm not only outperforms previous MEF algorithms on static scenes but also consistently produces high quality fused images with little ghosting artifacts for dynamic scenes. Moreover, it maintains a lower computational cost compared with state-of-the-art MEF de-ghosting schemes.

Multi-Exposure Image Fusion by Optimizing A Structural Similarity Index

We propose a multi-exposure image fusion (MEF) algorithm by optimizing a novel objective quality measure, namely the color MEF structural similarity (MEF-SSIM $_c$ ) index. The design philosophy we introduce here is substantially different from existing ones. Instead of pre-defining a systematic computational structure for MEF ( e.g. , multiresolution transformation and transform domain fusion followed by image reconstruction), we directly operate in the space of all images, searching for the image that optimizes MEF-SSIM $_c$ . Specifically, we first construct the MEF-SSIM $_c$ index by improving upon and expanding the application scope of the existing MEF-SSIM algorithm. We then describe a gradient ascent-based algorithm, which starts from any initial point in the space of all possible images and iteratively moves towards the direction that improves MEF-SSIM $_c$ until convergence. Numerical and subjective experiments demonstrate that the proposed algorithm consistently produces better quality fused images both visually and in terms of MEF-SSIM $_c$ . The final high quality fused image appears to have little dependence on the initial image. The proposed optimization framework is readily extensible to construct better MEF algorithms when better objective quality models for MEF are available.

A Multi-exposure Image Fusion Method with Detail Preservation

In view of the problems of uneven exposure in the image acquisition and the serious loss of details in the traditional multi-exposure image fusion algorithm, a method of image fusion with details preservation is proposed. A weighted approach to multi-exposure image fusion is used, taking into account the features such as local contrast, exposure brightness, and color information to better preserve detail. For the purpose of eliminating the noise and interference, using the recursive filter to filter. Compared with other algorithms, the proposed algorithm can retain the rich detail information to meet the quality requirements of spot welding image fusion and has certain application value.

Robust Multi-Exposure Image Fusion: A Structural Patch Decomposition Approach

We propose a simple yet effective structural patch decomposition approach for multi-exposure image fusion (MEF) that is robust to ghosting effect. We decompose an image patch into three conceptually independent components: signal strength, signal structure, and mean intensity. Upon fusing these three components separately, we reconstruct a desired patch and place it back into the fused image. This novel patch decomposition approach benefits MEF in many aspects. First, as opposed to most pixel-wise MEF methods, the proposed algorithm does not require post-processing steps to improve visual quality or to reduce spatial artifacts. Second, it handles RGB color channels jointly, and thus produces fused images with more vivid color appearance. Third and most importantly, the direction of the signal structure component in the patch vector space provides ideal information for ghost removal. It allows us to reliably and efficiently reject inconsistent object motions with respect to a chosen reference image without performing computationally expensive motion estimation. We compare the proposed algorithm with 12 MEF methods on 21 static scenes and 12 deghosting schemes on 19 dynamic scenes (with camera and object motion). Extensive experimental results demonstrate that the proposed algorithm not only outperforms previous MEF algorithms on static scenes but also consistently produces high quality fused images with little ghosting artifacts for dynamic scenes. Moreover, it maintains a lower computational cost compared with the state-of-the-art deghosting schemes.