Multi-focus Image Fusion Method Research Articles

Fusion of Multi-Focus Images using Jellyfish Search Optimizer

When obtaining an image of a scene, the lens focuses on objects at a certain distance, and objects at other distances are blurred. This is called the limited depth of field problem. An approach for solving this problem is multi-focus image fusion. A clearer view of the entire scene is obtained by using the multi-focus image fusion method. For this method, at least two images captured at different focuses are combined. Various algorithms have been developed for multi-focus image fusion methods. For multi-focus image fusion, pixel-level block-based methods are commonly used. The block size is a factor that significantly affects the fusion performance. As a result, the block size parameter must be improved. The Jellyfish search optimization algorithm (JSA) is used to propose a block-based multi-focus image fusion approach based on the optimal selection of clearer image blocks from source images. The results of DWTPCA, DCHWT, APCA, PCA, SWTDWT and SWT methods, which are traditional image fusion methods, and ABC (artificial bee colony) and JSA optimization algorithms, which are metaheuristic methods, are compared. In addition, it has been determined that the JSA method has better performance than other traditional methods when compared both visually and quantitatively.

Multi-focus image fusion with deep residual learning and focus property detection

Multi-focus image fusion methods can be mainly divided into two categories: transform domain methods and spatial domain methods. Recent emerged deep learning (DL)-based methods actually satisfy this taxonomy as well. In this paper, we propose a novel DL-based multi-focus image fusion method that can combine the complementary advantages of transform domain methods and spatial domain methods. Specifically, a residual architecture that includes a multi-scale feature extraction module and a dual-attention module is designed as the basic unit of a deep convolutional network, which is firstly used to obtain an initial fused image from the source images. Then, the trained network is further employed to extract features from the initial fused image and the source images for a similarity comparison, aiming to detect the focus property of each source pixel. The final fused image is obtained by selecting corresponding pixels from the source images and the initial fused image according to the focus property map. Experimental results show that the proposed method can effectively preserve the original focus information from the source images and prevent visual artifacts around the boundary regions, leading to more competitive qualitative and quantitative performance when compared with the state-of-the-art fusion methods.

A Survey of Multi-Focus Image Fusion Methods

As an important branch in the field of image fusion, the multi-focus image fusion technique can effectively solve the problem of optical lens depth of field, making two or more partially focused images fuse into a fully focused image. In this paper, the methods based on boundary segmentation was put forward as a group of image fusion method. Thus, a novel classification method of image fusion algorithms is proposed: transform domain methods, boundary segmentation methods, deep learning methods, and combination fusion methods. In addition, the subjective and objective evaluation standards are listed, and eight common objective evaluation indicators are described in detail. On the basis of lots of literature, this paper compares and summarizes various representative methods. At the end of this paper, some main limitations in current research are discussed, and the future development of multi-focus image fusion is prospected.

A new multi-focus image fusion method based on multi-classification focus learning and multi-scale decomposition

A new multi-focus image fusion method based on multi-classification focus learning and multi-scale decomposition

Multi-Focus Image Fusion Based on Multi-Scale Generative Adversarial Network.

The methods based on the convolutional neural network have demonstrated its powerful information integration ability in image fusion. However, most of the existing methods based on neural networks are only applied to a part of the fusion process. In this paper, an end-to-end multi-focus image fusion method based on a multi-scale generative adversarial network (MsGAN) is proposed that makes full use of image features by a combination of multi-scale decomposition with a convolutional neural network. Extensive qualitative and quantitative experiments on the synthetic and Lytro datasets demonstrated the effectiveness and superiority of the proposed MsGAN compared to the state-of-the-art multi-focus image fusion methods.

Multi-focus image fusion based on quad-tree decomposition and edge-weighted focus measure

Multi-focus image fusion is an important way to obtain all-in-focus images. The goal is to reconstruct all the focused pixels in the source image into the fused image. Generally, in block-based fusion methods, the source image is often decomposed into fixed-size blocks. However, the size of the blocks will affect the fusion quality, and the problems such as blockiness is prone to occur in the fusion results. To this end, we propose a novel multi-focus image fusion method based on optimal block decomposition. Different from the conventional fixed-block-based methods, our method adopts the optimal decomposition of the quad-tree to process the source images. First, a new sum of edge-weighted modified Laplacian (SEWML) is proposed based on the sum of modified Laplacian (SML), which is used as a focus measure to detect the focus information of the source image, and this improved focus measure is more robust than SML. Then, an efficient quad-tree decomposition strategy is proposed to decompose the source images into optimally sized blocks. At the same time, using SEWML to detect the focused blocks in the quad-tree structure of the source images, the focused blocks are naturally combined to form the initial decision map, and the initial decision map is optimized to obtain the final decision map. Finally, the fused image is obtained by the weighted-average rule according to the final decision map. Experimental results show that the proposed method achieves better performance compared to 14 other state-of-the-art multi-focus image fusion methods.

A fuzzy convolutional neural network for enhancing multi-focus image fusion

The images captured by the cameras contain distortions, misclassified pixels, uncertainties and poor contrast. Therefore, the multi-focus image fusion (MFIF) integrates various input image features to produce a single fused image using all its objects in focus. However, it is computationally complex, which leads to inconsistency. Hence, the MFIF method is employed to generate the fused image by integrating the fuzzy sets (FS) and convolutional neural network (CNN) to detect focused and unfocused parts in both source images. It is also compared with other competing six MFIF methods like Neutrosophic set based stationary wavelet transform (NSWT), guided filters, CNN, ensemble CNN, image fusion-based CNN and deep regression pair learning (DRPL). Benchmark datasets validate the superiority of the proposed FCNN method in terms of four non-reference assessment measures having mutual information (1.1678), edge information (0.7281), structural similarity (0.9850) and human perception (0.8020) and two reference metrics such as Peak signal-to-noise ratio (57.23) and root mean square error (1.814).

Color multi-focus image fusion based on transfer learning

Multi-focus image fusion is a technique that integrates the focused areas in a pair or set of source images with the same scene into a fully focused image. Inspired by transfer learning, this paper proposes a novel color multi-focus image fusion method based on deep learning. First, color multi-focus source images are fed into VGG-19 network, and the parameters of convolutional layer of the VGG-19 network are then migrated to a neural network containing multilayer convolutional layers and multilayer skip-connection structures for feature extraction. Second, the initial decision maps are generated using the reconstructed feature maps of a deconvolution module. Third, the initial decision maps are refined and processed to obtain the second decision maps, and then the source images are fused to obtain the initial fused images based on the second decision maps. Finally, the final fused image is produced by comparing the QABF metrics of the initial fused images. The experimental results show that the proposed method can effectively improve the segmentation performance of the focused and unfocused areas in the source images, and the generated fused images are superior in both subjective and objective metrics compared with most contrast methods.

A simple and effective multi-focus image fusion method based on local standard deviations enhanced by the guided filter

This paper develops a simple and effective multi-focus image fusion method based on local standard deviations of the corresponding Laplacian images of source images and further enhanced by the guided filter. The underlying idea of this pixel-based approach is that the sharper pixels generally should have a comparatively higher local variance and hence higher local standard deviation. We first apply the Laplacian operation on each partially focused source image of the same scene, estimate the local standard deviation for each pixel, and enhance the local standard deviations using the guided filter. We then employ the filtered local standard deviation of the Laplacian image as an initial focus measure and combine it with the small region removal strategy to construct a decision map for pixel selection. Finally, according to the decision map, the target all-in-focus fused image is formed pixel-by-pixel. A variant of the proposed method with further guided filtering on the decision map is also developed. Numerical results demonstrate the proposed methods’ high performance compared with some state-of-the-art techniques.

A New Robust Multi focus image fusion Method

In today's digital era, multi focus picture fusion is a critical problem in the field of computational image processing. In the field of fusion information, multi-focus picture fusion has emerged as a significant research subject. The primary objective of multi focus image fusion is to merge graphical information from several images with various focus points into a single image with no information loss. We provide a robust image fusion method that can combine two or more degraded input photos into a single clear resulting output image with additional detailed information about the fused input images. The targeted item from each of the input photographs is combined to create a secondary image output. The action level quantities and the fusion rule are two key components of picture fusion, as is widely acknowledged. The activity level values are essentially implemented in either the "spatial domain" or the "transform domain" in most common fusion methods, such as wavelet. The brightness information computed from various source photos is compared to the laws developed to produce brightness / focus maps by using local filters to extract high-frequency characteristics. As a result, the focus map provides integrated clarity information, which is useful for a variety of Multi focus picture fusion problems. Image fusion with several modalities, for example. Completing these two jobs, on the other hand. As a consequence, we offer a strategy for achieving good fusion performance in this study paper. A Convolutional Neural Network (CNN) was trained on both high-quality and blurred picture patches to represent the mapping. The main advantage of this idea is that it can create a CNN model that can provide both the Activity level Measurement" and the Fusion rule, overcoming the limitations of previous fusion procedures. Multi focus image fusion is demonstrated using microscopic images, medical imaging, computer visualization, and Image information improvement is also a benefit of multi-focus image fusion. Greater precision is necessary in terms of target detection and identification. Face recognition" and a more compact work load, as well as enhanced system consistency, are among the new features.

Multi-Focus Image Fusion Based on Convolution Neural Network for Parkinson's Disease Image Classification.

Parkinson’s disease (PD) is a common neurodegenerative disease that has a significant impact on people’s lives. Early diagnosis is imperative since proper treatment stops the disease’s progression. With the rapid development of CAD techniques, there have been numerous applications of computer-aided diagnostic (CAD) techniques in the diagnosis of PD. In recent years, image fusion has been applied in various fields and is valuable in medical diagnosis. This paper mainly adopts a multi-focus image fusion method primarily based on deep convolutional neural networks to fuse magnetic resonance images (MRI) and positron emission tomography (PET) neural photographs into multi-modal images. Additionally, the study selected Alexnet, Densenet, ResNeSt, and Efficientnet neural networks to classify the single-modal MRI dataset and the multi-modal dataset. The test accuracy rates of the single-modal MRI dataset are 83.31%, 87.76%, 86.37%, and 86.44% on the Alexnet, Densenet, ResNeSt, and Efficientnet, respectively. Moreover, the test accuracy rates of the multi-modal fusion dataset on the Alexnet, Densenet, ResNeSt, and Efficientnet are 90.52%, 97.19%, 94.15%, and 93.39%. As per all four networks discussed above, it can be concluded that the test results for the multi-modal dataset are better than those for the single-modal MRI dataset. The experimental results showed that the multi-focus image fusion method according to deep learning can enhance the accuracy of PD image classification.

Two‐stage progressive residual learning network for multi‐focus image fusion

Boundary artifacts and color detail distortion are easily caused by the common multi-focus image fusion methods. In order to solve this problem, we propose a two-stage progressive residual learning network for multi-focus image fusion. The proposed network can progressively learn color information and detail features through end-to-end mapping. The whole network is composed of two sub-networks: the initial fusion block network and the enhanced fusion block network. First, the color information in the source image is fused by the initial fusion block network to generate the initial fusion image. Then on the basis of the initial fusion image, the detailed features of the source image are further fused by enhanced the fusion network to form the final fusion image. In order to solve the problem of lack of groundtruth when multi-focus image fusion is carried out with supervised method, the multi-focus image fusion problem is compared to the easy-to-solve image restoration problem. A synthetic dataset for network training is generated by ”degenerating” the VOC2012 dataset according to the set rules. After training, the method works well for fusion tasks without further processing. Experimental results show that the proposed method is superior to the existing methods in subjective visual perception and objective quantitative evaluation.

A novel multi-focus image fusion method based on joint regularization optimization layering and sparse representation

In this paper, we reported a novel Multi-Focus Image Fusion (MFIF) method based on regularization optimization and sparse representation model, which takes full advantages of the imaging characteristics of multi-focus images. First, a multi-source regularization optimization model was proposed to divide source images into a common background layer and respective detail layers jointly. This makes the focused details scattered in different sources are extracted in concert. Then the resulting detail layers are fused in sparse representation domain. To be more task-specific for MFIF, an atom focus degree measurement based fusion rule was proposed to highlight the focused atoms. Finally, the fusion result is reconstructed by combining the fused detail layer and the common background layer. Thorough experimental evaluations confirm the effectiveness of the proposed fusion approach both visually and quantitatively, and show that our approach outperforms some state-of-the-art fusion methods.

Multi-Focus Image Fusion Using Focal Area Extraction in a Large Quantity of Microscopic Images.

The non-invasive examination of conjunctival goblet cells using a microscope is a novel procedure for the diagnosis of ocular surface diseases. However, it is difficult to generate an all-in-focus image due to the curvature of the eyes and the limited focal depth of the microscope. The microscope acquires multiple images with the axial translation of focus, and the image stack must be processed. Thus, we propose a multi-focus image fusion method to generate an all-in-focus image from multiple microscopic images. First, a bandpass filter is applied to the source images and the focus areas are extracted using Laplacian transformation and thresholding with a morphological operation. Next, a self-adjusting guided filter is applied for the natural connections between local focus images. A window-size-updating method is adopted in the guided filter to reduce the number of parameters. This paper presents a novel algorithm that can operate for a large quantity of images (10 or more) and obtain an all-in-focus image. To quantitatively evaluate the proposed method, two different types of evaluation metrics are used: “full-reference” and “no-reference”. The experimental results demonstrate that this algorithm is robust to noise and capable of preserving local focus information through focal area extraction. Additionally, the proposed method outperforms state-of-the-art approaches in terms of both visual effects and image quality assessments.

Multi-Focus Image Fusion Method Based on Multi-Scale Decomposition of Information Complementary.

Multi-focus image fusion is an important method used to combine the focused parts from source multi-focus images into a single full-focus image. Currently, to address the problem of multi-focus image fusion, the key is on how to accurately detect the focus regions, especially when the source images captured by cameras produce anisotropic blur and unregistration. This paper proposes a new multi-focus image fusion method based on the multi-scale decomposition of complementary information. Firstly, this method uses two groups of large-scale and small-scale decomposition schemes that are structurally complementary, to perform two-scale double-layer singular value decomposition of the image separately and obtain low-frequency and high-frequency components. Then, the low-frequency components are fused by a rule that integrates image local energy with edge energy. The high-frequency components are fused by the parameter-adaptive pulse-coupled neural network model (PA-PCNN), and according to the feature information contained in each decomposition layer of the high-frequency components, different detailed features are selected as the external stimulus input of the PA-PCNN. Finally, according to the two-scale decomposition of the source image that is structure complementary, and the fusion of high and low frequency components, two initial decision maps with complementary information are obtained. By refining the initial decision graph, the final fusion decision map is obtained to complete the image fusion. In addition, the proposed method is compared with 10 state-of-the-art approaches to verify its effectiveness. The experimental results show that the proposed method can more accurately distinguish the focused and non-focused areas in the case of image pre-registration and unregistration, and the subjective and objective evaluation indicators are slightly better than those of the existing methods.

A Deep and Supervised Atrous Convolutional Model for Multi-Focus Image Fusion

Due to the limitations of equipment and technology, few images captured by a single sensor can describe all the information in a scene. Multi-focus image fusion techniques provide an effective means of integrating images acquired from different sensors to produce a single fused image that is richer in scene information. To obtain better fusion results, we propose a novel deep learning structure for multi-focus image fusion to achieve end-to-end network structure without post-processing procedures. Specifically, an innovative Siamese multi-scale feature extraction module was introduced to extract multi-scale image features. Besides, aiming to enhance the quality of the fused image, we further designed a new adaptive fusion strategy by using the attention mechanism instead of following the previous manual design. Extensive experimental results confirm that the proposed algorithm is more effective, which surpasses the existing state-of-the-art multi-focus image fusion methods both quantitatively and qualitatively. Furthermore, additional expanded experiment conducted on the infrared and visible image dataset demonstrates that the proposed fusion framework is also capable of performing other fusion tasks with excellent fusion performance. The code of the proposed fusion method is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/govenda/</uri> MSPA-Fuse.

A multi‐focus image fusion method based on multi‐source joint layering and convolutional sparse representation

In this paper, a new Multi-Focus Image Fusion (MFIF) method based on multi-source joint layering and Convolutional Sparse Representation (CSR) is proposed. Based on the characteristics of multi-focus source images, a multi-source joint layering regularization model was designed to divide the sources into a common base-layer and respective focus detail-layers. This strategy can overcome the defects caused by source layering separately effectively. In detail-layer fusion, CSR was employed to extract and global features. It can avoid detail blur and high computational cost caused by image blocking in the conventional sparse representation model. The proposed detail-layer fusion rule combined the CSR coefficient maps pairwise with the window based select-max rule. In the experiments, the optimal layering parameter was selected by experiments at first, and then five recently proposed specific MFIF or general image fusion algorithms were contrasted with the proposed method by plenty of subjective and objective experimental comparisons. The experimental results demonstrated the superiority of the authors’ method.

Multi-focus image fusion via Joint convolutional analysis and synthesis sparse representation

Recently, various sparse representation methods have been successfully used in multi-focus image fusion. Most of them produce some spatial artifacts and blurring effects because they only consider the image local information due to the patch processing strategy. In order to reduce the spatial artifacts and blurring effects on the edge details and improve the robustness of the multi-focus image fusion, a novel fusion method based on joint convolutional analysis and synthesis (JCAS) sparse representation is presented. The JCAS model, which integrates the analysis sparse representation and the synthesis sparse representation by using convolutional operation, can effectively separate large-scale structures and fine-scale textures of a single image. First, each source image is decomposed into a base layer and a detail layer using the JCAS model. Second, a Laplacian pyramid transform method is used to fuse the base layers, and a weighted regional energy method is used to fuse the detail layers. Finally, the fused image is reconstructed by combining the fused base and detail layers. Experimental results demonstrate that the proposed method can obtain clearer edge details compared with some popular multi-focus image fusion methods, thus exhibiting state-of-the-art performance in terms of both visual quality and objective assessment.

Multi-focus Image Fusion Method Based on PCANet in NSST Domain

Multi-focus Image Fusion Method Based on PCANet in NSST Domain

Identification of overlapped wool/cashmere fibers based on multi-focus image fusion and convolutional neural network

ABSTRACT Identification of cashmere and wool fibers has always been one of the essential topics in the field of textile. To solve this problem, this paper proposes anew overlapped fiber recognition method based on multi-focus image fusion and convolutional neural network. Firstly, the images of wool and cashmere fibers were captured by an optical microscope and adigital camera. Secondly, the fiber images with clear features were acquired by the multi-focus image fusion method. Then, image preprocessing operations were used to obtain single fiber images. Finally, the features of overlapped fibers are extracted and classified by the classical convolutional neural network. The results show that when the ratio of the training set to the test set is 8:2, the recognition accuracy of the method proposed in this paper reaches the highest of 98.7%, which can be used to realize the fast and accurate identification of overlapped fibers.