Multi-focus Image Fusion Technique Research Articles

Robust multi-focus image fusion using focus property detection and deep image matting

Multi-focus image fusion techniques are designed to integrate the focused regions of multiple source images to produce an all-in-focus image. Unfortunately, the fusion results generated by existing methods are prone to suffer from chromatic aberrations and blurred boundary regions. In this paper, a multi-focus image fusion framework combining focus property detection and deep image matting is proposed to achieve high-quality fusion results. Specifically, we introduce spatial frequency detection blocks, which integrate the spatial frequency information of source images at different levels to achieve effective focus property detection for the initial decision. Then, the trimap acquired from the initial decision is optimized using a deep matting model so as to further detect the focused/defocused boundaries and obtain a more precise decision map. Meanwhile, a new hybrid loss function is introduced, and a large-scale dataset is created to improve the performance of the fusion framework. Finally, the fusion result is obtained by integrating the focused regions via the decision map. Compared to 18 other advanced fusion methods, the proposed method outperforms all of them in terms of the seven objective metrics evaluated on two publicly available datasets. These experimental results provide strong evidence that the proposed fusion framework excels not only in subjective vision but also in quantitative analysis. The code of the proposed method will be available at https://github.com/govenda/DIMF.

Mathematical modeling and simulation of multi-focus image fusion techniques using the effect of image enhancement criteria: a systematic review and performance evaluation

Mathematical modeling and simulation of multi-focus image fusion techniques using the effect of image enhancement criteria: a systematic review and performance evaluation

A Survey of Multi-Focus Image Fusion Methods

As an important branch in the field of image fusion, the multi-focus image fusion technique can effectively solve the problem of optical lens depth of field, making two or more partially focused images fuse into a fully focused image. In this paper, the methods based on boundary segmentation was put forward as a group of image fusion method. Thus, a novel classification method of image fusion algorithms is proposed: transform domain methods, boundary segmentation methods, deep learning methods, and combination fusion methods. In addition, the subjective and objective evaluation standards are listed, and eight common objective evaluation indicators are described in detail. On the basis of lots of literature, this paper compares and summarizes various representative methods. At the end of this paper, some main limitations in current research are discussed, and the future development of multi-focus image fusion is prospected.

A Deep and Supervised Atrous Convolutional Model for Multi-Focus Image Fusion

Due to the limitations of equipment and technology, few images captured by a single sensor can describe all the information in a scene. Multi-focus image fusion techniques provide an effective means of integrating images acquired from different sensors to produce a single fused image that is richer in scene information. To obtain better fusion results, we propose a novel deep learning structure for multi-focus image fusion to achieve end-to-end network structure without post-processing procedures. Specifically, an innovative Siamese multi-scale feature extraction module was introduced to extract multi-scale image features. Besides, aiming to enhance the quality of the fused image, we further designed a new adaptive fusion strategy by using the attention mechanism instead of following the previous manual design. Extensive experimental results confirm that the proposed algorithm is more effective, which surpasses the existing state-of-the-art multi-focus image fusion methods both quantitatively and qualitatively. Furthermore, additional expanded experiment conducted on the infrared and visible image dataset demonstrates that the proposed fusion framework is also capable of performing other fusion tasks with excellent fusion performance. The code of the proposed fusion method is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/govenda/</uri> MSPA-Fuse.

Wavelet-Based Multi-Focus Image Fusion Using Average Method Noise Diffusion (AMND)

Aim: This paper presents a new and upgraded wavelet-based multi-focus image fusion technique using average method noise diffusion (AMND). Objective: Improved visual appearance of the final image, no blurring in the final fused image and clearly visible objects (fine edges). Methods: This method extends the standard wavelet-based image fusion technique on multi-focus images by incorporating the hybrid of method noise and anisotropic diffusion in it. This hybrid structure of method noise and anisotropic diffusion is implemented as the post-processing operation in the proposed method. Results: The proposed work shows excellent results in terms of visual appearance and edge preservation. The experimental results of the proposed method are compared with some of the traditional and non-traditional approaches where the proposed method shows comparatively better results. Conclusion: This paper depicts the robustness, effectiveness and adaptive nature of method noise in the field of image enhancement especially in the field of image fusion. The performance of the proposed method is analyzed qualitatively (good visual appearance) and quantitatively (entropy, spatial frequency, and standard deviation). This method has the capability to be incorporated in real-time applications like surveillance in the visual sensor network (VSN).

A new wavelet-based multi-focus image fusion technique using method noise and anisotropic diffusion for real-time surveillance application

A new wavelet-based multi-focus image fusion technique using method noise and anisotropic diffusion for real-time surveillance application

Multi-focus Image Fusion using Neutrosophic based Wavelet Transform

The key aim of Multi-focus Image Fusion (MFIF) is to gather all the necessary and useful information as well as features from the source images and then merge that information to develop a fused image. This fused image is having more information and better image quality than the source images. There exist a number of MFIF techniques in spatial as well as transform domain. However, since the source images have low resolution, high level of noise and are blurred, it becomes quite difficult for the traditional MFIF techniques to provide a fused image with high image quality. Thus, to fuse the source images efficiently, a hybrid MFIF method is proposed which consists of Neutrosophic Set and Stationary Wavelet Transform. Various state-of-the-art MFIF techniques have been used to compare the performance of the Neutrosophic Stationary Wavelet Transform (NSWT) technique. To evaluate the proposed technique, quantitative as well as qualitative evaluation has been done on two different datasets. To quantitatively evaluate the proposed method based on the NSWT, two different types of evaluation metrics are used namely ”reference-based” and ”reference-less”. From the experimental results and comparison with the existing state-of-the-art techniques it has been found that the NSWT has achieved better quantitative and qualitative results that in turn helpful in extending the Depth-of-Field of the imaging system. The achieved fusion results has shown the effectiveness of the proposed MFIF technique.

Multi-focus image fusion techniques: a survey

Multi-Focus Image Fusion (MFIF) is a method that combines two or more source images to obtain a single image which is focused, has improved quality and more information than the source images. Due to limited Depth-of-Field of the imagining system, extracting all the useful information from a single image is challenging. Thus two or more defocused source images are fused together to obtain a composite image. This paper provides a comprehensive overview of existing MFIF methods. A new classification scheme is developed for categorizing the existing MFIF methods. These methods are classified into four major categories: spatial domain, transform domain, deep leaning and their hybrids and have been discussed well along with their drawbacks and challenges. In addition to this, both the parametric evaluation metrics i.e. "with reference" and "without reference" have also discussed. Then, a comparative analysis for nine image fusion methods is performed based on 30 pairs of publicly available images. Finally, various challenges that remain unaddressed and future work is also discussed in this work.

Multifocus image fusion using te-cnn technique

A combination of two or more images is called an image fusion which produce an image that teaches more about computer perspective or human perception. The multi-focus image fusion technique in this paper uses CNN (Convolutional Neural Networks) to create a focus pixel-level maps on dynamic neural networks from multicolored images. This proposed CNN technique is entirely new approach when compared to the other integration methods which are utilized the same CNN method. In this proposed approach, the binary division is completely accessed to generate the focus maps. The result of this implementation prepares high-resolution pure images, view maps using bright field data. Also, produces exact focus maps next to a clear visual image. These mixing methods produce a proportional measurement. The deep learning method is used to perform the pixel separation for multi focus image fusion technique. The deep learning algorithm creates the datasets which are necessary to perform the image fusion in multi focus images by using focus maps and focus images.

A neuro fuzzy image fusion using block based feature level method

Image fusion is a sub field of image processing in which more than one images are fused to create an image where all the objects are in focus. The process of image fusion is performed for multi-sensor and multi-focus images of the same scene. Multi-sensor images of the same scene are captured by different sensors whereas multi-focus images are captured by the same sensor. In multi-focus images, the objects in the scene which are closer to the camera are in focus and the farther objects get blurred. Contrary to it, when the farther objects are focused then closer objects get blurred in the image. To achieve an image where all the objects are in focus, the process of images fusion is performed either in spatial domain or in transformed domain. In recent times, the applications of image processing have grown immensely. Usually due to limited depth of field of optical lenses especially with greater focal length, it becomes impossible to obtain an image where all the objects are in focus. Thus it plays an important role to perform other tasks of image processing such as image segmentation, edge detection, stereo matching and image enhancement. Hence, a novel feature-level multi-focus image fusion technique has been proposed which fuses multi-focus images. Thus the results of extensive experimentation performed to highlight the efficiency and utility of the proposed technique is presented. The proposed work further explores comparison between fuzzy based image fusion and neuro fuzzy fusion technique along with quality evaluation indices.

Analysis and Identification of Tumor Cells In MRI And CT Images Based On Nanotechnology Using Neuro Fuzzy Image Fusion

Image fusion is a sub field of Medical imaging in which more than one images are fused to create an image where all the objects are in focus. The process of image fusion is performed for multi-sensor and multi-focus images of the same scene. Multi-sensor images of the same scene are captured by different sensors whereas multi-focus images are captured by the same sensor. In multi-focus images, the objects in the scene which are closer to the camera are in focus and the farther objects get blurred. Contrary to it, when the farther objects are focused then closer objects get blurred in the image. To achieve an image where all the objects are in focus, the process of images fusion is performed either in spatial domain or in transformed domain. In recent times, the applications of image processing have grown immensely. Usually due to limited depth of field of optical lenses especially with greater focal length, it becomes impossible to obtain an image where all the objects are in focus. Thus it plays an important role to perform tasks of analysis and identification of tumor cells in MRI and CT images based on neuro fuzzy image fusion. Hence, a novel feature-level multi-focus image fusion technique has been proposed which fuses multi-focus images. Thus the results of extensive experimentation performed to highlight the efficiency and utility of the proposed technique is presented. The proposed work further explores comparison between fuzzy based image fusion and neuro fuzzy fusion technique along with quality evaluation indices. In order to benefit from the highly conformal irradiation of, sophisticated treatment planning and simulation are required to analysis and identify the tumor in MRI and CT Scan images. The purpose of this study was to investigate the potential of MRI and CT for treatment plan simulation and adaptation using a neuro fuzzy image fusion.

Hardware-Friendly Laplacian-Based Multi-Focus Image Fusion in DCT Domain for Visual Sensor Network

Visual sensor network (VSN) requires a multi-focus image or video frame fusion technique involving focus measure computation in the DCT-domain to generate an all-in-focus image. Such techniques are implemented on resource-constrained on-board systems requiring hardware-friendly implementations. In this article, we first show that components of the Laplacian matrix are related to the discrete cosine transform (DCT) basis. The relation is that the eigenvalues of the Laplacian with proper boundary condition form the diagonal elements of the diagonal matrix generated by the DCT operation on the Laplacian. Exploiting this relation, we propose a focus measure which works on the DCT coefficients reflecting the spatial-domain Laplacian operation. Certain simplifications allow our focus measure computation through hardware-friendly integer multiplication and summation, where matrix multiplication involves just N scalar multiplications for an N × N 2D signal. Finally, we propose an approach which suitably fuses multi-focus images or video frames in DCT based image or video coding framework through detection of properly focused area and neighborhood consistency analysis. We show that our proposed approach is hardwarefriendly, computationally simple, and is fast enough for VSN. Through experimental results, we show that our approach outperforms the relevant state-of-the-art in multi-focus image fusion for VSN both quantitatively and subjectively. We also show that our approach is effective in comparison to the state-of-the-art and a few latest generic multi-focus image fusion techniques in terms of quantitative and subjective evaluations.

3D shape reconstruction from multifocus image fusion using a multidirectional modified Laplacian operator

Multifocus image fusion techniques primarily emphasize human vision and machine perception to evaluate an image, which often ignore depth information contained in the focus regions. In this paper, a novel 3D shape reconstruction algorithm based on nonsubsampled shearlet transform (NSST) microscopic multifocus image fusion method is proposed to mine 3D depth information from the fusion process. The shift-invariant property of NSST guarantees the spatial corresponding relationship between the image sequence and its high-frequency subbands. Since the high-frequency components of an image represent the focus level of the image, a new multidirectional modified Laplacian (MDML) as the focus measure maps the high-frequency subbands to images of various levels of depth. Next, the initial 3D reconstruction result is obtained by using an optimal level selection strategy based on the summation of the multiscale Laplace responses to exploit these depth maps. Finally, an iterative edge repair method is implemented to refine the reconstruction result. The experimental results show that the proposed method has better performance, especially when the source images have low-contrast regions.

Multi-Focus Fused Image using Inception-Resnet V2

Multi-focus image fusion is the process of integration of pictures of the equivalent view and having various targets into one image. The direct capturing of a 3D scene image is challenging, many multi-focus image fusion techniques are involved in generating it from some images focusing at diverse depths. The two important factors for image fusion is activity level information and fusion rule. The necessity of designing local filters for extracting high-frequency details the activity level information is being implemented, and then by using various elaborated designed rules we consider clarity information of different source images which can obtain a clarity/focus map. However, earlier fusion algorithms will excerpt high-frequency facts by considering neighboring filters and by adopting various fusion conventions to achieve the fused image. However, the performance of the prevailing techniques is hardly adequate. Convolutional neural networks have recently used to solve the problem of multi-focus image fusion. By considering the deep neural network a two-stage boundary aware is proposed to address the issue in this paper. They are: (1) for extracting the entire defocus info of the two basis images deep network is suggested. (2) To handle the patches information extreme away from and close to the focused/defocused boundary, we use Inception ResNet v2. The results illustrate that the approach specified in this paper will result in an agreeable fusion image, which is superior to some of the advanced fusion algorithms in comparison with both the graphical and objective evaluations.

Multi-focus image fusion with random walks and guided filters

Multi-focus image fusion technique is able to help obtaining an all-focused image, which is advantage to human vision and image processing. In this paper, a novel multi-focus image fusion method is proposed based on random walk and guided filter. In the proposed method, the decomposition function and the optimizing function of random walk are used in multi-focus image fusion. And the random walk is also utilized for weight maps directly. The advantages of random walk and guided filter in image fusion are fully utilized by regulating proportional coefficients artificially. The proposed method concludes six steps: first, decomposing source images into detail layers and base layers with random walk; second, the random walk is used for weight maps directly and the guided filter is used as smoothing filters to get the streamlined weight maps of the detail layers and the base layers, respectively; third, the weight maps of the detail layers and the base layers are acquired by summing the initializing weight maps in different proportions; and then, the final weight maps of the detail layers are acquired using random walk for optimizing. After that, the fused detail layer and base layer are obtained by weighted average of detail layers and base layers, singly. Finally, the fused image is gained by summing up the fused base layer and the fused detail layer. Experiments demonstrate that the proposed method outperforms many other approaches in both subjective and objective assessments.

A Two-Stage Multi-Focus Image Fusion Framework Robust to Image Mis-Registration

Multi-focus image fusion technique is to yield a composite image with all objects in focus. However, most of fusion methods do not obtain the satisfactory performance when the focused objects in the source images are not registered. In this paper, a novel classification and probability optimization based multi-focus image fusion method is proposed, which consists of two main steps. First, by the integration of multinomial logistic regression classifier and random walker based optimization in a two-stage framework, we can extract the focused regions of each source image. Then, in order to construct the fused image, the focused regions are combined together. Experimental results show that compared with other advanced fusion methods, the proposed method can obtain competitive performance in terms of subjective and objective evaluations.

Multifocus Image Fusion Based on Fast Guided Filter and Focus Pixels Detection

As an effective technique of information fusion, multifocus image fusion has attracted increasing attention for image processing and computer vision. The goal of multifocus image fusion method is integrating all focus information from source images into fused result. In this paper, we propose a novel simple and effective multifocus image fusion technique based on fast guided filter and focus pixels detection. In order to detect the focused pixels correctly from source images, we develop a new multi-scale sum modified Laplacian technique. For the decision maps learning stage, based on the block consistency verification and faster guided filter techniques, we generate a series of binary focus decision maps to express the focus property of each pixel. In the fusion stage, we employ neighbor distance filter to extract detail pixels of source images, then the informative highpass images and the energetic lowpass images can be generated. The fused results are developed by constructing the corresponding decision maps and the neighbor distance filtered images. Compared with some state-of-the-art fusion methods, experimental results clearly demonstrate the superiority of the proposed method in terms of both comprehensive subjective assessment and some well-known quantitative evaluations.

Efficient Multi Focus Image Fusion Technique Optimized Using MOPSO for Surveillance Applications

This article describes how image fusion has taken giant leaps and emerged as a promising field with diverse applications. A fused image provides more information than any of the source images and it is very helpful in surveillance applications. In this article, an efficient multi focus image fusion technique is proposed in cascaded wavelet transform domain using swarm intelligence and spatial frequency (SF). Spatial frequency is used for computing the activity level and consistency verification (CV) based decision map is employed for acquiring the final fused coefficients. Justification for employing SF and CV is also discussed. This technique performs well compared to existing techniques even when the source images are severely blurred. The proposed framework is evaluated using quantitative metrics, such as root mean square error, peak signal to noise ratio, mean absolute error, percentage fit error, structural similarity index, standard deviation, mean gradient, Petrovic metric, SF, feature mutual information and entropy. Experimental outcomes demonstrate that the proposed technique outperforms the state-of-the art techniques, in terms of visual impact as well as objective assessment.

Multi-Focus Image Fusion in DCT Domain using Variance and Energy of Laplacian and Correlation Coefficient for Visual Sensor Networks

The purpose of multi-focus image fusion is gathering the essential information and the focused parts from the input multi-focus images into a single image. These multi-focus images are captured with different depths of focus of cameras. A lot of multi-focus image fusion techniques have been introduced using considering the focus measurement in the spatial domain. However, the multi-focus image fusion processing is very time-saving and appropriate in discrete cosine transform (DCT) domain, especially when JPEG images are used in visual sensor networks (VSN). So the most of the researchers are interested in focus measurements calculation and fusion processes directly in DCT domain. Accordingly, many researchers developed some techniques which are substituting the spatial domain fusion process with DCT domain fusion process. Previous works in DCT domain have some shortcomings in selection of suitable divided blocks according to their criterion for focus measurement. In this paper, calculation of two powerful focus measurements, energy of Laplacian (EOL) and variance of Laplacian (VOL), are proposed directly in DCT domain. In addition, two other new focus measurements which work by measuring correlation coefficient between source blocks and artificial blurred blocks are developed completely in DCT domain. However, a new consistency verification method is introduced as a post-processing, improving the quality of fused image significantly. These proposed methods reduce the drawbacks significantly due to unsuitable block selection. The output images quality of our proposed methods is demonstrated by comparing the results of proposed algorithms with the previous algorithms.

IMAGE FUSION FOR MULTIFOCUS IMAGES USING SPEEDUP ROBUST FEATURES

The multi-focus image fusion technique has emerged as major topic in image processing in order to generate all focus images with increased depth of field from multi focus photographs. Image fusion is the process of combining relevant information from two or more images into a single image. The image registration technique includes the entropy theory. Speed up Robust Features (SURF), feature detector and Binary Robust Invariant Scalable Key points (BRISK) feature descriptor is used in feature matching process. An improved RANDOM Sample Consensus (RANSAC) algorithm is adopted to reject incorrect matches. The registered images are fused using stationary wavelet transform (SWT).The experimental results prove that the proposed algorithm achieves better performance for unregistered multiple multi-focus images and it especially robust to scale and rotation translation compared with traditional direct fusion method.