Multi-focus Image Research Articles

FusionGCN: Multi-focus image fusion using superpixel features generation GCN and pixel-level feature reconstruction CNN

In recent years, convolutional neural networks have demonstrated significant advancements in the domain of computer vision, effectively addressing numerous previously challenging issues. An increasing number of researchers are focusing their investigations on this field, proposing innovative network architectures. However, many existing networks necessitate intricate module designs and a substantial number of parameters to achieve satisfactory fusion outcomes, which poses challenges for lightweight devices with constrained computational resources. To mitigate this concern, the present study introduces a novel methodology that integrates block segmentation with pixel optimization. Specifically, we initially employ graph convolutional networks to execute flexible convolutions on large-scale, irregular regions generated through superpixel clustering, thereby achieving coarse segmentation at the block level. Subsequently, we utilize parallel lightweight convolutional networks to provide pixel-level guidance, ultimately resulting in a more accurate decision map. Furthermore, to leverage the strengths of both networks and facilitate the optimization of feature generation from the graph convolutional network for non-Euclidean data, we meticulously design a superpixel-based graph decoder alongside a pixel-based convolutional neural network extraction block to enhance feature acquisition and propagation. In comparison to numerous state-of-the-art methodologies, our approach demonstrates commendable competitiveness in both qualitative and quantitative analyses, as well as in efficiency evaluations. The code can be downloaded at https://github.com/ouyangbaicai/FusionGCN.

UUD-Fusion: An unsupervised universal image fusion approach via generative diffusion model

Image fusion is a classical problem in the field of image processing whose solutions are usually not unique. The common image fusion methods can only generate a fixed fusion result based on the source image pairs. They tend to be applicable only to a specific task and have high computational costs. Hence, in this paper, a two-stage unsupervised universal image fusion with generative diffusion model is proposed, termed as UUD-Fusion. For the first stage, a strategy based on the initial fusion results is devised to offload the computational effort. For the second stage, two novel sampling algorithms based on generative diffusion model are designed. The fusion sequence generation algorithm (FSGA) searches for a series of solutions in the solution space by iterative sampling. The fusion image enhancement algorithm (FIEA) greatly improves the quality of the fused images. Qualitative and quantitative evaluations of multiple datasets with different modalities demonstrate the great versatility and effectiveness of UUD-Fusion. It is capable of solving different fusion problems, including multi-focus image fusion task, multi-exposure image fusion task, infrared and visible fusion task, and medical image fusion task. The proposed approach is superior to current state-of-the-art methods. Our code is publicly available at https://github.com/xiangxiang-wang/UUD-Fusion.

Multi-Focus Image Fusion Algorithm Based on Multi-Task Learning and PS-ViT

Multi-Focus Image Fusion Algorithm Based on Multi-Task Learning and PS-ViT

SwinMFF: toward high-fidelity end-to-end multi-focus image fusion via swin transformer-based network

SwinMFF: toward high-fidelity end-to-end multi-focus image fusion via swin transformer-based network

Multi-Focus Image Fusion Using Energy Valley Optimization Algorithm

When a natural scene is photographed using imaging sensors commonly used today, part of the image is obtained sharply while the other part is obtained blurry. This problem is called limited depth of field. This problem can be solved by fusing the sharper parts of multi-focus images of the same scene. These methods are called multi-focus image fusion methods. This study proposes a block-based multi-focus image fusion method using the Energy Valley Optimization Algorithm (EVOA), which has been introduced in recent years. In the proposed method, the source images are first divided into uniform blocks, and then the sharper blocks are determined using the criterion function. By fusing these blocks, a fused image is obtained. EVOA is used to optimize the block size. The function that maximizes the quality of the fused image is used as the fitness function of the EVOA. The proposed method has been applied to commonly used image sets. The obtained experimental results are compared with the well-known Genetic Algorithm (GA), Differential Evolution Algorithm (DE), and Artificial Bee Colony Optimization Algorithm (ABC). The experimental results show that EVOA can compete with the other block-based multi-focus image fusion algorithms.

MMAE: A universal image fusion method via mask attention mechanism

As an important carrier of data, images contain a huge amount of information. The purpose of image fusion is to integrate the information from source images into a single image. Since the source images are from the same scene, there is much redundant information between them. Common fusion methods do not filter this information and the fusion process is often disturbed. This leads to degradation in the quality of the reconstructed fused image. To solve this problem, this paper explores the strategies of information filtering and fusion control, and proposes a universal image fusion method based on the mask attention mechanism. It can be divided into pre-training stage and formal fusion stage. In the pre-training stage, coarse-grained mask maps are generated which are employed to improve the mask autoencoder and the mask attention mechanism. In the image formal fusion stage, with the help of coarse-grained mask maps, the mask autoencoder changes the process of random masking and discards redundant features between source images. Meanwhile the mask attention mechanism focuses on the distinctions between various source images and retains effective complementary information. Qualitative and quantitative extension experiments on different modal datasets validate the applicability of the model in multi-focus image fusion, infrared and visible image fusion, and medical image fusion. Our method achieves excellent performance in all these tasks and performs better than existing fusion methods. Our code is publicly available at https://github.com/xiangxiang-wang/MMAE.

Multi-focus image fusion method based on adaptive weighting and interactive information modulation

Multi-focus image fusion method based on adaptive weighting and interactive information modulation

Improved multi-focus image fusion using online convolutional sparse coding based on sample-dependent dictionary

Multi-focus image fusion merges multiple images captured from different focused regions of a scene to create a fully-focused image. Convolutional sparse coding (CSC) methods are commonly employed for accurate extraction of focused regions, but they often disregard computational costs. To overcome this, an online convolutional sparse coding (OCSC) technique was introduced, but its performance is still limited by the number of filters used, affecting overall performance negatively. To address these limitations, a novel approach called Sample-Dependent Dictionary-based Online Convolutional Sparse Coding (SCSC) was proposed. SCSC enables the utilization of additional filters while maintaining low time and space complexity for processing high-dimensional or large data. Leveraging the computational efficiency and effective global feature extraction of SCSC, we propose a novel method for multi-focus image fusion. Our method involves a two-layer decomposition of each source image, yielding a base layer capturing the predominant features and a detail layer containing finer details. The amalgamation of the fused base and detail layers culminates in the reconstruction of the final image. The proposed method significantly mitigates artifacts, preserves fine details at the focus boundary, and demonstrates notable enhancements in both visual quality and objective evaluation of multi-focus image fusion.

MSI-DTrans: A multi-focus image fusion using multilayer semantic interaction and dynamic transformer

Multi-focus image fusion (MFIF) aims to utilize multiple images with different focal lengths to fuse into a single full-focus image. This process enhances the realism and clarity of the resulting image. In this paper, a MFIF method called MSI-DTrans was proposed. On the one hand, in order to fully utilize all the effective information that the source image carries, the proposed method adopts a multilayer semantic interaction strategy to enhance the interaction of high-frequency and low-frequency information. This approach gradually mines more abstract semantic information, guiding the generation of feature maps from coarse to fine. On the other hand, a parallel multi-scale joint self-attention computation model is designed. The model adopts dynamic sense field and dynamic token embedding to overcome the performance degradation problem when dealing with multi-scale objects. This enables self-attention to integrate long-range dependencies between objects of different scales and reduces computational overhead. Numerous experimental results show that the proposed method effectively avoids image distortion, achieves better visualization results, and demonstrates good competitiveness with many state-of-the-art methods in terms of qualitative and quantitative analysis, as well as efficiency comparison. The source code is available at https://github.com/ouyangbaicai/MSI-DTrans.

Multi-focus Image Fusion via Blind Deconvolution and Registration based on Photoacoustic Microscopy

Abstract Optical resolution photoacoustic microscopy (OR-PAM), which can present the microstructure of biological tissues is useful in the disease diagnosis. However, conventional OR-PAM can only obtain images with high resolution in a narrow range of depths around the focal plane. In this study, we proposed a multi-focus photoacoustic image fusion method based on the blind deconvolution. This method is divided into two steps. First, we used the method of blind deconvolution to obtain an approximate point spread function (PSF) for each photoacoustic images obtained from multiple layers, and then got the corresponding deblurred photoacoustic images. Second, we used the method of multi-focus images registration and fusion to accurately align the multi-focus photoacoustic images and effectively fuse the features of different images. Finally, the proposed method was applied to process the photoacoustic images obtained from photoacoustic imaging experiments on ants with irregular morphology using fast controllable confocal focus PAM (FC-PAM). The results show that our method improves the quality of photoacoustic images, and can quickly obtain multi-focus fused photoacoustic images with high resolution. It indicates that the proposed multi-focus photoacoustic image fusion method based on FC-PAM has great potential in fast imaging of living biological tissues at multiple imaging depths.

EDOM-MFIF: an end-to-end decision optimization model for multi-focus image fusion

EDOM-MFIF: an end-to-end decision optimization model for multi-focus image fusion

FHFN: content and context feature hierarchical fusion networks for multi-focus image fusion

FHFN: content and context feature hierarchical fusion networks for multi-focus image fusion

Rethinking the Effectiveness of Objective Evaluation Metrics in Multi-Focus Image Fusion: A Statistic-Based Approach.

As an effective technique to extend the depth-of-field (DOF) of optical lenses, multi-focus image fusion has recently become an active topic in image processing community. However, a major problem remaining unsolved in this field is the lack of universal criteria in selecting objective evaluation metrics. Consequently, the metrics utilized in different studies often vary significantly, leading to high difficulties in achieving unbiased evaluation. To address this problem, this paper proposes a statistic-based approach for verifying the effectiveness of objective metrics in multi-focus image fusion. The core idea is to adopt statistical correlation measures to evaluate the performance consistency between a certain fusion metric and some popular full-reference image quality assessment models. In addition, a convolutional neural network (CNN)-based fusion metric is presented to measure the similarity between the source images and the fused image based on the semantic features at multiple abstraction levels. A comparative study is conducted to evaluate 20 existing fusion metrics using the proposed statistic-based approach on a large-scale, realistic and with-ground-truth multi-focus image fusion dataset recently released. Experimental results demonstrate the feasibility of the proposed approach in evaluating the effectiveness of objective metrics and the advantage of our CNN-based metric.

FS-Transformer: A new frequency Swin Transformer for multi-focus image fusion

FS-Transformer: A new frequency Swin Transformer for multi-focus image fusion

A Novel Method for CSAR Multi-Focus Image Fusion

Circular synthetic aperture radar (CSAR) has attracted a lot of interest, recently, for its excellent performance in civilian and military applications. However, in CSAR imaging, the result is to be defocused when the height of an object deviates from a reference height. Existing approaches for this problem rely on digital elevation models (DEMs) for error compensation. It is difficult and costly to collect DEM using specific equipment, while the inversion of DEM based on echo is computationally intensive, and the accuracy of results is unsatisfactory. Inspired by multi-focus image fusion in optical images, a spatial-domain fusion method is proposed based on the sum of modified Laplacian (SML) and guided filter. After obtaining CSAR images in a stack of different reference heights, an all-in-focus image can be computed by the proposed method. First, the SMLs of all source images are calculated. Second, take the rule of selecting the maximum value of SML pixel by pixel to acquire initial decision maps. Secondly, a guided filter is utilized to correct the initial decision maps. Finally, fuse the source images and decision maps to obtain the result. A comparative experiment has been processed to verify the exceptional performance of the proposed method. The final processing result of real-measured CSAR data demonstrated that the proposed method is effective and practical.

Multi-Focus Images Fusion for Fluorescence Imaging Based on Local Maximum Luminosity and Intensity Variance.

Due to the limitations on the depth of field of high-resolution fluorescence microscope, it is difficult to obtain an image with all objects in focus. The existing image fusion methods suffer from blocking effects or out-of-focus fluorescence. The proposed multi-focus image fusion method based on local maximum luminosity, intensity variance and the information filling method can reconstruct the all-in-focus image. Moreover, the depth of tissue's surface can be estimated to reconstruct the 3D surface model.

Automated Morphological Grading of Human Blastocysts From Multi-Focus Images

This paper reports, for the first time, automated grading of human blastocysts (day-5 embryos) from multi-focus images. Based on a novel attention module, a convolutional neural network (CNN) was developed to predict the morphological grade of a blastocyst. The attention module integrates high-level features extracted from the blastocyst’s multi-focus images. Experimental results revealed that multi-focus blastocyst images help improve the grading accuracies than a single blastocyst image. Comparisons of the accuracy achieved by the model and the average accuracy of five embryologists demonstrated that the proposed model can outperform embryologists in the morphological grading of blastocysts (88% versus 86% for development stage prediction, 83% versus 79% for inner cell mass grade prediction, 89% versus 82% for trophectoderm grade prediction). <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This work was motivated by the subjectivity and significant intra-and inter-evaluator variations in manual morphological grading of blastocysts. Existing approaches to automate the grading process mainly use a single blastocyst image although multi-focus images captured at different focal planes reveal more morphological features of a blastocyst than a single blastocyst image. This paper describes a new CNN-based method using multi-focus images to improve the grading accuracy. The accuracy of the proposed method was verified on multi-focus images of human blastocysts captured by a standard time-lapse incubator at fixed focal depths ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$-$</tex-math> </inline-formula> 45 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu$</tex-math> </inline-formula> m, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$-$</tex-math> </inline-formula> 30 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu$</tex-math> </inline-formula> m, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$-$</tex-math> </inline-formula> 15 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu$</tex-math> </inline-formula> m, 0 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu$</tex-math> </inline-formula> m, 15 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu$</tex-math> </inline-formula> m, 30 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu$</tex-math> </inline-formula> m, 45 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu$</tex-math> </inline-formula> m).

Generation and Recombination for Multifocus Image Fusion With Free Number of Inputs

Generation and Recombination for Multifocus Image Fusion With Free Number of Inputs

A Security Study of Multimodel Artificial Intelligence System: Adaptive Retention Attack for Object Detection System with Multifocus Image Fusion Model

Image preprocessing models are usually employed as the preceding operations of high‐level vision tasks to improve the performance. The adversarial attack technology makes both these models face severe challenges. Prior research is focused solely on attacking single object detection models, without considering the impact of the preprocessing models (multifocus image fusion) on adversarial perturbations within the object detection system. Multifocus image fusion models work in conjunction with the object detection models to enhance the quality of the images and improve the capability of object detection system. Herein, the problem of attacking object detection system that utilizes multifocus image fusion as its preprocessing models is addressed. To retain the attack capabilities of adversarial samples against as many perturbations as possible, new attack method called adaptive retention attack (ARA) is proposed. Additionally, adversarial perturbations concentration mechanism and image selection mechanism, which, respectively, enhance the transferability and attack capability of ARA‐generated adversarial samples. Extensive experiments have demonstrated the feasibility of the ARA. The results confirm that the ARA method can successfully bypass multifocus image fusion models to attack the object detection model.

Magnification of a multi-focus infinity corrected microscope

The problem of image magnification in a bright field multi-focus imaging microscope is discussed. Such microscope allows simultaneous observation of two objects axially separated by a distance that exceeds the depth-of-field of the microscope, so-called multi-planar imaging. It is shown that the tunable lens required for operation of the microscope can be used in constant magnification imaging in the single focus regime, but not in multi-focus. Additionally, it is presented that the microscope allows for multi-planar imaging with satisfactory quality with a selected magnification of the image of one of the two objects. While the microscope uses a thermal lens as the tunable lens, the scheme of the optical setup needed to achieve the desired operation of the microscope is universal and applies to all kinds of tunable lenses used in microscopy.