Multi-focus Image Research Articles

STFuse: Infrared and Visible Image Fusion via Semisupervised Transfer Learning.

Infrared and visible image fusion (IVIF) aims to obtain an image that contains complementary information about the source images. However, it is challenging to define complementary information between source images in the lack of ground truth and without borrowing prior knowledge. Therefore, we propose a semisupervised transfer learning-based method for IVIF, termed STFuse, which aims to transfer knowledge from an informative source domain to a target domain, thus breaking the above limitations. The critical aspect of our method is to borrow supervised knowledge from the multifocus image fusion (MFIF) task and to filter out task-specific attribute knowledge by using a guidance loss Lg , which motivates its cross-task use in IVIF tasks. Using this cross-task knowledge effectively alleviates the limitation of the lack of ground truth on fusion performance, and the complementary expression ability under the constraint of supervised knowledge is more instructive than prior knowledge. Moreover, we designed a cross-feature enhancement module (CEM) that utilizes self-attention and mutual-attention features to guide each branch to refine features and then facilitate the integration of cross-modal complementary features. Extensive experiments demonstrate that our method has good advantages in terms of visual quality and statistical metrics, as well as the docking of high-level vision tasks, compared with other state-of-the-art methods.

Multi-focused Image fusion in HDWT Domain

Multi-focused image fusion becomes crucial one to extend depth of field of machine vision cameras. It combines two or more images focusing different objects of the same scene to form multi-focused fused image. This paper proposes simple and effective method of multi-focused image fusion using local gradient energy in higher density discrete wavelet transform domain, which is shift invariant and over complete by a factor of 5. This wavelet transform transfers all spectral and spatial information of all source images into multi-focused fused image without any distortions. The qualitative and quantitative analyses using metrics without reference prove the proposed method is considerably superior than other fusion methods using Discrete wavelet transform, Stationary wavelet transform, and double density discrete wavelet transform.

Exploit the Best of Both End-to-End and Map-Based Methods for Multi-Focus Image Fusion

Exploit the Best of Both End-to-End and Map-Based Methods for Multi-Focus Image Fusion

Algorithm for Multifocus Image Fusion Based on Low-Rank and Sparse Matrix Decomposition and Discrete Cosine Transform

Algorithm for Multifocus Image Fusion Based on Low-Rank and Sparse Matrix Decomposition and Discrete Cosine Transform

A new classification-based approach for multi-focus image fusion

A new classification-based approach for multi-focus image fusion

Focus Relationship Perception for Unsupervised Multi-Focus Image Fusion

Focus Relationship Perception for Unsupervised Multi-Focus Image Fusion

Improving multi-focus image fusion through Noisy image and feature difference network

Limited by optical constraints, the acquisition of a comprehensive set of multi-focus images remains a challenge. Leveraging the intrinsic accuracy of focused images, the concept of multi-focus image fusion emerges. However, distinguishing between focused and defocused regions, despite their visual similarities, is complex due to the absence of direct numerical indicators. This study unveils a novel insight: noisy source images lead to more substantial information loss in focus areas than in defocused regions. This observation prompts us to exploit this discrepancy by introducing noise to the source images. Motivated by this discovery, we introduce the Feature Difference Network (DDMF) for Multi-Focus Image Fusion (MFIF), aiming to leverage the differences present within feature dimensions. The pioneering DDMF approach incorporates the diffusion process from the Denoising Diffusion Probabilistic Models, employing it as a mechanism to introduce Gaussian noise to source images. Furthermore, the denoising process enhances feature representation. This equips DDMF to effectively capture hidden differences within features, allowing precise categorization of each pixel. Our extensive experimental evaluation underscores the prowess of DDMF. Through both subjective visual assessment and objective evaluation metrics, DDMF emerges as a frontrunner, surpassing established state-of-the-art MFIF methods.ARTICLE INFO.

MCDFD: Multifocus Image Fusion Based on Multiscale Cross-Difference and Focus Detection

MCDFD: Multifocus Image Fusion Based on Multiscale Cross-Difference and Focus Detection

An Implicit Neural Representation for the Image Stack: Depth, All in Focus, and High Dynamic Range

In everyday photography, physical limitations of camera sensors and lenses frequently lead to a variety of degradations in captured images such as saturation or defocus blur. A common approach to overcome these limitations is to resort to image stack fusion, which involves capturing multiple images with different focal distances or exposures. For instance, to obtain an all-in-focus image, a set of multi-focus images is captured. Similarly, capturing multiple exposures allows for the reconstruction of high dynamic range. In this paper, we present a novel approach that combines neural fields with an expressive camera model to achieve a unified reconstruction of an all-in-focus high-dynamic-range image from an image stack. Our approach is composed of a set of specialized implicit neural representations tailored to address specific sub-problems along our pipeline: We use neural implicits to predict flow to overcome misalignments arising from lens breathing, depth, and all-in-focus images to account for depth of field, as well as tonemapping to deal with sensor responses and saturation - all trained using a physically inspired supervision structure with a differentiable thin lens model at its core. An important benefit of our approach is its ability to handle these tasks simultaneously or independently, providing flexible post-editing capabilities such as refocusing and exposure adjustment. By sampling the three primary factors in photography within our framework (focal distance, aperture, and exposure time), we conduct a thorough exploration to gain valuable insights into their significance and impact on overall reconstruction quality. Through extensive validation, we demonstrate that our method outperforms existing approaches in both depth-from-defocus and all-in-focus image reconstruction tasks. Moreover, our approach exhibits promising results in each of these three dimensions, showcasing its potential to enhance captured image quality and provide greater control in post-processing.

Multi-Focus Microscopy Image Fusion Based on Swin Transformer Architecture

In this study, we introduce the U-Swin fusion model, an effective and efficient transformer-based architecture designed for the fusion of multi-focus microscope images. We utilized the Swin-Transformer with shifted window and path merging as the encoder for extracted hierarchical context features. Additionally, a Swin-Transformer-based decoder with patch expansion was designed to perform the un-sampling operation, generating the fully focused image. To enhance the performance of the feature decoder, the skip connections were applied to concatenate the hierarchical features from the encoder with the decoder up-sample features, like U-net. To facilitate comprehensive model training, we created a substantial dataset of multi-focus images, primarily derived from texture datasets. Our modulators demonstrated superior capability for multi-focus image fusion to achieve comparable or even better fusion images than the existing state-of-the-art image fusion algorithms and demonstrated adequate generalization ability for multi-focus microscope image fusion. Remarkably, for multi-focus microscope image fusion, the pure transformer-based U-Swin fusion model incorporating channel mix fusion rules delivers optimal performance compared with most existing end-to-end fusion models.

Optimized MobileNetV3: a deep learning-based Parkinson’s disease classification using fused images

Background and Objective Parkinson’s disease (PD) is a progressive neurological condition that manifests motor and non-motor symptoms. Early in the course of the disease, PD patients frequently experience vocal difficulties. In the beginning, preprocessing procedures were used with multi-focus image fusion to enhance the quality of input images. It is essential to diagnose and treat PD early to ensure that patients live healthy and productive lives. Methods Tremors, rigidity in the muscles, slow movement, difficulty balance, and other psychological symptoms are some of the disease’s symptoms. One of the critical mechanisms supporting PD identification and assessment is the dynamics of handwritten records. Several machine-learning techniques have been researched for the early detection of this disease. Yet the main problem with most of these manual feature extraction methods is their poor performance and accuracy. Results This cannot be acceptable when discovering such a chronic condition. For this purpose, a powerful deep learning model is suggested to help with the early diagnosis of Parkinson’s disease. Therefore, we proposed MobileNetV3-based classification. To enhance the classification performances even more, the MobileNetV3-based approach was optimized by the Improved Dwarf Mongoose Optimization algorithm (IDMO). Conclusion The Pyramid channel-based feature attention network (PCFAN) chooses the critical features. The efficiency of the approaches is tested using the PPMI and NTUA datasets. Our proposed approach obtains 99.34% accuracy, 98.53% sensitivity, 97.78% specificity, and 99.12% F-score compared to previous methods.

A defocus and similarity attention-based cascaded network for multi-focus and misaligned image fusion

Multi-focus image fusion uses multiple images focused on different depths to generate a clear image covering the whole scene. The existing multi-focus image fusion methods do not consider the defocus successive variation in close-range photography and the camera shaking in sequence image shooting, and most methods cannot process multiple images simultaneously. We proposed an end-to-end deep learning network to generate the all-in-focus image from multi-focus and misaligned images to solve these problems. Specifically, taking multiple multi-focus and misaligned source images as input, our Defocus and Similarity Attention Fusion Network (DSAF-Net) first generates the corresponding defocus map through the Defocus-Net, and warps the source images and defocus maps to a unified camera view according to the optical flow estimated by the OpticalFlow-Net. Finally, our model applies a coarse-to-fine correspondence matching scheme to obtain the similarity weight maps, combined with the warped source images and defocus maps to fuse into a clear image. For training and testing our DSAF-Net, a multi-focus and misaligned cultural heritage photography dataset (WHU-MFM) is constructed using Blender Cycles renderer. Experiments results demonstrate that our method outperforms the state-of-the-art methods both qualitatively and quantitatively. DSAF-Net is available at https://github.com/PeimingCHEN/DSAF-Net. And WHU-MFM is available at https://github.com/PeimingCHEN/WHU-MFM-Dataset.

All-in-focus large-FOV GRIN lens imaging by multi-focus image fusion

Gradient refractive index (GRIN) lenses are useful for miniaturized and in-vivo imaging. However, the intrinsic field-dependent aberrations of these lenses can deteriorate imaging resolution and limit the effective field of view. To address these aberrations, adaptive optics (AO) has been applied which inevitably requires the incorporation of additional hardware. Here we focus on field curvature aberration and propose a computational correction scheme which fuses a z-stack of images into a single in-focus image over the entire field of view (FOV), with no AO required. We validate our method by all-in-focus wide-field imaging of a printed letter sample and fluorescently labeled mouse brain slices. The method can also provide what we believe to be a new and valuable option for imaging enhancement in the scanning-modality use of GRIN lens microscopy.

Novel hybrid CNN with Bi-LSTM multi-focus image fusion method based on modified tetrolet transform in MRI and CT images

The process of fusing different images from various imaging modalities into a single, fused image that contains a wealth of information and improves the usability of medical images in real-world applications is known as medical image fusion. The most useful features from data can be automatically extracted by deep learning models. In the recent past, the field of image fusion has been preparing to introduce a deep learning model. In this work we can achieve the multi-Focus medical image fusion by hybrid deep learning models. Here the relevant health care data are collected from database (CT & MRI brain images). Following the input images are pre-processed using sliding window and the abnormal data is eliminated using distribution map method. Further the proposed work comprises 3 steps, 1) the proposed method is used to extract the features from the input image using the modified Tetrolet transform (MMT), which uses a brain image as an input image. This model is capable of identifying anomalous trends in time series data and automatically deriving from the input data characteristics that characterise the system state.2) Propose a novel hybrid model based on CNN with Bi-LSTM (Bi-directional Short Term Memory) multi-focus image fusion method to overcome the difficulty faced by the existing fusion methods. 3) This hybrid model are used to predict the brain tumor present in the fused image. Finally, experimental results are evaluated using a variety of performance measures. From the results, we can see that our suggested model contributes to an increase in predictive performance while also lowering the complexity in terms of storage and processing time.

An autoencoder deep residual network model for multi focus image fusion

An autoencoder deep residual network model for multi focus image fusion

Multi-focus image fusion via adaptive fractional differential and guided filtering

Multi-focus image fusion via adaptive fractional differential and guided filtering

FusionCPP: Cooperative fusion of infrared and visible light images based on PCNN and PID control systems

The imaging results of infrared and visible light sensors are highly complementary. Thus, combining the two images can compensate for the shortcomings of a single sensor. However, previous fusion methods have not fully utilized the collaborative guidance of infrared and visible images in the fusion process, leading to a poor correlation between extracted features and cannot guarantee feature fusion accuracy. In order to address these challenges, this paper proposes a collaborative fusion method for infrared and visible images based on pulse-coupled neural networks (PCNN) and proportional-integral-derivative (PID) control systems, called FusionCPP. First, a coupled neural network with a dual pulse structure is designed for the feature extraction section. Unlike traditional PCNN, this network includes two pulse generators, one of which is utilized to extract the salient features of the image, while the other is employed to extract the common pulse layer of the source image and multiply the pulse layer by the coupling iteration value to obtain the base layer. Second, a detail layer is obtained by subtracting the base layer from the source image. This method accurately extracts salient and detailed features based on the coupling between source images. Finally, a closed-loop PID control system is utilized as the fusion strategy in the image fusion task, which judges the difference between the fusion result and the source image in real-time through the feedback effect of the system. The controller adaptively adjusts the fusion weight based on the difference to fuse image features into the new image accurately. Experimental results indicate that the proposed FusionCPP can maintain significant contrast and rich textures. Besides, the proposed FusionCPP has extended to multi-focus image fusion and target detection tasks to verify its effectiveness.

FusionDiff: Multi-focus image fusion using denoising diffusion probabilistic models

Multi-focus image fusion (MFIF) is an image enhancement technology with broad application prospects that can effectively extend the depth-of-field of optical lenses. This paper presents a novel MFIF method, named FusionDiff, based on denoising diffusion probabilistic models (DDPM). FusionDiff uses DDPM to fuse two source images by iteratively performing multiple denoising operations. To predict noise accurately and train the model efficiently, a lightweight U-Net architecture is designed as the conditional noise predictor. FusionDiff does not depend on any specific activity level measurement method, fusion rule or complex feature extraction network. It overcomes many algorithm design and training difficulties in existing image fusion methods. FusionDiff is noise-resistant and can still produce outstanding fusion results from source images with noise interference. In addition, FusionDiff is a few-shot learning method, which makes it suitable for image fusion tasks where training samples are relatively scarce. Experiments show that FusionDiff outperforms representative state-of-the-art methods in both visual perception and quantitative metrics. The code is available at https://github.com/lmn-ning/ImageFusionhttps://github.com/lmn-ning/ImageFusion

Multi-image transformer for multi-focus image fusion

Multi-Focus Image Fusion (MFIF) is an image enhancement task that fuses images in which different regions are in focus to achieve an all-in-focus image. In recent years, Generative Adversarial Networks (GANs)-based approaches have significantly improved the MFIF on Convolutional Neural Network (CNN) architectures. However, despite vision transformers (ViTs) achieving more successful results than CNNs in many high and low-level vision problems due to their ability to provide global connectivity, they have not been employed for MFIF until this study. We develop a Multi-image Transformer (MiT) for MFIF by being inspired by a Spatial-Temporal Transformer Network (STTN) so that global connection can be modeled along multiple input images. We call the proposed transformers-based MFIF model MiT-MFIF as it uses the developed MiT as a core component. We have made various modifications to the baseline transformer to be able to utilize vision transformers in MFIF tasks. Comprehensive experiments on standard MFIF datasets demonstrate the effectiveness of the proposed MiT-MFIF. Moreover, proposed method does not require any post-processing step like in competitor GAN-based methods while outperforming the state-of-the-art MFIF methods.

AFCANet: An adaptive feature concatenate attention network for multi-focus image fusion

For multi-focus image fusion, the existing deep learning based methods cannot effectively learn the texture features and semantic information of the source image to generate high-quality fused images. Thus, we develop a new adaptive feature concatenate attention network named AFCANet, which adaptively learns cross-layer features and retains the texture features and semantic information of images to generate visually appealing fully focused images. In AFCANet, the encoder-decoder network is used as the backbone network. In the unsupervised training stage, an adaptive cross-layer skip connection mode is designed, and a cross-layer adaptive coordinate attention module is built to acquire meaningful information from the image along with ignoring unimportant information to obtain a better image fusion effect. In addition, in the middle of the encoder-decoder network, we also introduce an effective channel attention module to fully learn the output of the encoder, and accelerate network convergence. In the inference stage, we apply the pixel-based spatial frequency fusion rules to fuse the adaptive features learned by the encoder, which can successfully combine the texture and semantic information of the image and produce a more precise decision map. Extensive experiments on public datasets and the HBU-CVMDSP dataset show that our AFCANet can effectively improve the accuracy of the decision map in the focus and defocus regions, as well as improve the ability to retain the abundant details and edge features of the source image.