Multi-focus Image Research Articles

Spectral- and spatial-based multi-focus image fusion method towards multi-layer nonwovens

In the microscopic imaging scenario where the object thickness exceeds the depth of field of the microscope, multi-focus image fusion (MFF) is an effective method to generate an all-in-focus image. However, for nonwoven fabric for which the captured image number is up to 100 or more, the existing methods often underperform in areas near the fiber edges, owing to image ghosting and noise accumulation caused by the platform moving. To address the above problem, this paper presents a method designed to fuse multi-layer micro-images based on the combination of spectral and spatial features of the images. Firstly, the spectral domain-based map is generated by decomposition and reconstruction of the high-frequency and low-frequency components of the images, aimed at obtaining the edge information. Simultaneously, the spatial domain-based fuse map is built through sharpness measurement, referring to visual perception. Finally, the two methods are combined via an optimized weight to obtain an all-in-focus fused image. Four groups of real-world data consisting of 100 multi-focus nonwoven images are utilized to verify the superiority of this method. The experimental results demonstrate that the proposed method can obtain satisfactory performance in terms of both human visual evaluation and objective evaluation compared with the image fusion framework based on the convolutional neural network, MFF, region-based image fusion algorithm and convolutional neural network state-of-the-art fusion methods.

High-speed real 3D scene acquisition and 3D holographic reconstruction system based on ultrafast optical axial scanning.

The lack of three-dimensional (3D) content is one of the challenges that have been faced by holographic 3D display. Here, we proposed a real 3D scene acquisition and 3D holographic reconstruction system based on ultrafast optical axial scanning. An electrically tunable lens (ETL) was used for high-speed focus shift (up to 2.5 ms). A CCD camera was synchronized with the ETL to acquire multi-focused image sequence of real scene. Then, the focusing area of each multi-focused image was extracted by using Tenengrad operator, and the 3D image were obtained. Finally, 3D holographic reconstruction visible to the naked eye can be achieved by the layer-based diffraction algorithm. The feasibility and effectiveness of the proposed method have been demonstrated by simulation and experiment, and the experimental results agree well with the simulation results. This method will further expand the application of holographic 3D display in the field of education, advertising, entertainment, and other fields.

Coherent Stokes Raman scattering microscopy (CSRS)

We report the first implementation of laser scanning coherent Stokes Raman scattering (CSRS) microscopy. To overcome the major challenge in CSRS imaging, we show how to suppress the fluorescence background by narrow bandpass filter and a lock-in based demodulation. Near background free CSRS imaging of polymer beads, human skin, onion cells, avocado flesh and the wing disc of a drosphila larva are presented. Finally, we explain and demonstrate numerically that CSRS solves a major obstacle of other coherent Raman techniques by sending a significant part (up to 100%) of the CSRS photons into the backward direction under tight focusing conditions. We believe that this discovery will pave the way for numerous technological advances, e.g., in epi-detected coherent Raman multi-focus imaging, real-time laser scanning based spectroscopy or efficient endoscopy.

VSP-Fuse: Multifocus Image Fusion Model Using the Knowledge Transferred From Visual Salience Priors

Multifocus image fusion (MFIF), as an efficient way to improve the visual effect of images with partial focus defects, is of great significance in the field of image enhancement. According to the imaging principle of the lens, we summarize the visual salience priors (VSP) from the daily photo scene and two relationships from MFIF. Thereby, an edge-sensitive model for MFIF is presented in this study. Supported by VSP, we consider the correlation between salience object detection (SOD) and MFIF, and select the former as a pre-training task. SOD provides the network with realistic depth of field and bokeh effects to learn, and enhances the network’s ability to extract and express the edges of focused objects. Meanwhile, given the scarcity of real multifocus training sets, we propose a randomized approach to generate massive training sets and pseudo-labels based on limited unlabeled data. Besides, two attention modules are designed based on isometric domain transformation (IDT) in the traditional edge-preservation field. IDT removes interference information from feature maps in a low-cost manner, thereby facilitating channel-wise and spatial-wise weight assignments. Experimental results on four datasets show that the performance of our model is superior to that of many supervised models, without the need of any real MFIF training set.

Multi-image-distance imaging system for extending depth-of-field

Depth of field (DoF) defines the longitudinal range of objects in sharp focus. Imaging systems for machine vision are generally limited to a small DoF by various conditions, constraining the visual inspection of objects with large depth. In this paper, we develop a multi-image-distance imaging system, which can extend the DoF by capturing multi-focus images for fusion. A three-way beam-splitter prism is designed to support independent imaging of three image sensors. Through adjusting the sensors to different back focal lengths, multiple images focusing at different depths are captured in a single exposure. Finally, the focused regions of each image are fused for extending the DoF. Analyses of the proposed DoF extension strategy illustrate excellent enhancement more than three times. Moreover, we introduce a DoF measurement method based on the estimation of the point spread function, and measurement experiments are conducted using the fabricated prototype. Imaging experiments for various scenes demonstrate great imaging quality and DoF extension performance. The proposed multi-image-distance imaging system provides an effective and practical DoF extension solution for close-range visual inspection.

Multi-focus image fusion framework based on transformer and feedback mechanism

How to efficiently and accurately identify and extract the focused regions in the source image is a difficult problem in the field of multi-focus image fusion. The existing fusion methods suffer from color distortion, loss of detail information, and high time cost, which limit the subsequent processing and real-time application of fused images. Based on this, this paper proposes a multi-focus image fusion method based on Transformer and feedback mechanism. The method uses a combination of Transformer and convolutional neural network, and integrates the local information extracted by CNN and the global information obtained by transformer, which improves the accuracy of focus region identification. In addition, this paper uses a feedback mechanism to provide more contextual information so that the features can be fully utilized, which improves the performance of the network in feature fusion. This paper conducts comparison tests with seven advanced fusion methods on the Lytro and Grayscale datasets, and the results show that the algorithm in this paper is superior in both subjective and objective evaluations.

PATN: Polarized Attention based Transformer Network for Multi-focus image fusion

PATN: Polarized Attention based Transformer Network for Multi-focus image fusion

Divide-and-conquer model based on wavelet domain for multi-focus image fusion

Multi-focus image fusion aims to generate an all-in-focus image from multiple images focused on different regions. The targets of multi-focus image fusion vary with different regions of the image, e.g., flat regions retain smoothness, while edges and textures should be sharpened. However, the existing deep learning-based multi-focus image fusion methods usually treat the image as a whole for fusion and train the model by optimizing homogenous pixel-wise loss (e.g., MSE). This leads to the trained model tends to generate flat regions that are easy to reconstruct, failing to infer realistic details. In this paper, we propose a component divide-and-conquer model for multi-focus image fusion, which uses discrete wavelet transform to decompose source images into low-frequency and high-frequency components, and feeds them into different branches to aggregate them separately by the proposed attention feature fusion network. Finally, the fused image is obtained by inverse discrete wavelet transform. Such a strategy not only can address the challenges of different difficulties in the fusion of low-frequency and high-frequency components, but also able to supervise different components flexibly by the intermediate supervision learning strategy to generate realistic details of the fused image. Extensive experiments show that the proposed component divide-and-conquer model achieves significant improvements in both quantitative and qualitative evaluation.

Mathematical modeling and simulation of multi-focus image fusion techniques using the effect of image enhancement criteria: a systematic review and performance evaluation

Mathematical modeling and simulation of multi-focus image fusion techniques using the effect of image enhancement criteria: a systematic review and performance evaluation

Multi-focus image fusion based on gradient tensor HOSVD

Multi-focus image fusion is the process of fusing images of the same scene with different focus ranges. However, pixel misclassification and artifacts caused by noise have been unavoidable problems in fusion methods. To overcome this problem, we propose gradient tensor high-order singular value decomposition (HOSVD) to fuse multi-focus images. The eight-direction Sobel gradient operator is used to obtain second-order gradients in different directions. Tensor-based information processing can effectively represent high-dimensional information and extract structural information from multi-focus images. Therefore, second-order gradient matrices of different images are formed into tensors. The gradient tensor of the images is transformed by HOSVD to obtain the core tensor and factor matrixes. The images are better represented by multiplying the core tensor with the factor matrix corresponding to the direction of the image. In the core tensor, the larger singular values represent basic information and the small singular values high-frequency information. The noise generally belongs to high-frequency information, so we select the larger singular values in the core tensor as activity level variables of the image to reduce noise. Experimental results demonstrate that the proposed fusion method can effectively reduce noise effects and achieve state-of-the-art fusion performance in both objective metrics and visual quality.

Volume monitoring of the milling tool tip wear and breakage based on multi-focus image three-dimensional reconstruction

Volume monitoring of the milling tool tip wear and breakage based on multi-focus image three-dimensional reconstruction

Empirical Wavelet Transform Method for Enhancement of Medical Image Fusion

The process of creating an image's emulsion is selecting the crucial details from numerous images and combining them into smaller images, often one bone. In the areas of satellite imaging, remote seeing, target shadowing, medical imaging, and many other areas, image emulsion is quite useful. This design tries to illustrate how Empirical Wavelet transfigures work when used with the Simple Average Emulsion Rule to emulsify multi-focus images. The suggested approach has been tested using common datasets for merging images with various focal points. Empirical Wavelet Transform is primarily a method that uses an adaptive approach to produce a Multi-Resolution Analysis of the signal. The effectiveness of the suggested approach is calculated in a variety of ways. Visual perception and the evaluation of common quality metrics, such as Root Mean Squared Error, Entropy, and Peak Signal to Noise ratio, are used to compare the performance of the proposed system. The proposed fashion based on the Empirical Wavelet Transform (EWT) outperforms the existing methods, according to the study of the experimental results. According to the suggested criteria, the fused image's entropy should be higher than the component images' because the emulsion's efficiency decreases as entropy increases. This technique takes MRI and CT scans into account.

Empirical Wavelet Transform Method for Enhancement of Medical Image Fusion

The process of creating an image's emulsion is selecting the crucial details from numerous images and combining them into smaller images, often one bone. In the areas of satellite imaging, remote seeing, target shadowing, medical imaging, and many other areas, image emulsion is quite useful. This design tries to illustrate how Empirical Wavelet transfigures work when used with the Simple Average Emulsion Rule to emulsify multi-focus images. The suggested approach has been tested using common datasets for merging images with various focal points. Empirical Wavelet Transform is primarily a method that uses an adaptive approach to produce a Multi-Resolution Analysis of the signal. The effectiveness of the suggested approach is calculated in a variety of ways. Visual perception and the evaluation of common quality metrics, such as Root Mean Squared Error, Entropy, and Peak Signal to Noise ratio, are used to compare the performance of the proposed system. The proposed fashion based on the Empirical Wavelet Transform (EWT) outperforms the existing methods, according to the study of the experimental results. According to the suggested criteria, the fused image's entropy should be higher than the component images' because the emulsion's efficiency decreases as entropy increases. This technique takes MRI and CT scans into account.

Sparse Representation-Based Multi-Focus Image Fusion Method via Local Energy in Shearlet Domain

Multi-focus image fusion plays an important role in the application of computer vision. In the process of image fusion, there may be blurring and information loss, so it is our goal to obtain high-definition and information-rich fusion images. In this paper, a novel multi-focus image fusion method via local energy and sparse representation in the shearlet domain is proposed. The source images are decomposed into low- and high-frequency sub-bands according to the shearlet transform. The low-frequency sub-bands are fused by sparse representation, and the high-frequency sub-bands are fused by local energy. The inverse shearlet transform is used to reconstruct the fused image. The Lytro dataset with 20 pairs of images is used to verify the proposed method, and 8 state-of-the-art fusion methods and 8 metrics are used for comparison. According to the experimental results, our method can generate good performance for multi-focus image fusion.

An Efficiency Correlation between Various Image Fusion Techniques

Multi-focus images can be fused by the deep learning (DL) approach. Initially, multi-focus image fusion (MFIF) is used to perform the classification task. The classifier of the convolutional neural network (CNN) is implemented to determine whether the pixel is defocused or focused. The lack of available data to train the system is one of the demerits of the MFIF methodology. Instead of using MFIF, the unsupervised model of the DL approach is affordable and appropriate for image fusion. By establishing a framework of feature extraction, fusion, and reconstruction, we generate a Deep CNN of [Formula: see text] End-to-End Unsupervised Model. It is defined as a Siamese Multi-Scale feature extraction model. It can extract only three different source images of the same scene, which is the major disadvantage of the system. Due to the possibility of low intensity and blurred images, considering only three source images may lead to poor performance. The main objective of the work is to consider [Formula: see text] parameters to define [Formula: see text] source images. Many existing systems are compared to the proposed method for extracting features from images. Experimental results of various approaches show that Enhanced Siamese Multi-Scale feature extraction used along with Structure Similarity Measure (SSIM) produces an excellent fused image. It is determined by undergoing quantitative and qualitative studies. The analysis is done based on objective examination and visual traits. By increasing the parameters, the objective assessment increases in performance rate and complexity with time.

Fine-grained multi-focus image fusion based on edge features

Multi-focus image fusion is a process of fusing multiple images of different focus areas into a total focus image, which has important application value. In view of the defects of the current fusion method in the detail information retention effect of the original image, a fusion architecture based on two stages is designed. In the training phase, combined with the polarized self-attention module and the DenseNet network structure, an encoder-decoder structure network is designed for image reconstruction tasks to enhance the original information retention ability of the model. In the fusion stage, combined with the encoded feature map, a fusion strategy based on edge feature map is designed for image fusion tasks to enhance the attention ability of detail information in the fusion process. Compared with 9 classical fusion algorithms, the proposed algorithm has achieved advanced fusion performance in both subjective and objective evaluations, and the fused image has better information retention effect on the original image.

Improved Procedure for Multi-Focus Image Quality Enhancement Using Image Fusion with Rules of Texture Energy Measures in the Hybrid Wavelet Domain

Feature extraction is a collection of the necessary detailed information from the given source, which holds the information for further analysis. The quality of the fused image depends on many parameters, particularly its directional selectivity and shift-invariance. On the other hand, the traditional wavelet-based transforms produce ringing distortions and artifacts due to poor directionality and shift invariance. The Dual-Tree Complex Wavelet Transforms (DTCWT) combined with Stationary Wavelet Transform (SWT) as a hybrid wavelet fusion algorithm overcomes the deficiencies of the traditional wavelet-based fusion algorithm and preserves the directional and shift invariance properties. The purpose of SWT is to decompose the given source image into approximate and detailed sub-bands. Further, approximate sub-bands of the given source are decomposed with DTCWT. In this extraction, low-frequency components are considered to implement Texture Energy Measures (TEM), and high-frequency components are considered to implement the absolute-maximum fusion rule. For the detailed sub-bands, the absolute-maximum fusion rule is implemented. The texture energy rules have significantly classified the image and improved the output image’s accuracy after fusion. Finally, inverse SWT is applied to generate an extended fused image. Experimental results are evaluated to show that the proposed approach outperforms approaches reported earlier. This paper proposes a fusion method based on SWT, DTCWT, and TEM to address the inherent defects of both the Parameter Adaptive-Dual Channel Pulse coupled neural network (PA-DCPCNN) and Multiscale Transform-Convolutional Sparse Representation (MST-CSR).

MCT-Net: Multi-hierarchical cross transformer for hyperspectral and multispectral image fusion

Taking into account the limitations of optical imaging, image acquisition equipment is usually designed to make a trade-off between spatial information and spectral information. Hyperspectral image(HSI) can finely identify and classify imaging objects owing to its rich spectral information, while multispectral image(MSI) can provide fine geometric features because of its sufficient spatial information. Hence, fusing HSI and MSI to achieve information complementarity has become a prevalent manner, which increases the reliability and accuracy of the information obtained. However, unlike traditional optical multi-focus image fusion and pan-sharpening of MSI, existing HSI and MSI fusion methods still face with problems in achieving cross-modality information interaction and lack effective utilization of spatial location information. To solve the above problems and to achieve more effective information integration between HSI and MSI, this paper proposes a novel multi-hierarchical cross transformer for hyperspectral and multispectral image fusion (MCT-Net). The proposed MCT-Net consists of two components: (1) a multi-hierarchical cross-modality interacting module (MCIM), which first extracts the deep multi-scale features of HSI and MSI, and then performs cross-modality information interaction between them at identical scales by applying a multi-hierarchical cross transformer (MCT), to reconstruct the spectral information lacking in MSI and the spatial information lacking in HSI; (2) a feature aggregation reconstruction module (FARM) which combines features from MCIM, uses strip convolution to further restore edge features, and reconstructs the fusion results through cascaded upsampling. We conduct comparative experiments on five mainstream HSI datasets to prove the effectiveness and superiority of the proposed method, including the Pavia Center, Pavia University, Urban, Botswana, and Washington DC Mall. For instance, on the Washington DC Mall dataset, compared with the state-of-the-art(SOTA) method in the comparison algorithms, our method improves PSNR by 18.52% and reduces RMSE, ERGAS and SAM by 56.63%, 56.90% and 58.58%, respectively. The source code for MCT-Net can be downloaded from https://github.com/wxy11-27/MCT-Net.

Robust Copyright Protection Technique with High-embedding Capacity for Color Images

Copyright violation issues have a growing impact on applications of the digital era, especially images. It is not easy to guarantee the copyright protection of essential information. This paper presents a robust copyright protection technique with high embedding capacity for color images. The technique first fuses the multi-focus images using non-subsampled contourlet transform (NSCT) to ensure that the fused image have rich information than a single image. Further, hash value of the cover media is computed for authentication purposes. The fused image and hash value of the cover is then embedded into the cover media with the help of transformed-domain schemes. To achieve higher security, we apply an encryption scheme on fused media watermark before embedding into the cover. Lastly, a hybrid optimization algorithm is employed to compute an optimal factor, which makes a good imperceptibility and robustness at the same time. We demonstrate that the proposed technique is effective and resistant to common attacks on image datasets. Compared to existing works, this work increases the 9.5% robustness, 8.8% quality with high embedding capacity.

Gradient-based multi-focus image fusion using foreground and background pattern recognition with weighted anisotropic diffusion filter

Gradient-based multi-focus image fusion using foreground and background pattern recognition with weighted anisotropic diffusion filter