Image Fusion Algorithm Research Articles

An image fusion algorithm based on image clustering theory

An image fusion algorithm based on image clustering theory

Multi-spectral Remote Sensing Image Fusion Method Based on Gradient Moment Matching

Image fusion is a popular research direction in the field of computer vision. Traditional image fusion algorithms can achieve good results in fusing grayscale images, but it is difficult to achieve ideal results in processing multi-spectral images. To address the shortcomings of multi-spectral image fusion, this study proposes a low computational complexity and low latency multi-spectral image fusion model by utilizing a multi-step degree moment matching algorithm and a generative adversarial network for fusion. Through experiments, it was found that the F1 score of the GAN-MMN model on the TinyPerson dataset was 89.79%, with an average recall rate of 89.76%. The GAN-MMN performance was higher than that of the control model. Meanwhile, the GAN-MMN model also exhibited superior performance in high-frequency feature extraction and time delay compared to the control model. According to the experimental results, the proposed multi-spectral remote sensing image fusion model had a high feature extraction effect, and its recall rate and F1 score were better than the control model, so the research model had a certain progressiveness. The proposal of this model gives a new approach for the processing of multi-spectral remote sensing images, effectively promoting the development of the computer vision industry.

An Image Fusion Algorithm for Sustainable Development Goals Satellite-1 Night-Time Light Images Based on Optimized Image Stretching and Dual-Domain Fusion

The Glimmer Imager of Urbanization (GIU) on SDGSAT-1 provides high-resolution and global-coverage images of night-time lights (NLs) with 10 m panchromatic (PAN) and 40 m multispectral (MS) imagery. High-resolution 10 m MS NL images after ideal fusion can be used to better study subtle manifestations of human activities. Most existing remote sensing image-fusion methods are based on the fusion of daytime optical remote sensing images, which do not apply to lossless compressed images of the GIU. To address this limitation, we propose a novel approach for 10 m NL data fusion, namely, a GIU NL image fusion model based on PAN-optimized OIS (OIS) and DDF (DDF) fusion for SDGSAT-1 high-resolution products. The OIS of PAN refers to the optimized stretching method that integrates linear and gamma stretching while DDF indicates a fusion process that separately merges the dark and light regions of NL images using different fusion methods, then stitches them together. In this study, fusion experiments were conducted in four study areas—Beijing, Shanghai, Moscow, and New York—and the proposed method was compared to traditional methods using visual evaluation and five quantitative evaluation metrics. The results demonstrate that the proposed method achieves superior visual quality and outperforms conventional methods across all quantitative metrics. Additionally, the ablation study confirmed the necessity of the methodological steps employed in this study.

SIFusion: Lightweight infrared and visible image fusion based on semantic injection.

The objective of image fusion is to integrate complementary features from source images to better cater to the needs of human and machine vision. However, existing image fusion algorithms predominantly focus on enhancing the visual appeal of the fused image for human perception, often neglecting their impact on subsequent high-level visual tasks, particularly the processing of semantic information. Moreover, these fusion methods that incorporate downstream tasks tend to be overly complex and computationally intensive, which is not conducive to practical applications. To address these issues, a lightweight infrared and visible light image fusion method known as SIFusion, which is based on semantic injection, is proposed in this paper. This method employs a semantic-aware branch to extract semantic feature information, and then integrates these features into the fused features through a Semantic Injection Module (SIM) to meet the semantic requirements of high-level visual tasks. Furthermore, to simplify the complexity of the fusion network, this method introduces an Edge Convolution Module (ECB) based on structural reparameterization technology to enhance the representational capacity of the encoder and decoder. Extensive experimental comparisons demonstrate that the proposed method performs excellently in terms of visual appeal and advanced semantics, providing satisfactory fusion results for subsequent high-level visual tasks even in challenging scenarios.

CMIFDF: A lightweight cross-modal image fusion and weight-sharing object detection network framework

In today’s research, unimodal target detection can no longer meet the needs of target detection in complex backgrounds as well as harsh environments. To solve the problems of the existing cross-modal image fusion and cross-modal image target detection algorithms with network-heavy parameters and redundant network design, a selectable cross-modal image fusion and target detection algorithm framework (CMIFDF) is proposed. The framework consists of a lightweight dual-branch cross-modal image fusion network (LDFnet) and a cross-modal object detection algorithm with shareable weights (CM-YOLO) to rationally utilize the cross-modal image information and improve the performance of target detection under complex backgrounds. LDFnet is a two-branch fusion module based on depth-separable convolutional and attentional mechanisms. It can quickly and fully extract feature information from visible and infrared images. In CM-YOLO, fused images or raw images (visible and infrared) are fed into a target detection network with shareable weights for training and detection. A simplified asymptotic feature pyramid network (SAFPN) is proposed, and a lightweight multilayer perceptual attention module (LMA) is designed to enhance the fusion efficiency of the fusion network, so that efficient fusion of features can be achieved with fewer model parameters and low dissipation power to improve the network detection performance. Experiments on publicly available datasets show that the algorithmic framework can make full use of the feature information of cross-modal images as inputs and can effectively improve detection performance in complex environments.

Multi-Focus Image Fusion Algorithm Based on Multi-Task Learning and PS-ViT

Multi-Focus Image Fusion Algorithm Based on Multi-Task Learning and PS-ViT

Advances in the diagnosis of prostate cancer based on image fusion

Image fusion currently plays an important role in the diagnosis of prostate cancer (PCa). Selecting and developing a good image fusion algorithm is the core task of achieving image fusion, which determines whether the fusion image obtained is of good quality and can meet the actual needs of clinical application. In recent years, it has become one of the research hotspots of medical image fusion. In order to make a comprehensive study on the methods of medical image fusion, this paper reviewed the relevant literature published at home and abroad in recent years. Image fusion technologies were classified, and image fusion algorithms were divided into traditional fusion algorithms and deep learning (DL) fusion algorithms. The principles and workflow of some algorithms were analyzed and compared, their advantages and disadvantages were summarized, and relevant medical image data sets were introduced. Finally, the future development trend of medical image fusion algorithm was prospected, and the development direction of medical image fusion technology for the diagnosis of prostate cancer and other major diseases was pointed out.

Traumatic Brain Injury Structure Detection Using Advanced Wavelet Transformation Fusion Algorithm with Proposed CNN-ViT

Detecting Traumatic Brain Injuries (TBI) through imaging remains challenging due to limited sensitivity in current methods. This study addresses the gap by proposing a novel approach integrating deep-learning algorithms and advanced image-fusion techniques to enhance detection accuracy. The method combines contextual and visual models to effectively assess injury status. Using a dataset of repeat mild TBI (mTBI) cases, we compared various image-fusion algorithms: PCA (89.5%), SWT (89.69%), DCT (89.08%), HIS (83.3%), and averaging (80.99%). Our proposed hybrid model achieved a significantly higher accuracy of 98.78%, demonstrating superior performance. Metrics including Dice coefficient (98%), sensitivity (97%), and specificity (98%) verified that the strategy is efficient in improving image quality and feature extraction. Additional validations with “entropy”, “average pixel intensity”, “standard deviation”, “correlation coefficient”, and “edge similarity measure” confirmed the robustness of the fused images. The hybrid CNN-ViT model, integrating curvelet transform features, was trained and validated on a comprehensive dataset of 24 types of brain injuries. The overall accuracy was 99.8%, with precision, recall, and F1-score of 99.8%. The “average PSNR” was 39.0 dB, “SSIM” was 0.99, and MI was 1.0. Cross-validation across five folds proved the model’s “dependability” and “generalizability”. In conclusion, this study introduces a promising method for TBI detection, leveraging advanced image-fusion and deep-learning techniques, significantly enhancing medical imaging and diagnostic capabilities for brain injuries.

Research on capacitance-coupled electrical impedance imaging fusion algorithm based on wavelet transform.

Capacitively Coupled Electrical Impedance Tomography (CCEIT), as a new electrical tomography method, effectively overcomes the drawbacks of traditional electrical resistance tomography that requires direct contact with liquids, so as to achieve non-contact measurement. In order to make full use of the complete electrical impedance information (real and imaginary) obtained by CCEIT and further improve the quality of image reconstruction, an image fusion algorithm based on wavelet transform was proposed. First, the Landweber algorithm is used to image the real part information and the imaginary part information, and the wavelet transform algorithm is used to decompose the two images at multiple scales, in which the low-frequency coefficient is fused by the improved logic filter algorithm based on the adaptive threshold, the high-frequency coefficient is selected based on the absolute value maximum method for fusion, and finally, the fused image is obtained by wavelet reconstruction. Numerical simulations and experiments verify the feasibility of the proposed method, and the results show that the relative error is reduced by up to 17% and the correlation coefficient is increased by up to 10%.

Multi-Focus Image Fusion Using Energy Valley Optimization Algorithm

When a natural scene is photographed using imaging sensors commonly used today, part of the image is obtained sharply while the other part is obtained blurry. This problem is called limited depth of field. This problem can be solved by fusing the sharper parts of multi-focus images of the same scene. These methods are called multi-focus image fusion methods. This study proposes a block-based multi-focus image fusion method using the Energy Valley Optimization Algorithm (EVOA), which has been introduced in recent years. In the proposed method, the source images are first divided into uniform blocks, and then the sharper blocks are determined using the criterion function. By fusing these blocks, a fused image is obtained. EVOA is used to optimize the block size. The function that maximizes the quality of the fused image is used as the fitness function of the EVOA. The proposed method has been applied to commonly used image sets. The obtained experimental results are compared with the well-known Genetic Algorithm (GA), Differential Evolution Algorithm (DE), and Artificial Bee Colony Optimization Algorithm (ABC). The experimental results show that EVOA can compete with the other block-based multi-focus image fusion algorithms.

Spatial correction and Deblurring fusion algorithm for vehicle license plate images based on deep learning

Spatial correction and Deblurring fusion algorithm for vehicle license plate images based on deep learning

Multi-Source Image Fusion Based on BEMD and Region Sharpness Guidance Region Overlapping Algorithm

Multi-focal image and multi-modal image fusion technology can fully take advantage of different sensors or different times, retaining the image feature information and improving the image quality. A multi-source image fusion algorithm based on bidimensional empirical mode decomposition (BEMD) and a region sharpness-guided region overlapping algorithm are studied in this article. Firstly, source images are decomposed into multi-layer bidimensional intrinsic mode functions (BIMFs) and residuals from high-frequency layer to low-frequency layer by BEMD. Gaussian bidimensional intrinsic mode functions (GBIMFs) are obtained by applying Gaussian filtering operated on BIMF and calculating the sharpness value of segmented regions using an improved weighted operator based on the Tenengrad function, which is the key to comparison selection and fusion. Then, the GBIMFs and residuals selected by sharpness comparison strategy are fused by the region overlapping method, and the stacked layers are weighted to construct the final fusion image. Finally, based on qualitative evaluation and quantitative evaluation indicators, the proposed algorithm is compared with six typical image fusion algorithms. The comparison results show that the proposed algorithm can effectively capture the feature information of images in different states and reduce the redundant information.

A Method for Sorting High-Quality Fresh Sichuan Pepper Based on a Multi-Domain Multi-Scale Feature Fusion Algorithm.

Post-harvest selection of high-quality Sichuan pepper is a critical step in the production process. To achieve this, a visual system needs to analyze Sichuan pepper with varying postures and maturity levels. To quickly and accurately sort high-quality fresh Sichuan pepper, this study proposes a multi-scale frequency domain feature fusion module (MSF3M) and a multi-scale dual-domain feature fusion module (MS-DFFM) to construct a multi-scale, multi-domain fusion algorithm for feature fusion of Sichuan pepper images. The MultiDomain YOLOv8 Model network is then built to segment and classify the target Sichuan pepper, distinguishing the maturity level of individual Sichuan peppercorns. A selection method based on the average local pixel value difference is proposed for sorting high-quality fresh Sichuan pepper. Experimental results show that the MultiDomain YOLOv8-seg achieves an mAP50 of 88.8% for the segmentation of fresh Sichuan pepper, with a model size of only 5.84 MB. The MultiDomain YOLOv8-cls excels in Sichuan pepper maturity classification, with an accuracy of 98.34%. Compared to the YOLOv8 baseline model, the MultiDomain YOLOv8 model offers higher accuracy and a more lightweight structure, making it highly effective in reducing misjudgments and enhancing post-harvest processing efficiency in agricultural applications, ultimately increasing producer profits.

DDBFusion: An unified image decomposition and fusion framework based on dual decomposition and Bézier curves

Existing image fusion algorithms mostly concentrate on the design of network architecture and loss functions, and using unified feature extraction strategies while neglecting the division of redundant and effective information. However, for complementary information, unified feature extractor may not appropriate. Thus, this paper presents a unified image fusion algorithm based on Bézier curves image augmentation and hierarchical decomposition, and a self-supervised learning task is constructed to learn the meaningful information. Where Bézier curves aim to simulate different image features and constructed special self-supervised learning samples, so our method does not require task specific data and can be easily trained on public natural image datasets. Meanwhile, our dual decomposition self-supervised training method can bring redundant information filtering capability to the model. During the decomposition stage, we classify and extract different features of the images and only utilize the extracted effective information in the fusion stage, and the decomposition ability of images provides a foundation for advanced visual tasks, such as image segmentation and object detection. Finally, more detailed and comprehensive fusion images are generated, and the existence of redundant information is effectively reduced. The validity of the proposed method is verified through qualitative and quantitative analysis of multiple image fusion tasks, and our algorithm gets the state-of-the-art results on multiple datasets of different image fusion tasks. The code of our fusion method is available at https://github.com/Yukarizz/DDBFusion.

Research on fractional wavelet transform combined with parameter adaptive PCNN for infrared and visible image fusion algorithm

Infrared and visible image fusion holds significant value across various fields due to its ability to provide complementary information. However, existing fusion algorithms often suffer from susceptibility to external physical conditions of source images, leading to inconsistent fusion quality and limited adaptive capability. In this paper, we propose a fractional wavelet transform fusion algorithm for infrared and visible images. This algorithm uses fractional wavelet domain image decomposition to more accurately locate the image structures of different scales. Initially, we perform image decomposition using a non-subsampled shearlet transform (NSST) with multi-scale and multi-directional decomposition. Subsequently, to effectively fuse detailed information, we employ discrete fractional wavelet transform (DFRWT) to decompose low-frequency subbands while selecting an appropriate p-value. High-frequency subbands are fused using a parameterized adaptive pulse-coupled neural network (PCNN) model. To validate the superiority of our algorithm, we conduct visual and quantitative comparative analyses against several existing algorithms. The results confirm that our proposed algorithm exhibits robustness against external physical conditions and surpasses these existing algorithms, making it highly applicable for infrared–visible fusion tasks.

Improved Unsupervised Stitching Algorithm for Multiple Environments SuperUDIS.

Large field-of-view images are increasingly used in various environments today, and image stitching technology can make up for the limited field of view caused by hardware design. However, previous methods are constrained in various environments. In this paper, we propose a method that combines the powerful feature extraction capabilities of the Superpoint algorithm and the exact feature matching capabilities of the Lightglue algorithm with the image fusion algorithm of Unsupervised Deep Image Stitching (UDIS). Our proposed method effectively improves the situation where the linear structure is distorted and the resolution is low in the stitching results of the UDIS algorithm. On this basis, we make up for the shortcomings of the UDIS fusion algorithm. For stitching fractures of UDIS in some complex situations, we optimize the loss function of UDIS. We use a second-order differential Laplacian operator to replace the difference in the horizontal and vertical directions to emphasize the continuity of the structural edges during training. Combined with the above improvements, the Super Unsupervised Deep Image Stitching (SuperUDIS) algorithm is finally formed. SuperUDIS has better performance in both qualitative and quantitative evaluations compared to the UDIS algorithm, with the PSNR index increasing by 0.5 on average and the SSIM index increasing by 0.02 on average. Moreover, the proposed method is more robust in complex environments with large color differences or multi-linear structures.

DGFusion: An effective dynamic generalizable network for infrared and visible image fusion

The objective of infrared and visible image fusion is to generate a unified image that highlights prominent targets and retains intricate texture details, even in scenarios with imbalanced source image information. However, current image fusion algorithms primarily consider factors like illumination, restricting their applicability to certain scenes and compromising their adaptability. To tackle the issue, this paper proposes the DGFusion, which utilizes TWSSLoss to balance the contribution of source images in the fused output, effectively mitigating the limitations linked to relying solely on illumination guidance. Additionally, modality-complement feature attention harmonizer (MCFAH) facilitates cross-modal channel attention learning. This process assigns weights to features and accomplishes fusion by exchanging cross-modal differential information, thereby enriching each feature with details from the other modality. Furthermore, the multi convolution attentive net (MCAN) dynamically adjusts the contributions of features from different modalities. It prioritizes the most expressive characteristics to accentuate complementary information, enabling efficient fusion. In conclusion, our method outperforms seven state-of-the-art alternatives in terms of preserving target details and retaining texture information. Rigorous generalization experiments across five diverse datasets demonstrate the robustness of our DGFusion model in various scenarios.

Infrared remote-sensing image super-resolution based on physical characteristic deduction

In this paper, we propose a high-confidence, super-resolution method for short-wave infrared remote-sensing images based on characteristic deduction form high-resolution visible images. First, based on the radiative transmission mechanism of full remote-sensing imaging link, a ground object characterization model based on the dark object method was established to achieve the inversion of the object’s ground reflection characteristics of high-resolution visible images. Second, the reflectance information of the specified infrared band can be deduced combined with the reflectance matching of the typical ground object spectral library. On this basis, combined with the characterization modeling of remote-sensing imaging link—such as atmosphere, optical system, platform, and detector—a high-resolution infrared remote-sensing image reconstruction method is proposed. Based on the imaging degradation model between high- and low-resolution infrared images, the results of high-resolution reconstruction were corrected using measured infrared images. Lastly, the final fusion-super-resolution image can be obtained. The results show that the proposed fusion-super-resolution method yields high-resolution infrared images more realistically than traditional image fusion algorithms and shows better structural similarity index and visual information fidelity results, verifying the effectiveness of the proposed infrared remote-sensing image fusion-super-resolution method.

Optical design and development of an underwater dual-channel microlens array integral field snapshot hyperspectral imager

Compared to push-scan hyperspectral imagers, snapshot hyperspectral imagers offer an advantage by minimizing sensitivity to attitude jitter in underwater mobile platforms. Here we present the optical design and development of an underwater microlens array integral field hyperspectral imager. The system comprises a panchromatic imaging channel with a high spatial resolution and a spectral imaging channel with a lower spatial resolution. Through the fusion of high-resolution panchromatic images and low-resolution spectral images, we achieve high spatial resolution hyperspectral images. Both the panchromatic imaging channel and the spectral imaging channel share a common front objective, featuring a 25 mm focal length and a wide 36° field of view angle. Utilizing prism dispersion, the spectral imaging system spans a band range from 465 to 700 nm with a spectral resolution of less than 10 nm. Specialized algorithms for spectral image reconstruction and image fusion have been developed. The experimental results across diverse scenes confirm the exemplary spectral imaging performance of the system, positioning it as a robust solution for underwater snapshot hyperspectral imaging.

CFNet: An infrared and visible image compression fusion network

Image fusion aims to acquire a more complete image representation within a limited physical space to more effectively support practical vision applications. Although the currently popular infrared and visible image fusion algorithms take practical applications into consideration. However, they did not fully consider the redundancy and transmission efficiency of image data. To address this limitation, this paper proposes a compression fusion network for infrared and visible images based on joint CNN and Transformer, termed CFNet. First of all, the idea of variational autoencoder image compression is introduced into the image fusion framework, achieving data compression while maintaining image fusion quality and reducing redundancy. Moreover, a joint CNN and Transformer network structure is proposed, which comprehensively considers the local information extracted by CNN and the global long-distance dependencies emphasized by Transformer. Finally, multi-channel loss based on region of interest is used to guide network training. Not only can color visible and infrared images be fused directly but more bits can be allocated to the foreground region of interest, resulting in a superior compression ratio. Extensive qualitative and quantitative analyses affirm that the proposed compression fusion algorithm achieves state-of-the-art performance. In particular, rate–distortion performance experiments demonstrate the great advantages of the proposed algorithm for data storage and transmission. The source code is available at https://github.com/Xiaoxing0503/CFNet.