Efficient Image Fusion Research Articles

Efficient Selective Image Fusion: A PCB Diagnosis Approach and Implementation

Efficient Selective Image Fusion: A PCB Diagnosis Approach and Implementation

An efficient cross-view image fusion method based on selected state space and hashing for promoting urban perception

In the field of cross-view image geolocation, traditional convolutional neural network (CNN)-based learning models generate unsatisfactory fusion performance due to their inability to model global correlations. The Transformer-based fusion methods can well compensate for the above problems, however, the Transformer has quadratic computational complexity and huge GPU memory consumption. The recent Mamba model based on the selection state space has a strong ability to model long sequences, lower GPU memory occupancy, and fewer GFLOPs. It is thus attractive and worth studying to apply Mamba to the cross-view image geolocation task. In addition, in the image-matching process (i.e., fusion of satellite/aerial and street view data.), we found that the storage occupancy of similarity measures based on floating-point features is high. Efficiently converting floating-point features into hash codes is a possible solution. In this study, we propose a cross-view image geolocation method (S6HG) based purely on Vision Mamba and hashing. S6HG fully utilizes the advantages of Vision Mamba in global information modeling and explicit location information encoding and the low storage occupancy of hash codes. Our method consists of two stages. In the first stage, we use a Siamese network based purely on vision Mamba to embed features for street view images and satellite images respectively. Our first-stage model is called S6G. In the second stage, we construct a cross-view autoencoder to further refine and compress the embedded features, and then simply map the refined features to hash codes. Comprehensive experiments show that S6G has achieved superior results on the CVACT dataset and comparable results to the most advanced methods on the CVUSA dataset. It is worth noting that other floating-point feature-based methods (4096-dimension) are 170.59 times faster than S6HG (768-bit) in storing 90,618 retrieval gallery data. Furthermore, the inference efficiency of S6G is higher than ViT-based computational methods.

An advanced heat source localization technology for intelligent warehousing: A multi-source fusion image segmentation approach leveraging infrared and visible light data

Infrared thermography technology, leveraging its unique ability to capture temperature features, has significantly improved the precision of high-temperature target localization. However, infrared imaging technology is limited by issues such as low image contrast, difficulty in distinguishing object categories, and limited image clarity. To enable intelligent detection of high-temperature objects that may cause fires in warehouses, this paper proposes an innovative method that integrates deep learning image segmentation with infrared and visible light image technology. We developed a new image segmentation model based on improved Fully Convolutional Networks and Deconvolutional Networks, introducing a batch normalization layer to accelerate convergence and employing the PReLU activation function to prevent neuron death, thereby enhancing convergence speed and accuracy. Through a feature dynamic image registration method combining a joint model and a cross-modulation strategy, we achieved efficient image fusion. In addition, a game theory-based strategy was adopted to correct localization results, ensuring accuracy. Experimental results demonstrate that the improved model achieves localization accuracy and precision rates of up to 89.30% and 88.00%, respectively, in real-world warehouse heat source scenarios, representing a significant improvement of 9.90% and 2.85% compared to the pre-improvement model, fully validating its advancement and effectiveness.

Data augmentation for deep visual recognition using superpixel based pairwise image fusion

Data augmentation is an important paradigm for boosting the generalization capability of deep learning in image classification tasks. Image augmentation using cut-and-paste strategies has shown very good performance improvement for deep learning. However, these existing methods often overlook the image’s discriminative local context and rely on ad hoc regions consisting of square or rectangular local regions, leading to the loss of complete semantic object parts. In this work, we attempt to overcome these limitations and propose a superpixel-wise local-context-aware efficient image fusion approach for data augmentation. Our approach requires only one forward propagation using a superpixel attention-based label fusion with less computational complexity. The model is trained using a combination of a global classification of the fused (augmented) image loss, a superpixel-wise weighted local classification loss, and a superpixel-based weighted contrastive learning loss. The last two losses are based on the superpixel-aware attentive embeddings. Thus, the resulting deep encoder can learn both local and global features of the images while capturing object-part local context and information. Experiments on diverse benchmark image datasets indicate that the proposed method out-performs many region-based augmentation methods for visual recognition. We have demonstrated its effectiveness not only on CNN models but also on transformer models. The codes are accessible at https://github.com/DanielaPlusPlus/SAFuse.

Efficient image fusion method using improved Bi-dimensional Empirical Mode Decomposition

ABSTRACT Several transformations are used in image fusion to extract the different image spatial details. In this paper, an improved version of the BEMD (Bi-dimensional Empirical Mode Decomposition) is proposed and is used to propose an image fusion method aiming to have the high spatial resolution of the panchromatic image PAN and the spectral resolution of the multispectral image MS in the same fused image. The proposed improved BEMD permits to avoid using injection models that are used in image fusion for preserving the spectral signatures in the fused image. Mainly, we propose a new 2D extrema points extraction for BEMD. One of the most important characteristics of the proposed BEMD components is that they are more faithful to the EMD components; their local behaviour being of pure oscillating 2D mono-components with zero mean values. Comparisons, among predecessor methods with qualitative evaluation and various spectral and spatial quantitative measures, show the effectiveness and efficiency of the proposed image fusion method. In addition, this fusion method is computationally fast and can be used to quickly merge a massive volume of data.

Generative Adversarial Network for Trimodal Medical Image Fusion Using Primitive Relationship Reasoning.

Medical image fusion has become a hot biomedical image processing technology in recent years. The technology coalesces useful information from different modal medical images onto an informative single fused image to provide reasonable and effective medical assistance. Currently, research has mainly focused on dual-modal medical image fusion, and little attention has been paid on trimodal medical image fusion, which has greater application requirements and clinical significance. For this, the study proposes an end-to-end generative adversarial network for trimodal medical image fusion. Utilizing a multi-scale squeeze and excitation reasoning attention network, the proposed method generates an energy map for each source image, facilitating efficient trimodal medical image fusion under the guidance of an energy ratio fusion strategy. To obtain the global semantic information, we introduced squeeze and excitation reasoning attention blocks and enhanced the global feature by primitive relationship reasoning. Through extensive fusion experiments, we demonstrate that our method yields superior visual results and objective evaluation metric scores compared to state-of-the-art fusion methods. Furthermore, the proposed method also obtained the best accuracy in the glioma segmentation experiment.

A Lightweight Pixel-Level Unified Image Fusion Network.

In recent years, deep-learning-based pixel-level unified image fusion methods have received more and more attention due to their practicality and robustness. However, they usually require a complex network to achieve more effective fusion, leading to high computational cost. To achieve more efficient and accurate image fusion, a lightweight pixel-level unified image fusion (L-PUIF) network is proposed. Specifically, the information refinement and measurement process are used to extract the gradient and intensity information and enhance the feature extraction capability of the network. In addition, these information are converted into weights to guide the loss function adaptively. Thus, more effective image fusion can be achieved while ensuring the lightweight of the network. Extensive experiments have been conducted on four public image fusion datasets across multimodal fusion, multifocus fusion, and multiexposure fusion. Experimental results show that L-PUIF can achieve better fusion efficiency and has a greater visual effect compared with state-of-the-art methods. In addition, the practicability of L-PUIF in high-level computer vision tasks, i.e., object detection and image segmentation, has been verified.

Multiscale Fusion of Panchromatic and Multispectral Images Based on Adaptive Iterative Filtering

This paper proposes an efficient and high-fidelity image fusion method based on adaptive smoothing filtering for panchromatic (PAN) and multispectral (MS) image fusion. The scale ratio reflects the ratio of spatial resolution between the panchromatic image and the multispectral image. When facing a multiscale fusion task, traditional methods are unable to simultaneously handle the problems of spectral resolution loss resulting from high scale ratios and the issue of reduced spatial resolution due to low scale ratios. To adapt to the fusion of panchromatic and multispectral satellite images of different scales, this paper improves the problem of the insufficient filtering of high-frequency information of remote sensing images of different scales by the classic filter-based intensity modulation (SFIM) model. It uses Gaussian convolution kernels instead of traditional mean convolution kernels and builds a Gaussian pyramid to adaptively construct convolution kernels of different scales to filter out high-frequency information of high-resolution images. It can adaptively process panchromatic multispectral images of different scales, iteratively filter the spatial information in panchromatic images, and ensure that the scale transformation is consistent with the definition of multispectral images. Using 15 common fusion methods, this paper compares the experimental results of ZY-3 with scale ratio 2.7 and SV-1 with scale ratio 4 data. The results show that the method proposed in this paper retains good spatial information for image fusion at different scales and has good spectral preservation.

Efficient and natural image fusion method for low-light images based on active contour model and adaptive gamma correction

Efficient and natural image fusion method for low-light images based on active contour model and adaptive gamma correction

Multiband Image Fusion via Regularization on a Riemannian Submanifold

Multiband image fusion aims to generate high spatial resolution hyperspectral images by combining hyperspectral, multispectral or panchromatic images. However, fusing multiband images remains a challenge due to the identifiability and tracking of the underlying subspace across varying modalities and resolutions. In this paper, an efficient multiband image fusion model is proposed to investigate the latent structures and intrinsic physical properties of a multiband image, which is characterized by the Riemannian submanifold regularization method, nonnegativity and sum-to-one constraints. An alternating minimization scheme is proposed to recover the latent structures of the subspace via the manifold alternating direction method of multipliers (MADMM). The subproblem with Riemannian submanifold regularization is tackled by the projected Riemannian trust-region method with guaranteed convergence. The effectiveness of the proposed method is demonstrated on two multiband image fusion problems: (1) hyperspectral and panchromatic image fusion and (2) hyperspectral, multispectral and panchromatic image fusion. The experimental results confirm that our method demonstrates superior fusion performance with respect to competitive state-of-the-art fusion methods.

Gaussian of Differences: A Simple and Efficient General Image Fusion Method.

The separate analysis of images obtained from a single source using different camera settings or spectral bands, whether from one or more than one sensor, is quite difficult. To solve this problem, a single image containing all of the distinctive pieces of information in each source image is generally created by combining the images, a process called image fusion. In this paper, a simple and efficient, pixel-based image fusion method is proposed that relies on weighting the edge information associated with each pixel of all of the source images proportional to the distance from their neighbors by employing a Gaussian filter. The proposed method, Gaussian of differences (GD), was evaluated using multi-modal medical images, multi-sensor visible and infrared images, multi-focus images, and multi-exposure images, and was compared to existing state-of-the-art fusion methods by utilizing objective fusion quality metrics. The parameters of the GD method are further enhanced by employing the pattern search (PS) algorithm, resulting in an adaptive optimization strategy. Extensive experiments illustrated that the proposed GD fusion method ranked better on average than others in terms of objective quality metrics and CPU time consumption.

Multimodal medical image fusion based on visual saliency map and multichannel dynamic threshold neural P systems in sub-window variance filter domain

As a powerful image enhancement technique, multimodal medical image fusion has been widely used for biomedical diagnosis and surgical navigation. However, the trade-off between efficiency and fusion quality for existing fusion images remains a great challenge. In this study, a robust and efficient medical image fusion method based on a sub-window variance filter (SVF) is proposed to overcome the above problems. First, the input images are decomposed two-layer using SVF, the base layer including rich energy and contour intensity information, and the detail layer containing integral detail features. Then, a contrast function is employed to improve the MRI base layer and a neighbor energy function to refine the detail layer. Next, a novel multichannel dynamic threshold neural P system is proposed to fuse the detail layer, which can provide a comprehensive consideration of the information in the detail layer and compensate well for the defect of the single-channel model. Moreover, a visual saliency map (VSM-based) is designed to fuse the base layer, preserving important energy information and improving image contrast. Finally, the fused result is reconstructed by the inverse SVF. Ten representative current multimodal medical image fusion methods are compared, and six typical quality evaluation metrics are combined to objectively evaluate the fused images. Tremendous experimental results indicate that the proposed model can effectively preserve higher contrast, more complete edge features, and better color than some state-of-the-art algorithms, both visual quality and quantitative evaluations on public datasets.

An Enhancement Process for Multi-focus Images Resulted from Image Fusion using qshiftN DTCWT and MPCA in Multiresolution Domain

Images having different focus values are combined using image fusion methodologies. The fused image shows superiority in the quality and highly informative than compared o any other multifocus images. The fused image would show its suitability for visual perception object detection due to its high quality than compared to multifocus images. Among the various quality parameters of the fused image, directional invariance and shift invariance significantly influence the fused image quality. The traditional image fusion methods using wavelets severely suffers with poor invariance and lack of directionality. An efficient image fusion method is developed using DTCWT with qshiftN and MPCA algorithms in the multiresolution (MR) domain. Multifocus input images are decomposed into high and low frequency components using MR algorithm. The decomposed frequency components of the input images are fused using DTCWT with qshiftN algorithm. The proposed fusion algorithm preserves both sift invariance and directional properties of the multifocus source image. Lastly, the fused image is processed using MPCA algorithm to enhance the features. The proposed methodology is assessed using various multifocus images and the evaluated metrics are contrasted with technologically advanced algorithms reported recently. The statistical metrics evaluated for the proposed method for different multifocus images shows superior properties than compared to the recently reported multifocus image fusion algorithms.

FusionGRAM: An Infrared and Visible Image Fusion Framework Based on Gradient Residual and Attention Mechanism

Infrared and visible images are fused to obtain a high-quality visible image with salient infrared targets that can be used in other vision tasks. The fusion strategy is a key to achieving efficient image fusion. The strategies currently used in deep-learning-based image fusion methods are based on manual computation and are unlearnable, limiting further improvements in infrared and visible image fusion. In this article, FusionGRAM, a novel end-to-end infrared and visible image fusion framework, is proposed. It adaptively fuses visible detailed information and infrared thermal information. The framework is fully convolutional, and the parameters of each part can be learned. The proposed method applied dense connections with an attention mechanism and gradient residual, thereby improving the network’s ability to obtain critical information during feature extraction. Furthermore, pixel intensity loss and detail loss functions were used to train FusionGRAM. Fused features were reconstructed using four convolution layers to obtain an informative fused image. Ablation experiments and comparisons on public datasets showed that FusionGRAM outperformed current state-of-the-art fusion methods in subjective and objective evaluations.

Dual-band optoelectronic poaching detection systems

Subject of study. We present details regarding the development of dual-band optoelectronic scanning systems for surveillance and detection of poachers and poaching equipment and the inclusion of image fusion and geolocation capabilities. Aim. We present research on a dual-band optoelectronic system for scanning the surface along a quasicircular trajectory that supports overlapping of frames for efficient fusion of images made from different points of view into a single image used to detect, recognize, and geolocate poaching. Methods. We present simulation and experimental study of a prototype system including television and thermal vision channels, a Global Positioning System (GPS) antenna, and inertial navigation system modules mounted on a stabilized common platform. Main results. We propose a system design that will support simultaneous scanning of a search area in television and thermal imaging channels along a quasicircular trajectory, with the capability to expand the search area and provide 30% frame overlap for efficient image fusion. Gyroscopic sensors on the stabilized common platform for the system and global navigation system antennas will support the requisite accuracy of the surveillance platform and target geolocation. The change in system viewing angle per unit time that would enable the resulting image to be obtained without missing any lines was determined. The primary components of the error in the coordinates of the surveillance platform when surveilling an object were also determined. The combination of field-of-view scanning and use of geolocation equipment supports the recognition of poachers and poaching equipment and the determination of their coordinates within a global coordinate system. An integrated high-precision GPS receiver (ProPak-V3-424) with an inertial system and data processing technology using Tightly Coupled IMU algorithms (Inertial Explorer) was found to be capable of determining the horizontal coordinates of a surveillance platform to within 12 m at a probability of 95% or better. Practical significance. A prototype of the proposed design increased the maximum reliable detection and recognition range for poachers and poaching equipment (cars and trucks) in a forest through the fusion of data obtained in the visible and infrared spectral bands.

Efficient Multi-focus Image Fusion Using Parameter Adaptive Pulse Coupled Neural Network Based Consistency Verification

Efficient Multi-focus Image Fusion Using Parameter Adaptive Pulse Coupled Neural Network Based Consistency Verification

An efficient biomedical cell image fusion method based on the multilevel low rank representation

An efficient biomedical cell image fusion method based on the multilevel low rank representation

Multimodal Classification on PET/CT Image Fusion for Lung Cancer: A Comprehensive Survey

Medical image fusion has become essential for accurate diagnosis. For example, a lung cancer diagnosis is currently conducted with the help of multimodality image fusion to find anatomical and functional information about the tumor and metabolic measurements to identify the lung cancer stage and metastatic information of the disease. Generally, the success of multimodality imaging for lung cancer diagnosis is due to the combination of PET and CT imaging advantages while minimizing their respective limitations. However, medical image fusion involves the registration of two different modalities, which is time-consuming and technically challenging, and it is a cause of concern in a clinical setting. Therefore, the paper's main objective is to identify the most efficient medical image fusion techniques and the recent advances by conducting a collective survey. In addition, the study delves into the impact of deep learning techniques for image fusion and their effectiveness in automating the image fusion procedure with better image quality while preserving essential clinical information.

Fusion of multi-modality biomedical images using deep neural networks

With the recent advancement in the medical diagnostic tools, multi-modality medical images are extensively utilized as a lifesaving tool. An efficient fusion of medical images can improve the performance of various medical diagnostic tools. But, gathering of all modalities for a given patient is defined as an ill-posed problem as medical images suffer from poor visibility and frequent patient dropout. Therefore, in this paper, an efficient multi-modality image fusion model is proposed to fuse multi-modality medical images. To tune the hyper-parameters of the proposed model, a multi-objective differential evolution is used. The fusion factor and edge strength metrics are utilized to form a multi-objective fitness function. Performance of the proposed model is compared with nine competitive models over fifteen benchmark images. Performance analyses reveal that the proposed model outperforms the competitive fusion models.

Fusion of Infrared and Visible Images Using Fast Global Smoothing Decomposition and Target-Enhanced Parallel Gaussian Fuzzy Logic.

As a powerful technique to merge complementary information of original images, infrared (IR) and visible image fusion approaches are widely used in surveillance, target detecting, tracking, and biological recognition, etc. In this paper, an efficient IR and visible image fusion method is proposed to simultaneously enhance the significant targets/regions in all source images and preserve rich background details in visible images. The multi-scale representation based on the fast global smoother is firstly used to decompose source images into the base and detail layers, aiming to extract the salient structure information and suppress the halos around the edges. Then, a target-enhanced parallel Gaussian fuzzy logic-based fusion rule is proposed to merge the base layers, which can avoid the brightness loss and highlight significant targets/regions. In addition, the visual saliency map-based fusion rule is designed to merge the detail layers with the purpose of obtaining rich details. Finally, the fused image is reconstructed. Extensive experiments are conducted on 21 image pairs and a Nato-camp sequence (32 image pairs) to verify the effectiveness and superiority of the proposed method. Compared with several state-of-the-art methods, experimental results demonstrate that the proposed method can achieve more competitive or superior performances according to both the visual results and objective evaluation.