Medical Image Fusion Algorithm Research Articles

Advances in the diagnosis of prostate cancer based on image fusion

Image fusion currently plays an important role in the diagnosis of prostate cancer (PCa). Selecting and developing a good image fusion algorithm is the core task of achieving image fusion, which determines whether the fusion image obtained is of good quality and can meet the actual needs of clinical application. In recent years, it has become one of the research hotspots of medical image fusion. In order to make a comprehensive study on the methods of medical image fusion, this paper reviewed the relevant literature published at home and abroad in recent years. Image fusion technologies were classified, and image fusion algorithms were divided into traditional fusion algorithms and deep learning (DL) fusion algorithms. The principles and workflow of some algorithms were analyzed and compared, their advantages and disadvantages were summarized, and relevant medical image data sets were introduced. Finally, the future development trend of medical image fusion algorithm was prospected, and the development direction of medical image fusion technology for the diagnosis of prostate cancer and other major diseases was pointed out.

FATFusion: A functional–anatomical transformer for medical image fusion

Medical image fusion (MIF) strives to obtain an informative fused image by integrating complementary information, e.g., functional metabolic properties and anatomical soft tissues, to facilitate disease diagnosis and treatment planning. To solve the problems of existing MIF algorithms, including inconvenient and manually designed transformation methods, complicated fusion strategies, global complementary information loss, and insufficient consideration of the modality-specific intrinsic characteristics, in this paper, we innovatively propose a Functional–Anatomical Transformer for MIF named FATFusion. In the proposed method, a functional multiscale branch (FMB) and an anatomical multiscale branch (AMB) are designed to extract interscale characteristics belonging to each modality. In addition, functional-guided transformer modules (FGTMs) and anatomical-guided transformer modules (AGTMs) are elaborated to communicate and gather the perceived features for further global complementary information interaction and aggregation. Moreover, the pixel loss and total variation loss are designed to train FATFusion in an end-to-end and unsupervised manner, guiding the deep fusion model to leverage more significant properties. Promising quantitative and qualitative assessments illustrate that FATFusion surpasses alternative state-of-the-art MIF methodologies, excelling in both objective evaluation and perceptible observation. Furthermore, successful generalization results reveal that FATFusion has impressive generalization ability. The source code of the proposed method is available at https://github.com/tthinking/FATFusion.

Multi-modal medical image fusion via multi-dictionary and truncated Huber filtering

Multi-modal medical image fusion provides comprehensive and objective descriptions of lesions for clinical medical assistance. However, retaining useful information while achieving noise robustness remains challenging for existing techniques. In this paper, we propose a novel medical image fusion algorithm based on multi-dictionary convolutional sparse representation. Especially, truncated Huber filtering is first introduced to achieve detail-base layer decomposition of source images. Subsequently, multiple-dictionary decisions and nuclear energy-based rules are proposed to fuse the details and base layers, respectively. The fused image is reconstructed by synthesizing the fused detail and base components. The proposed model effectively fuses the source global structure and texture information and exhibits strong robustness against noise. Experiments involving extensive noise-free and noisy anatomical and functional medical image fusion on a public dataset covering five fusion categories demonstrate that the proposed method outperforms other state-of-the-art methods in subjective and objective evaluations. The source code of this study is publicly available at https://github.com/JEI981214/MDHU-fusion.

Performance comparison of different medical image fusion algorithms for clinical glioma grade classification with advanced magnetic resonance imaging (MRI)

Non-invasive glioma grade classification is an exciting area in neuroimaging. The primary purpose of this study is to investigate the performance of different medical image fusion algorithms for glioma grading purposes by fusing advanced Magnetic Resonance Imaging (MRI) images. Ninety-six subjects underwent an Apparent diffusion coefficient (ADC) map and Susceptibility-weighted imaging (SWI) MRI scan. After preprocessing, the different medical image fusion methods used to fuse ADC maps and SWI were Principal Component Analysis (PCA), Structure-Aware, Discrete Cosine Harmonic Wavelet Transform (DCHWT), Deep-Convolutional Neural network (DNN), Dual-Discriminator conditional generative adversarial network (DDcGAN), and Laplacian Re-Decomposition (LRD). The Entropy, standard deviation (STD), peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and Relative Signal Contrast (RSC) were calculated for qualitative and quantitative analysis. We found high fused image quality with LRD and DDcGAN methods. Further quantitative analysis showed that RSCs in fused images in Low-Grade glioma (LGG) were significantly higher than RSCs in High-Grade glioma (HGG) with PCA, DCHWT, LRD, and DDcGAN. The Receiver Operating Characteristic (ROC) curve test highlighted that LRD and DDcGAN have the highest performance for glioma grade classification. Our work suggests using the DDcGAN and LRD networks for glioma grade classification by fusing ADC maps and SWI images.

A three-layer decomposition method based on structural texture perception for fusion of CT and MRI images

The fusion of computed tomography (CT) images and magnetic resonance imaging (MRI) images has made essential contribution to clinical diagnosis because of its ability of complementary visual information enhancement and redundancy elimination. However, some methods that involve upsampling and downsampling operations may lose information during image processing, which affects the fusion results. In this paper, a medical image fusion algorithm based on structural texture perception is proposed. The proposed method demonstrates enhanced performance in preserving both detailed texture information and energy information of source images. First, the source image is decomposed into a signal strength layer, a base layer, and a texture layer based on the proposed three-layer decomposition framework. Then, the signal strength layers are fused using a saliency detection method based on iterative least squares. The texture layer is fused using the principle of spatial frequency maximization. For the fusion of the base layer, a non-linear function is designed to calculate the fusion weights. Finally, the final fusion result is obtained through image reconstruction. The proposed algorithm is compared with nine state-of-the-art fusion algorithms to verify its superiority. Experimental results show that the proposed method can effectively preserve the intensity information of CT images and the detailed texture of MRI images.

SW-UNet: a U-Net fusing sliding window transformer block with CNN for segmentation of lung nodules.

Medical images are information carriers that visually reflect and record the anatomical structure of the human body, and play an important role in clinical diagnosis, teaching and research, etc. Modern medicine has become increasingly inseparable from the intelligent processing of medical images. In recent years, there have been more and more attempts to apply deep learning theory to medical image segmentation tasks, and it is imperative to explore a simple and efficient deep learning algorithm for medical image segmentation. In this paper, we investigate the segmentation of lung nodule images. We address the above-mentioned problems of medical image segmentation algorithms and conduct research on medical image fusion algorithms based on a hybrid channel-space attention mechanism and medical image segmentation algorithms with a hybrid architecture of Convolutional Neural Networks (CNN) and Visual Transformer. To the problem that medical image segmentation algorithms are difficult to capture long-range feature dependencies, this paper proposes a medical image segmentation model SW-UNet based on a hybrid CNN and Vision Transformer (ViT) framework. Self-attention mechanism and sliding window design of Visual Transformer are used to capture global feature associations and break the perceptual field limitation of convolutional operations due to inductive bias. At the same time, a widened self-attentive vector is used to streamline the number of modules and compress the model size so as to fit the characteristics of a small amount of medical data, which makes the model easy to be overfitted. Experiments on the LUNA16 lung nodule image dataset validate the algorithm and show that the proposed network can achieve efficient medical image segmentation on a lightweight scale. In addition, to validate the migratability of the model, we performed additional validation on other tumor datasets with desirable results. Our research addresses the crucial need for improved medical image segmentation algorithms. By introducing the SW-UNet model, which combines CNN and ViT, we successfully capture long-range feature dependencies and break the perceptual field limitations of traditional convolutional operations. This approach not only enhances the efficiency of medical image segmentation but also maintains model scalability and adaptability to small medical datasets. The positive outcomes on various tumor datasets emphasize the potential migratability and broad applicability of our proposed model in the field of medical image analysis.

MBRARN: multibranch residual attention reconstruction network for medical image fusion.

Medical image fusion aims to integrate complementary information from multimodal medical images and has been widely applied in the field of medicine, such as clinical diagnosis, pathology analysis, and healing examinations. For the fusion task, feature extraction is a crucial step. To obtain significant information embedded in medical images, many deep learning-based algorithms have been proposed recently and achieved good fusion results. However, most of them can hardly capture the independent and underlying features, which leads to unsatisfactory fusion results. To address these issues, a multibranch residual attention reconstruction network (MBRARN) is proposed for the medical image fusion task. The proposed network mainly consists of three parts: feature extraction, feature fusion, and feature reconstruction. Firstly, the input medical images are converted into three scales by image pyramid operation and then are input into three branches of the proposed network respectively. The purpose of this procedure is to capture the local detailed information and the global structural information. Then, convolutions with residual attention modules are designed, which can not only enhance the captured outstanding features, but also make the network converge fast and stably. Finally, feature fusion is performed with the designed fusion strategy. In this step, a new more effective fusion strategy is correspondently designed for MRI-SPECT based on the Euclidean norm, called feature distance ratio (FDR). The experimental results conducted on Harvard whole brain atlas dataset demonstrate that the proposed network can achieve better results in terms of both subjective and objective evaluation, compared with some state-of-the-art medical image fusion algorithms.

Deep Transfer Learning-Based Multi-Modal Digital Twins for Enhancement and Diagnostic Analysis of Brain MRI Image.

it aims to adopt deep transfer learning combined with Digital Twins (DTs) in Magnetic Resonance Imaging (MRI) medical image enhancement. MRI image enhancement method based on metamaterial composite technology is proposed by analyzing the application status of DTs in medical direction and the principle of MRI imaging. On the basis of deep transfer learning, MRI super-resolution deep neural network structure is established. To address the problem that different medical imaging methods have advantages and disadvantages, a multi-mode medical image fusion algorithm based on adaptive decomposition is proposed and verified by experiments. the optimal Peak Signal to Noise Ratio (PSNR) of 34.11dB can be obtained by introducing modified linear element and loss function of deep transfer learning neural network structure. The Structural Similarity Coefficient (SSIM) is 85.24%. It indicates that the MRI truthfulness and sharpness obtained by adding composite metasurface are improved greatly. The proposed medical image fusion algorithm has the highest overall score in the subjective evaluation of the six groups of fusion image results. Group III had the highest score in Magnetic Resonance Imaging- Positron Emission Computed Tomography (MRI-PET) image fusion, with a score of 4.67, close to the full score of 5. As for the objective evaluation in group I of Magnetic Resonance Imaging- Single Photon Emission Computed Tomography (MRI-SPECT) images, the Root Mean Square Error (RMSE), Relative Average Spectral Error (RASE) and Spectral Angle Mapper (SAM) are the highest, which are 39.2075, 116.688, and 0.594, respectively. Mutual Information (MI) is 5.8822. the proposed algorithm has better performance than other algorithms in preserving spatial details of MRI images and color information direction of SPECT images, and the other five groups have achieved similar results.

Medical image fusion using bilateral texture filtering

Many medical image fusion algorithms have been developed with the aim at enriching visual information for clinical diagnosis. However, these algorithms always suffer from texture loss, low contrast and pseudo-edges, which may result in the misdiagnosis in clinical applications. In order to solve these problems, a medical image fusion algorithm based on structural similarity detection (SSD), saliency detection and bilateral texture filter (BTF) is proposed. Our method enables the decomposition of source images into base and detail layers in the BTF scheme. For the base layers, a cluster-contrast based fusion rule using saliency detection is designed to preserve structure information. For the detail layers, we use the SSD to obtain the structural similarity part and dissimilarity part, respectively. An improved contrast-based saliency estimation method (CSE) is presented to fuse dissimilar textures; while the weighted least square optimization scheme (WLSO) is adopted to fuse similar textures. Finally, the fused image is obtained by performing reconstruction. The algorithm is compared with 17 state-of-the-art algorithms subjectively and objectively, and the experimental results have shown that the proposed method outperforms the comparative methods.

CDRNet: Cascaded dense residual network for grayscale and pseudocolor medical image fusion

ObjectiveMultimodal medical fusion images have been widely used in clinical medicine, computer-aided diagnosis and other fields. However, the existing multimodal medical image fusion algorithms generally have shortcomings such as complex calculations, blurred details and poor adaptability. To solve this problem, we propose a cascaded dense residual network and use it for grayscale and pseudocolor medical image fusion. MethodsThe cascaded dense residual network uses a multiscale dense network and a residual network as the basic network architecture, and a multilevel converged network is obtained through cascade. The cascaded dense residual network contains 3 networks, the first-level network inputs two images with different modalities to obtain a fused Image 1, the second-level network uses fused Image 1 as the input image to obtain fused Image 2 and the third-level network uses fused Image 2 as the input image to obtain fused Image 3. The multimodal medical image is trained through each level of the network, and the output fusion image is enhanced step-by-step. ResultsAs the number of networks increases, the fusion image becomes increasingly clearer. Through numerous fusion experiments, the fused images of the proposed algorithm have higher edge strength, richer details, and better performance in the objective indicators than the reference algorithms. ConclusionCompared with the reference algorithms, the proposed algorithm has better original information, higher edge strength, richer details and an improvement of the four objective SF, AG, MZ and EN indicator metrics.

MULTI-MODAL MEDICAL IMAGE FUSION BASED ON MULTI-BASED BINARY RESIDUALFEATURE FUSION

The term "medical image fusion" refers to the procedure of aligning and blending separate pictures captured using different imaging technologies. By enhancing imaging quality and decreasing unpredictability and redundancy, it aims to boost the clinical usefulness of medical pictures for the diagnosis and evaluation of medical conditions. Accuracy of clinical choices based on medical pictures has been enhanced via the use of multi model medical image fusion algorithms and tools. In this study, we show that combining pictures from chest X-rays and ultrasounds improve visibility. On the other hand, the merged picture is essential for a high-quality display. Improving the quality of the combined photos requires a classification technique. The accurate fusion and integration of medical images from multiple modalities is a critical step in the diagnostic process that leads to clinical activities and the appropriate treatment of patients, and this chapter suggests a Multi-based Binary Residual Feature Fusion (MBRFF) method for this purpose. The images are preprocessed usingthe Histogram equalization algorithm. For the feature extraction of medical images, the Multi- Threshold Contour Method is used. The shift variant decomposition transformation technique Discrete Wavelet Transform DWT is effective and it is used to decompose the medical image. The fused medical images are optimized by using Particle swarm optimization PSO. Finally, the simulation results reveal that the suggested mechanism outperforms the other techniques based on conventional approaches. Using the Origin Pro application, the study's outcomes are represented graphically

MDRANet: A multiscale dense residual attention network for magnetic resonance and nuclear medicine image fusion

Magnetic resonance and nuclear medicine images are the two categories of multimodal medical images. Magnetic resonance images reveal physiological anatomical information of patients, and nuclear medicine images accurately show tissue lesion information. Through medical image fusion algorithms, these fusion images containing both tissue lesion information and physiological anatomical information are obtained to provide sufficient information for clinical medical technologies. However, most existing fusion algorithms are based on mathematical transform domains, and these fusion results have the weaknesses of blurred edges, color distortion and detail loss. To address these problems, a multiscale dense residual attention network (MDRANet) is proposed and applied to magnetic resonance and nuclear medicine image fusion. MDRANet combines multiscale dense network and multiscale residual attention network to extract and enhance deep features. Moreover, four different loss functions are used to optimize MDRANet and improve the fusion quality. The experimental results show that the fusion results of our proposed algorithm have richer details and better objective metrics compared with the reference algorithms.

Multimodal medical image fusion algorithm based on pulse coupled neural networks and nonsubsampled contourlet transform

Combining two medical images from different modalities is more helpful for using the resulting image in the healthcare field. Medical image fusion means combining two or more images coming from multiple sensors. This technology obtains an output image that presents more effective and useful information from two images. This paper proposes a multi-modal medical image fusion algorithm based on the nonsubsampled contourlet transform (NSCT) and pulse coupled neural networks (PCNN) methods. The input images are decomposed using the NSCT method into low- and high-frequency subbands. The PCNN is a fusion rule for integrating both low- and high-frequency subbands. The inverse of the NSCT method is to reconstruct the fused image. The results of medical image fusion help doctors with disease diagnosis and patient treatment. The proposed algorithm is tested on six groups of multi-modal medical images using 100 pairs of input images. The proposed algorithm is compared with eight fusion methods. We evaluate the performance of the proposed algorithm using the fusion metrics: peak signal to noise ratio (PSNR), mutual information (MI), entropy (EN), weighted edge information (Q^{AB/F}), nonlinear correlation information entropy (Q_{ncie}), standard deviation (SD), and average gradient (AG). Experimental results show that the proposed algorithm can perform better than other medical image fusion methods and achieve promising results.Graphical abstract

An Adaptive Image Fusion Algorithm in the NSST Based on CDF 9/7 for Neurodegenerative Diseases

The neurodegenerative disease such as: Parkinson's disease (PD), mild Alzheimer’s affects many people and has a serious influence on their life, With the quick advancement of computer-aided diagnostic (CAD) methods, early detection is crucial since effective treatment halts the spread of the disease. Image fusion is useful for medical diagnostics. In this paper we propose a multi-modality medical image fusion algorithm in NSST domain. Shearlets (NSST) are decomposed similarly to contourlets (NSCT), except that instead of applying the Laplacian pyramid followed by directional filtering, shearlets use a shear matrix. In this article the Biorthogonal CDF9/7 filter is applied in the shift-invariant shearlet filter banks, then the coefficients of low frequency bands are selected using maximum rule, and using the gradient in each subband high frequency image to motivate the modified pulse coupled neural networks (Modified PCNN). Finally reverse IHS to get the fused color image, all this to optimize the calculation performance and improve the characteristics of the fused image for medical diagnosis. Our approach was validated with several brain diseases modalities: Alzheimer’s…etc. The findings reveal that the suggested image fusion technique has a higher quality than those fused by previous algorithms existing.

U-Patch GAN: A Medical Image Fusion Method Based on GAN.

Although medical imaging is frequently used to diagnose diseases, in complex diagnostic situations, specialists typically need to look at different modalities of image information. Creating a composite multimodal medical image can aid professionals in making quick and accurate diagnoses of diseases. The fused images of many medical image fusion algorithms, however, are frequently unable to precisely retain the functional and structural information of the source image. This work develops an end-to-end model based on GAN (U-Patch GAN) to implement the self-supervised fusion of multimodal brain images in order to enhance the fusion quality. The model uses the classical network U-net as the generator, and it uses the dual adversarial mechanism based on the Markovian discriminator (PatchGAN) to enhance the generator's attention to high-frequency information. To ensure that the network satisfies the Lipschitz continuity, we apply the spectral norm to each layer of the network. We also propose better adversarial loss and feature loss (feature matching loss and VGG-16 perceptual loss) based on the F-norm, which significantly enhance the quality of fused images. On public data sets, we performed a lot of tests. First, we studied how clinically useful the fused image was. The model's performance in single-slice images and continuous-slice images was then confirmed by comparison with other six most popular mainstream fusion approaches. Finally, we verify the effectiveness of the adversarial loss and feature loss.

Medical image fusion using fuzzy adaptive reduced pulse coupled neural networks

This paper addresses a novel neuro-fuzzy-based approach to set the weighted linking strength of parameter - adaptive reduced pulse coupled neural networks. In reduced PCNN based medical image fusion algorithms, it is quite essential to evaluate the prominence of each pixel in an image. The fusion performance in turn depends on the linking factor, internal activity. Thus, we need to set these values of reduced PCNN in a more adaptive manner with fewer complications and uncertainties. For this, the weighted linking strength i.e., lambda of the reduced PCNN neurons is attentively set by a fuzzy-based approach. Here, lambda of neurons is represented as fuzzy membership values using the activity level measures such as local information entropy and energy. Finally, a new model called-Fuzzy adaptive reduced pulse coupled neural networks is developed by reducing the number of parameters and fuzzy adaptive settings of them. This leads to a very less complicated network and more computational efficacy, which is a prominent part of health care requirements. The proposed scheme is free from the shortcomings such as loss of boundaries, structural details, unwanted artifacts, degradations, etc. Subjective and objective evaluations show better performance of this new approach compared to the existing techniques.

MULTIMODAL FUSION ANALYSIS OF MEDICAL IMAGES BY DISCRETE WAVELET TRANSFORMATION

Medical image fusion is the development of registry and fusion of different pots of one or more imaging modalities in order to improve image value and reduce uncertainty and concern, to improve the applicability of the medical image for clinical opinion or the evaluation of medical care. difficulties. Multimodal medical image fusion algorithms and devices have achieved a remarkable result in improving the veracity of clinical decision support in medical images. The main objective is to improve the understanding of medical images using Discrete Wavelet Transform technology. DWT mainly uses blending rules involving the average pixel. Discrete wavelet transform has been implemented using fusion technology for medical image fusion. Fusion Power is calculated based on PSNR, MSE, and Total Progression Moment. The result demonstrates the success of the fusion scheme on wavelet transform imaging (MRI) and positron production tomography (PET). Other variants of MRI (magnetic resonance imaging) and PET (positron emission tomography) performed for medical diagnosis. Keywords: Multimodal fusion of medical images, fusion rules, magnetic signs,PET.

Multi-Modal Medical Image Fusion With Geometric Algebra Based Sparse Representation.

Multi-modal medical image fusion can reduce information redundancy, increase the understandability of images and provide medical staff with more detailed pathological information. However, most of traditional methods usually treat the channels of multi-modal medical images as three independent grayscale images which ignore the correlation between the color channels and lead to color distortion, attenuation and other bad effects in the reconstructed image. In this paper, we propose a multi-modal medical image fusion algorithm with geometric algebra based sparse representation (GA-SR). Firstly, the multi-modal medical image is represented as a multi-vector, and the GA-SR model is introduced for multi-modal medical image fusion to avoid losing the correlation of channels. Secondly, the orthogonal matching pursuit algorithm based on geometric algebra (GAOMP) is introduced to obtain the sparse coefficient matrix. The K-means clustering singular value decomposition algorithm based on geometric algebra (K-GASVD) is introduced to obtain the geometric algebra dictionary, and update the sparse coefficient matrix and dictionary. Finally, we obtain the fused image by linear combination of the geometric algebra dictionary and the coefficient matrix. The experimental results demonstrate that the proposed algorithm outperforms existing methods in subjective and objective quality evaluation, and shows its effectiveness for multi-modal medical image fusion.

A new medical image enhancement algorithm using adaptive parameters

AbstractThe quality of medical images plays a vital role in many image processing applications such as image segmentation, feature extraction, image classification, image recognition, and image fusion. Some of the common problems with Medical images are noise, blur, or low contrast. According to our observations, current image enhancement algorithms only focus on solving individual problems such as gray level adjustment, noise reduction, or sharpness enhancement. This paper proposes a novel algorithm to solve problems on images simultaneously. First, we propose an image decomposition algorithm. This algorithm allows decomposing the image into three components: structure (IS), texture (IT), and noise (IN). Second, the structural component (IS) is enhanced by the contrast‐limited adaptive histogram equalization method to obtain ICLAHE. We use the structure tensor salient detection operator and the Laplace edge detection operator to add structural and texture features. These operators are applied to the ICLAHE and IT components to obtain the ISTS and ILED components, respectively. The IT and ILED components are used to generate the enhanced component (called ) by using the Max operator. Third, the Marine predators algorithm is used to find the optimal parameters β1, β2, β3, and β4 corresponding to ICLAHE, ISTS, , and IN. Finally, the enhanced image is made up of the sum of the ICLAHE, ISTS, , and IN images multiplied by the optimal parameters β1, β2, β3, and β4, respectively. Six state‐of‐the‐art image enhancement approaches, seven medical image fusion algorithms, and six image quality metrics have been utilized to verify the proposed approach's effectiveness. The experimental results show that the proposed method significantly improves the quality of the input medical images as well as significantly improves the efficiency of current medical image synthesis algorithms.

Multimodal Medical Image Fusion Based on Content-based and PCA-sigmoid.

The objective of any multimodal medical image fusion algorithm is to assist a radiologist for better decision-making during the diagnosis and therapy by integrating the anatomical (magnetic resonance imaging) and functional (positron emission tomography/ single-photon emission computed tomography) information. We proposed a new medical image fusion method based on content-based decomposition, Principal Component Analysis (PCA), and sigmoid function. We considered Empirical Wavelet Transform (EWT) for content-based decomposition purposes since it can preserve crucial medical image information such as edges and corners. PCA is used to obtain initial weights corresponding to each detail layer. In our experiments, we found that direct usage of PCA for detail layer fusion introduces severe artifacts into the fused image due to weight scaling issues. In order to tackle this, we considered using the sigmoid function for better weight scaling. We considered 24 pairs of MRI-PET and 24 pairs of MRI-SPECT images for fusion, and the results are measured using four significant quantitative metrics. Finally, we compared our proposed method with other state-of-the-art transformbased fusion approaches, using traditional and recent performance measures. An appreciable improvement is observed in both qualitative and quantitative results compared to other fusion methods.