Medical Image Fusion Technology Research Articles

GeSeNet: A General Semantic-Guided Network With Couple Mask Ensemble for Medical Image Fusion.

At present, multimodal medical image fusion technology has become an essential means for researchers and doctors to predict diseases and study pathology. Nevertheless, how to reserve more unique features from different modal source images on the premise of ensuring time efficiency is a tricky problem. To handle this issue, we propose a flexible semantic-guided architecture with a mask-optimized framework in an end-to-end manner, termed as GeSeNet. Specifically, a region mask module is devised to deepen the learning of important information while pruning redundant computation for reducing the runtime. An edge enhancement module and a global refinement module are presented to modify the extracted features for boosting the edge textures and adjusting overall visual performance. In addition, we introduce a semantic module that is cascaded with the proposed fusion network to deliver semantic information into our generated results. Sufficient qualitative and quantitative comparative experiments (i.e., MRI-CT, MRI-PET, and MRI-SPECT) are deployed between our proposed method and ten state-of-the-art methods, which shows our generated images lead the way. Moreover, we also conduct operational efficiency comparisons and ablation experiments to prove that our proposed method can perform excellently in the field of multimodal medical image fusion. The code is available at https://github.com/lok-18/GeSeNet.

An Improved Hybrid Network With a Transformer Module for Medical Image Fusion

Medical image fusion technology is an essential component of computer-aided diagnosis, which aims to extract useful cross-modality cues from raw signals to generate high-quality fused images. Many advanced methods focus on designing fusion rules, but there is still room for improvement in cross-modal information extraction. To this end, we propose a novel encoder-decoder architecture with three technical novelties. First, we divide the medical images into two attributes, namely pixel intensity distribution attributes and texture attributes, and thus design two self-reconstruction tasks to mine as many specific features as possible. Second, we propose a hybrid network combining a CNN and a transformer module to model both long-range and short-range dependencies. Moreover, we construct a self-adaptive weight fusion rule that automatically measures salient features. Extensive experiments on a public medical image dataset and other multimodal datasets show that the proposed method achieves satisfactory performance.

Multimodal medical image fusion using residual network 50 in non subsampled contourlet transform

ABSTRACT Medical image fusion technology and its collective diagnosis are becoming crucial day by day. This task confers the latest algorithm for image fusion of medical images to many diagnostic complications. Firstly, transform is employed on input source images. The result of the application of transform is the decomposition of source images into various subbands. Eminent features are extracted from these subbands by using resnet50. These features are fused by phase congruency and guided filtering fusion rules. Finally, inverse transform gives the original image. The experiment results of this algorithm are compared with different methods by taking some pairs of medical images. Subjective and objective outcomes prove that the proposed algorithm exceeds the current methods by giving optimal performance measures in the area of medical diagnosis. Thus, it is revealed that the suggested multimodal image fusion model provides elevated performance over existing models via diverse diseases using MRI-SPECT and MRI-PET.

Medical Image Fusion and Denoising Algorithm Based on a Decomposition Model of Hybrid Variation-Sparse Representation.

Medical image fusion technology integrates the contents of medical images of different modalities, thereby assisting users of medical images to better understand their meaning. However, the fusion of medical images corrupted by noise remains a challenge. To solve the existing problems in medical image fusion and denoising algorithms related to excessive blur, unclean denoising, gradient information loss, and color distortion, a novel medical image fusion and denoising algorithm is proposed. First, a new image layer decomposition model based on hybrid variation-sparse representation and weighted Schatten p-norm is proposed. The alternating direction method of multipliers is used to update the structure, detail layer dictionary, and detail layer coefficient map of the input image while denoising. Subsequently, appropriate fusion rules are employed for the structure layers and detail layer coefficient maps. Finally, the fused image is restored using the fused structure layer, detail layer dictionary, and detail layer coefficient maps. A large number of experiments confirm the superiority of the proposed algorithm over other algorithms. The proposed medical image fusion and denoising algorithm can effectively remove noise while retaining the gradient information without color distortion.

Multimodal Brain Image Fusion Based on Improved Rolling Guidance Filter and Wiener Filter.

Medical image fusion technology can integrate complementary information from different modality medical images to provide a more complete and accurate description of the specific diagnosed object, which is very helpful for image-guided clinical diagnosis and treatment. This paper proposes an effective brain image fusion framework based on improved rolling guidance filter (IRGF). Firstly, input images are decomposed into base layers and detail layers using the IRGF and Wiener filter. Secondly, the visual saliency maps of the input image are computed by pixel-level saliency value, and the weight maps of detail layers are constructed by max-absolute strategy and are further smoothed with Gaussian filter, the purpose of which is to make the fused image appear more naturally and more suitable for human visual perception. Lastly, base layers are fused by visual saliency map based fusion rule and the corresponding weight maps from detail layers are fused by the weighted least squares optimization scheme. Experimental results testify that our method is superior to some state-of-the-art methods in both subjective and objective assessments.

Application of Multimodal Fusion Technology in Image Analysis of Pretreatment Examination of Patients with Spinal Injury.

As one of the most common imaging screening techniques for spinal injuries, MRI is of great significance for the pretreatment examination of patients with spinal injuries. With rapid iterative update of imaging technology, imaging techniques such as diffusion weighted magnetic resonance imaging (DWI), dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI), and magnetic resonance spectroscopy are frequently used in the clinical diagnosis of spinal injuries. Multimodal medical image fusion technology can obtain richer lesion information by combining medical images in multiple modalities. Aiming at the two modalities of DCE-MRI and DWI images under MRI images of spinal injuries, by fusing the image data under the two modalities, more abundant lesion information can be obtained to diagnose spinal injuries. The research content includes the following: (1) A registration study based on DCE-MRI and DWI image data. To improve registration accuracy, a registration method is used, and VGG-16 network structure is selected as the basic registration network structure. An iterative VGG-16 network framework is proposed to realize the registration of DWI and DCE-MRI images. The experimental results show that the iterative VGG-16 network structure is more suitable for the registration of DWI and DCE-MRI image data. (2) Based on the fusion research of DCE-MRI and DWI image data. For the registered DCE-MRI and DWI images, this paper uses a fusion method combining feature level and decision level to classify spine images. The simple classifier decision tree, SVM, and KNN were used to predict the damage diagnosis classification of DCE-MRI and DWI images, respectively. By comparing and analyzing the classification results of the experiments, the performance of multimodal image fusion in the auxiliary diagnosis of spinal injuries was evaluated.

Multimodal medical image fusion based on multichannel coupled neural P systems and max-cloud models in spectral total variation domain

Medical image fusion technology can integrate the positive attributes of images from different imaging devices such that the fusion image contains rich and nonredundant information. Existing fusion images typically suffer from problems of color distortion, detail loss, and poor noise robustness. In this study, a medical image fusion algorithm based on spectral total variation (STV) is proposed to address these problems. First, the base and residual layers of the input images to be fused was acquired using STV. Subsequently, a Max-cloud fusion model was proposed to fuse the base layers, and the energy and contrast brightness information was effectively processed. A multichannel coupled neural P system was proposed to overcome the shortcomings of the single-channel model for the residual layers. Finally, the fused image was reconstructed. Numerous experiments with and without noise were performed, and qualitative and quantitative comparisons were made with the classical fusion methods. The experimental results indicate that the proposed medical image fusion algorithm is more effective and provides clearer edge details, higher luminance, and superior colors, exhibiting higher robustness than the other classical 11 fusion methods.

Functional and Anatomical Image Fusion Based on Gradient Enhanced Decomposition Model

Medical image fusion technology can synthesize complementary information from multimodal medical images while saving storage costs. Targeting the problems of reduced contrast, blurred details, color distortion, and excessive processing time requirement in fusion algorithms, we propose a new functional and anatomical image fusion algorithm based on a gradient enhanced decomposition model. First, a new gradient-enhanced decomposition (GED) model is proposed. Subsequently, the proposed GED model is expanded into a three-layer decomposition, wherein the input image is decomposed into a basic, local mean energy, and texture layers using alternating direction method of multipliers. Compared with other classic decomposition method, the proposed three-layer decomposition has the advantages of no additional noise and requiring less time while enhanced gradient. Second, we propose targeted fusion rules for each layer. After the basic layer is weighted averaged, the gamma function is used to adjust its brightness. The product of the local l2 norm and largest singular value constitutes the fusion weights of the local mean energy layers. The fusion weights of the texture layers are determined by the product of the local l1 norm and largest singular value. Finally, the fused image is reconstructed. We perform numerous experiments to prove that the proposed fusion algorithm providing clearer edge details and superior colors in a short processing time, and there was a statistically significant difference compare with other classical algorithms.

A multiscale double-branch residual attention network for anatomical–functional medical image fusion

Medical image fusion technology synthesizes complementary information from multimodal medical images. This technology is playing an increasingly important role in clinical applications. In this paper, we propose a new convolutional neural network, which is called the multiscale double-branch residual attention (MSDRA) network, for fusing anatomical–functional medical images. Our network contains a feature extraction module, a feature fusion module and an image reconstruction module. In the feature extraction module, we use three identical MSDRA blocks in series to extract image features. The MSDRA block has two branches. The first branch uses a multiscale mechanism to extract features of different scales with three convolution kernels of different sizes, while the second branch uses six 3 × 3 convolutional kernels. In addition, we propose the Feature L1-Norm fusion strategy to fuse the features obtained from the input images. Compared with the reference image fusion algorithms, MSDRA consumes less fusion time and achieves better results in visual quality and the objective metrics of Spatial Frequency (SF), Average Gradient (AG), Edge Intensity (EI), Quality-Aware Clustering (QAC), Variance (VAR), and Visual Information Fidelity for Fusion (VIFF).

Application of U-Net Based Multiparameter Magnetic Resonance Image Fusion in the Diagnosis of Prostate Cancer

Medical image fusion technology has been widely used in clinical practice by doctors to better understand lesion regions through the fusion of multiparametric medical images. This paper proposes an automated fusion method based on a U-Net. Through neural network learning, a weight distribution is generated based on the relationship between the image feature information and the multifocus training target. The MRI image pair of prostate cancer (axial T2-weighted and ADC map) is fused using a strategy based on local similarity and Gaussian pyramid transformation. Experimental results show that the fusion method can enhance the appearance of prostate cancer in terms of both visual quality and objective evaluation.

Regions Preserving Edge Enhancement for Multisensor-Based Medical Image Fusion

Multimodal medical image fusion technology has been widely used in various applications of clinical diagnosis, which aims to provide richer information by integrating effective features of multiple medical images. In this article, a novel fusion scheme based on region-preserving edge enhancement is proposed for medical images. The overlapping regions of source images that need to be fused are detected first. Next, according to the different visual features, the overlapping map is divided by Fuzzy c-means (FCM)-based algorithm. Then, different fusion strategies based on visual saliency and texture details are proposed for region-preserving and edge enhancement. Experimental results show that the proposed scheme can effectively highlight the visual features and retain key information. Furthermore, the performance of different medical images also demonstrates that the proposed method can obtain better results than those achieved by some of the state-of-the-art methods.

Medical image fusion based on hybrid three-layer decomposition model and nuclear norm

The aim of medical image fusion technology is to synthesize multiple-image information to assist doctors in making scientific decisions. Existing studies have focused on preserving image details while avoiding halo artifacts and color distortions. This paper proposes a novel medical image fusion algorithm based on this research objective. First, the input image is decomposed into structure, texture, and local mean brightness layers using a hybrid three-layer decomposition model that can fully extract the features of the original images without the introduction of artifacts. Secondly, the nuclear norm of the patches, which are obtained using a sliding window, are calculated to construct the weight maps of the structure and texture layers. The weight map of the local mean brightness layer is constructed by calculating the local energy. Finally, remapping functions are applied to enhance each fusion layer, which reconstructs the final fusion image with the inverse operation of decomposition. Subjective and objective experiments confirm that the proposed algorithm has a distinct advantage compared with other state-of-the-art algorithms.

Multi-modal brain image fusion based on multi-level edge-preserving filtering

Recently, multi-modal medical imaging technology and its collaborative diagnosis technology are developing rapidly. The application of medical image fusion technology in medical diagnosis becomes more important. In this paper, a multi-modal medical image fusion algorithm based on multi-level edge-preserving filtering (MLEPF) decomposition model is proposed. Firstly, an MLEPF model based on weighted mean curvature filtering is presented and used to decompose the multi-modal medical image into three types of layers: fine-structure (FS), coarse-structure (CS), and base (BS) layers. Secondly, a gradient domain pulse-coupled neural network (PCNN) fusion strategy is used to merge the FS and CS layers, and an energy attribute fusion strategy is used to merge the BS layers. Finally, the fused image is obtained by combining the three types of fused layers. The experiments are performed on six different disease datasets and one normal dataset, which contains more than 100 image pairs. Qualitative and quantitative evaluation testify that the proposed algorithm is superior to some excellent algorithms and can achieve close result to some state-of-the-art algorithms.

Research progress in nerve block guiding technology

Nerve block is one of the key techniques in clinical anesthesiology. In the past 20 years, from the initial method of abnormal sensation to the current techniques of nerve stimulator and ultrasound guidance, more and more guiding techniques and methods have been applied in nerve block. In recent years, with respect to the development of medical image fusion technology and artificial intelligence identification for medical imaging, the nerve block guidance technology has greatly developed. Ultrasound combined with other technologies (abbreviated as ultrasound+ technology) can help to solve the difficulty of puncture positioning caused by obesity, anatomical variation and other factors when ultrasound is used alone. In the future, more research and development of more technologies and ultrasound should be used. In order to understand the advantages and disadvantages of various nerve block guiding techniques in recent years and to guide the rational application in clinical setting, this paper reviews various nerve block guiding techniques in recent years, expounds the process from the initial abnormal sensation method to the latest ultrasound+ technology and the latest research progress of ultrasound+ technology, compares the advantages and disadvantages of various nerve block guidance techniques, and looks forward to the new nerve block guiding technology. Key words: Nerve block; Guidiance; Ultrasound; Artificial intelligence; Image fusion; Magnetic navigation

A preliminary methodology and precision study of producing 3D printing heart model by multimodel medical image fusion technology

Objective To obtain abundant anatomical information using fusion of ultrasonic valve images and CT heart images and provide new methods of image processing for 3D heart printing. Methods Cardiac examinations of three-dimensional transesophageal echocardiography (3D-TEE) and DICOM images of 41 patients who had atrial fibrillation without structural cardiac diseases were analyzed retrospectively and the data afterward were processed by using Mimics software. For the single-mode image group, only the heart CT were invoked as the data sources. For the multimodel image group, CT and ultrasound images were invoked as the data sources. Valve structures of the 3D reconstructions were graded and the circumferences, areas and other parameters of the two groups were measured. The images of two groups were overlapped, and the angles between the two valve planes of each heart were evaluated and analysed. Results Score of valves reconstructed by ultrasound 3D construction was higher than those by CT reconstruction. Measurements of correlated parameters between the 3D-TEE group and the CT group showed no significant statistical difference(P>0.05), and the consistency was well. Ultrasound measurements of 3D reconstructed valves were well concordant with CT reconstructed valves by Bland-Altman analysis. Mean value and standard deviation of the angles between mitral valve annulus and aortic valve ring plane of each heart in the two group were (3.15±0.88)°, (2.87±0.76)°, respectively. Conclusions Ultrasonic valve images can fuse precisely with CT images and it provides a better displayed morphology. This will help improving simulation quality of the 3D printed hearts. Key words: Echocardiography; Image integrate and registration; 3D printing

P1767A preliminary methodology study of producing 3D printing left heart model by multimodal medical image fusion technology

P1767A preliminary methodology study of producing 3D printing left heart model by multimodal medical image fusion technology

A novel dictionary learning approach for multi-modality medical image fusion

Multi-modality medical image fusion technology can integrate the complementary information of different modality medical images, obtain more precise, reliable and better description of lesions. Dictionary learning based image fusion draws a great attention in researchers and scientists, for its high performance. The standard learning scheme uses entire image for dictionary learning. However, in medical images, the informative region takes a small proportion of the whole image. Most of the image patches have limited and redundant information. Taking all the image patches for dictionary learning brings lots of unvalued and redundant information, which can influence the medical image fusion quality. In this paper, a novel dictionary learning approach is proposed for image fusion. The proposed approach consists of three steps. Firstly, a novel image patches sampling scheme is proposed to obtain the informative patches. Secondly, a local density peaks based clustering algorithm is conducted to classify the image patches with similar image structure information into several patch groups. Each patch group is trained to a compact sub-dictionary by K-SVD. Finally the sub-dictionaries are combined to a complete, informative and compact dictionary. In this dictionary,only important and useful information which can effectively describe the medical image are selected. To show the efficiency of the proposed dictionary learning approach, the sparse coefficient vectors are estimated by a simultaneous orthogonal matching pursuit (SOMP) algorithm with the trained dictionary, and fused by max-L1 rules. The comparative experimental results and analyses reveal that the proposed method achieves better image fusion quality than existing state-of-the-art methods.

Application of multimodal image fusion technology in brain tumor surgery

Objective To construct brain tumors and their surrounding anatomical structures using the multi-modality image data registration and fusion and three dimensional reconstruction method in order to provide guidance for brain tumor surgery. Methods Ten patients with brain tumor admitted to the Department of Neurosurgery, Yuquan Hospital, Qsinghua University from April 2015 to December 2015 were selected. Head CT and MRI were used to collect the data in patients with brain tumor. NeuroTech software was used to conduct registration and fusion and 3D reconstruction for image data. Results The 3D structures of brain tumor, brain tissue, and blood vessel in 10 patients with brain tumor were reconstructed. They clearly revealed the anatomic relationship of tumor size, location, shape, and tumor and surrounding structures. They assisted in the design of the surgical incision and the selection of operative approach and tumor resection. Conclusions Using multi-modality medical image fusion technology can visually display the relationship between tumors and blood vessels, brain tissue, skull, and other tissues, provide a large amount of visual information, guide the design of incision of neurosurgery, choose the operative approaches, resect the tumors, and play roles in the aspect of protecting brain function and reducing postoperative complications. Key words: Brain neoplasms; Neurosurgical procedures; Imaging, three-dimensional; Multimodal; Magnetic resonance imaging

Study on the Application of Segmentation Algorithm Based on Medical Image Fusion

The medical image fusion technology has become a hot research field of medical imaging; the image segmentation is an important task in image fusion. Due to the diversity and complexity of the medical images, the medical image fusion technology has a big difficulty in image segmentation. Threshold method becomes an important image segmentation way due to its high efficiency and simple feature. However, for the segmentation of the complex medical images, the effect of the threshold method is far from being ideal. Powell algorithm is the best direct search way; the application of the improved Powell algorithm can search the target better. Therefore, the authors propose an image segmentation method that is based on variable threshold and combines with Powell algorithm. Through the simulation experiment, this method can segment the images rapidly and effectively, and features a strong robustness.

Research and development of multi-modality medical image fusion technology

医学图像融合技术是影像学技术在医学上极具特色的应用,它将解剖影像与功能影像两者有机的结合起来,实现了从单一的形态学诊断进入到功能与形态相结合的综合影像诊断.重点介绍了多模态医学图像融合的意义、融合技术的分类以及各实现方法的原理,主要包括加权平均法、多分辨金字塔法、小波变换法等常用技术,并分析了当前该领域存在的技术难点以及发展前景。