Fusion Of MRI Images Research Articles

HBANet: A hybrid boundary-aware attention network for infrared and visible image fusion

Infrared and visible image fusion is an extensively investigated problem in infrared image processing, aiming to extract useful information from source images. However, the automatic fusion of these images presents a significant challenge due to the large domain difference and ambiguous boundaries. In this article, we propose a novel image fusion approach based on hybrid boundary-aware attention, termed HBANet, which models global dependencies across the image and leverages boundary-wise prior knowledge to supplement local details. Specifically, we design a novel mixed boundary-aware attention module that is capable of leveraging spatial information to the fullest extent and integrating long dependencies across different domains. To preserve the integrity of texture and structural information, we introduced a sophisticated loss function that comprises structure, intensity, and variation losses. Our method has been demonstrated to outperform state-of-the-art methods in terms of both visual and quantitative metrics, in our experiments on public datasets. Furthermore, our approach also exhibits great generalization capability, achieving satisfactory results in CT and MRI image fusion tasks.

CT and MRI Image Fusion via Coupled Feature-Learning GAN

The fusion of multimodal medical images, particularly CT and MRI, is driven by the need to enhance the diagnostic process by providing clinicians with a single, comprehensive image that encapsulates all necessary details. Existing fusion methods often exhibit a bias towards features from one of the source images, making it challenging to simultaneously preserve both structural information and textural details. Designing an effective fusion method that can preserve more discriminative information is therefore crucial. In this work, we propose a Coupled Feature-Learning GAN (CFGAN) to fuse the multimodal medical images into a single informative image. The proposed method establishes an adversarial game between the discriminators and a couple of generators. First, the coupled generators are trained to generate two real-like fused images, which are then used to deceive the two coupled discriminators. Subsequently, the two discriminators are devised to minimize the structural distance to ensure the abundant information in the original source images is well-maintained in the fused image. We further empower the generators to be robust under various scales by constructing a discriminative feature extraction (DFE) block with different dilation rates. Moreover, we introduce a cross-dimension interaction attention (CIA) block to refine the feature representations. The qualitative and quantitative experiments on common benchmarks demonstrate the competitive performance of the CFGAN compared to other state-of-the-art methods.

CT and MRI image fusion via multimodal feature interaction network

CT and MRI image fusion via multimodal feature interaction network

Brain MRI and CT Image Fusion Using Multiscale Local Extrema and Image Statistics

In medical applications such as radiotherapy and guided-image surgery, data fusion for diagnostic imaging has emerged as a critical issue. Because the objective of medical image fusion is to improve patient diagnosis accuracy, the fused image is created by preserving the source images' prominent details and features. It has been demonstrated that the Multi-Level Local Extrema representation has numerous advantages over traditional image modeling approaches. We propose an innovative MLE-based fusion method for multimodal medical images in this paper. In the MLE schema, the proposed algorithm decomposes source images into coarse and detailed layers, then fuses the source images using weights calculated from these detail images using image statistics. We visually and quantitatively compared the efficacy of the suggested approach to that of existing methods using five different types of medical images from various sources. The experimental results showed that the proposed scheme outperforms other current typical schemes in terms of both qualitative image quality and objective evaluation.

Adaptive convolutional sparsity with sub-band correlation in the NSCT domain for MRI image fusion

Objective. Multimodal medical image fusion (MMIF) technologies merges diverse medical images with rich information, boosting diagnostic efficiency and accuracy. Due to global optimization and single-valued nature, convolutional sparse representation (CSR) outshines the standard sparse representation (SR) in significance. By addressing the challenges of sensitivity to highly redundant dictionaries and robustness to misregistration, an adaptive convolutional sparsity scheme with measurement of the sub-band correlation in the non-subsampled contourlet transform (NSCT) domain is proposed for MMIF. Approach. The fusion scheme incorporates four main components: image decomposition into two scales, fusion of detail layers, fusion of base layers, and reconstruction of the two scales. We solved a Tikhonov regularization optimization problem with source images to obtain the base and detail layers. Then, after CSR processing, detail layers were sparsely decomposed using pre-trained dictionary filters for initial coefficient maps. NSCT domain’s sub-band correlation was used to refine fusion coefficient maps, and sparse reconstruction produced the fused detail layer. Meanwhile, base layers were fused using averaging. The final fused image was obtained via two-scale reconstruction. Main results. Experimental validation of clinical image sets revealed that the proposed fusion scheme can not only effectively eliminate the interference of partial misregistration, but also outperform the representative state-of-the-art fusion schemes in the preservation of structural and textural details according to subjective visual evaluations and objective quality evaluations. Significance. The proposed fusion scheme is competitive due to its low-redundancy dictionary, robustness to misregistration, and better fusion performance. This is achieved by training the dictionary with minimal samples through CSR to adaptively preserve overcompleteness for detail layers, and constructing fusion activity level with sub-band correlation in the NSCT domain to maintain CSR attributes. Additionally, ordering the NSCT for reverse sparse representation further enhances sub-band correlation to promote the preservation of structural and textural details.

CT and MRI image fusion based on variance and pixel significance

In the medical imaging field, images are obtained using various modalities, including the computed tomography procedure and the magnetic resonance imaging procedure. In which every image contains different information from the other image. For better treatment and diagnosis of a patient, a single composite image must be created by fusing all the pertinent data. This process is known as image fusion. We present an innovative and effective image fusion technique utilizing ILWT and DCT for combining brain-related medical images acquired through there are two steps to this strategy. First, the variance is employed as a contrast evaluation in the DCT domain to combine the approximation coefficients generated by the ILWT decomposition. Second, the coefficients representing the fine details are combined by finding the ideal weighted average based on the importance of the pixels in the ILWT domain. Our method is straightforward, making it simple and suitable for deployment in real-time applications. The experimental results demonstrate our method's outstanding performance with regard to both result quality and in contrast to a number of picture fusion techniques.

MRI Image Fusion Based on Sparse Representation with Measurement of Patch-Based Multiple Salient Features

Multimodal medical image fusion is a fundamental, but challenging, problem in the fields of brain science research and brain disease diagnosis, as it is challenging for sparse representation (SR)-based fusion to characterize activity levels with a single measurement and not lose effective information. In this study, the Kronecker-criterion-based SR framework was applied for medical image fusion with a patch-based activity level, integrating salient features of multiple domains. Inspired by the formation process of vision systems, the spatial saliency was characterized by textural contrast (TC), composed of luminance and orientation contrasts, to promote the participation of more highlighted textural information in the fusion process. As a substitute for the conventional l1-norm-based sparse saliency, the sum of sparse salient features (SSSF) was used as a metric for promoting the participation of more significant coefficients in the composition of the activity level measurement. The designed activity level measurement was verified to be more conducive to maintaining the integrity and sharpness of detailed information. Various experiments on multiple groups of clinical medical images verified the effectiveness of the proposed fusion method in terms of both visual quality and objective assessment. Furthermore, this study will be helpful for the further detection and segmentation of medical images.

HyperTDP-Net: A Hyper-densely Connected Compression-and-Decomposition Network Based on Trident Dilated Perception for PET and MRI Image Fusion

Objective. Since medical images generated by medical devices have low spatial resolution and quality, fusion approaches on medical images can generate a fused image containing a more comprehensive range of different modal features to help physicians accurately diagnose diseases. Conventional methods based on deep learning for medical image fusion usually extract only local features without considering their global features, which often leads to the problem of unclear detail information in the final fused image. Therefore, medical image fusion is a challenging task of great relevance. Approach. This paper proposes a novel end-to-end medical image fusion model for PET and MRI images to achieve information interaction between different pathways, termed as hyper-densely connected compression-and-decomposition network based on trident dilated perception for PET and MRI image fusion (HyperTDP-Net). In particular, in the compression network, a dual residual hyper densely module is constructed to take full advantage of middle layer information. Moreover, we establish the trident dilated perception module to precisely determine the location information of features, and improve the feature representation capability of the network. In addition, we abandon the ordinary mean square error as the content loss function and propose a new content-aware loss consisting of structural similarity loss and gradient loss, so that the fused image not only contains rich texture details but also maintains sufficient structural similarity with the source images.Main results. The experimental dataset used in this paper is derived from multimodal medical images published by Harvard Medical School. Extensive experiments illustrate that our model contains more edge information and texture detail information in the fusion result than the 12 state-of-the-art fusion models and ablation study results demonstrate the effectiveness of three technical innovations.Significance. As medical images continue to be used in clinical diagnosis, our method is expected to be a tool that can effectively improve the accuracy of physician diagnosis and automatic machine detection.

A Case of Rectal Dissemination of Gastric Cancer Diagnosed by Simultaneous 18 F-FDG PET/MRI.

Metastatic rectal cancer is rare and difficult to differentiate from primary rectal cancer. A 79-year-old man with a rectal mass detected by CT during postoperative follow-up of gastric cancer underwent 18 F-FDG PET/MRI. Fused PET/MRI images revealed a lower FDG uptake within the mass, which appeared to surround the outside of the rectum, than in the rectal wall, suggesting rectal dissemination of gastric cancer. PET/MRI was useful for differentiating between mass and rectal wall uptake, because of the high contrast resolution of MRI and precise image fusion made possible by simultaneous image acquisition.

MRI and CT image fusion using cartoon-texture and QWT decomposition and cuckoo search-grey wolf optimization

MRI and CT image fusion using cartoon-texture and QWT decomposition and cuckoo search-grey wolf optimization

Image Fusion of MRI and CT Images Using DTCWT

Image Fusion of MRI and CT Images Using DTCWT

Transperineal Prostate Biopsy Targeted by Magnetic Resonance Imaging Cognitive Fusion

Prostate cancer is among the most frequently diagnosed cancers and a leading cause of cancer-related death in men. Currently, the most reliable and widely used imaging test for prostate cancer diagnosis is multiparametric pelvic magnetic resonance imaging (mpMRI). Modern biopsy techniques are based on the computerised merging of ultrasound and MRI images to provide better vision during the biopsy procedure (Fusion Biopsy). However, the method is expensive due to high equipment cost. Cognitive fusion of ultrasound and MRI images has recently emerged as a cheaper and easier alternative to computerised fusion. The aim of this prospective study is to perform an in-patient comparison of the systematic prostate biopsy procedure (SB) vs. cognitive fusion (CF) guided prostate biopsy method in terms of safety, ease of use, cancer detection rate and clinically significant cancer detection. We enrolled 103 patients with suspected prostate cancer that were biopsy naive, with PSA > 4 ng/dL and PIRADS score of 3, 4 or 5. All patients received a transperineal standard 12-18 cores systematic biopsy (SB) and a four-cores targeted cognitive fusion (CF) biopsy. Following the prostate biopsy, 68% of the patients were diagnosed with prostate cancer (70/103 patients). SB diagnosis rate was 62% while CF biopsy was slightly better with a 66% rate. There was a significant 20% increase in clinically significant prostate cancer detection rate for the CF compared to SB (p < 0.05) and a significant prostate cancer risk upgrade from the low to the intermediate risk category (13%, p = 0.041). Transperineal cognitive fusion targeted prostate biopsy is a straightforward biopsy method that is easy to perform and is a safe alternative to standard systematic biopsy with improved significant cancer detection accuracy. A combined targeted and systematic approach should be used for the best diagnostic results.

CT and MRI image fusion via dual-branch GAN

CT and MRI image fusion via dual-branch GAN

Multi-modal MRI image fusion of the brain based on joint bilateral filter and non-subsampled shearlet transform

Multi-modal MRI image fusion of the brain based on joint bilateral filter and non-subsampled shearlet transform

Image Fusion Involving Real-Time Transabdominal or Endoscopic Ultrasound for Gastrointestinal Malignancies: Review of Current and Future Applications.

Image fusion of CT, MRI, and PET with endoscopic ultrasound and transabdominal ultrasound can be promising for GI malignancies as it has the potential to allow for a more precise lesion characterization with higher accuracy in tumor detection, staging, and interventional/image guidance. We conducted a literature review to identify the current possibilities of real-time image fusion involving US with a focus on clinical applications in the management of GI malignancies. Liver applications have been the most extensively investigated, either in experimental or commercially available systems. Real-time US fusion imaging of the liver is gaining more acceptance as it enables further diagnosis and interventional therapy of focal liver lesions that are difficult to visualize using conventional B-mode ultrasound. Clinical studies on EUS guided image fusion, to date, are limited. EUS-CT image fusion allowed for easier navigation and profiling of the target tumor and/or surrounding anatomical structure. Image fusion techniques encompassing multiple imaging modalities appear to be feasible and have been observed to increase visualization accuracy during interventional and diagnostic applications.

Comparative Analysis and Fusion of MRI and PET Images based on Wavelets for Clinical Diagnosis

Comparative Analysis and Fusion of MRI and PET Images based on Wavelets for Clinical Diagnosis

Multi-Scale Mixed Attention Network for CT and MRI Image Fusion.

Recently, the rapid development of the Internet of Things has contributed to the generation of telemedicine. However, online diagnoses by doctors require the analyses of multiple multi-modal medical images, which are inconvenient and inefficient. Multi-modal medical image fusion is proposed to solve this problem. Due to its outstanding feature extraction and representation capabilities, convolutional neural networks (CNNs) have been widely used in medical image fusion. However, most existing CNN-based medical image fusion methods calculate their weight maps by a simple weighted average strategy, which weakens the quality of fused images due to the effect of inessential information. In this paper, we propose a CNN-based CT and MRI image fusion method (MMAN), which adopts a visual saliency-based strategy to preserve more useful information. Firstly, a multi-scale mixed attention block is designed to extract features. This block can gather more helpful information and refine the extracted features both in the channel and spatial levels. Then, a visual saliency-based fusion strategy is used to fuse the feature maps. Finally, the fused image can be obtained via reconstruction blocks. The experimental results of our method preserve more textual details, clearer edge information and higher contrast when compared to other state-of-the-art methods.

Role of CT and MRI image fusion and computer assisted simulation technique in guiding type Ⅲ and Ⅳ primary pelvic sarcoma surgeries

Objective: To explore the possible clinical benefits of CT/MRI image fusion and computer-assisted simulation techniques in guiding type Ⅲ and Ⅳ primary pelvic bone tumor surgeries. Methods: The clinic data of primary bone sarcomas patients treated at Department of Bone and Soft Tissue,Zhenghzhou University Affiliated Cancer Hospital from January 2019 to December 2021 were retrospectively analyzed. Based on whether the CT and MRI image fusion technique was utilized for tumor evaluation and surgical planning,the patients were divided into image fusion group (n=21) or control group (n=27). There were 7 male and 14 female patients included in the image fusion group, with the age of (37.0±10.4) years(range: 18 to 67 years). In the control group, there were 10 males and 17 females with the age of (39.7±15.2) years (range: 16 to 65 years). Both groups included osteosarcoma,chondrosarcoma and undifferentiated polymorphic sarcoma as the pathological diagnosis. Clinical information such as gender,age,pathological diagnosis,location of disease,and metastasis at diagnosis were collected. Surgical related information such as duration of surgery,blood loss,surgical margin,and wound complications were also obtained. Periodical follow-ups every 3 months were performed for all patients to monitor the status of local recurrence,distant metastasis,and survival information. Independent t test and χ² test were used for data comparison between groups. Results: Significant reduced duration of surgery was observed in the image fusion group in comparison with control group both in type Ⅲ and Ⅳ surgeries ((144.0±31.6)min vs. (248.2±56) min,t=-8.084, P<0.01); (173.0±42.0)min vs. (306.1±62.0)min, t=-4.518, P<0.01). Blood loss was significantly reduced in the image fusion group compared with the control group ((484.8±226.3)ml vs. (836.1±359.8)ml,t=-4.130, P<0.01). In addition, significant lower ratio of R1 margin and recurrence rates of type Ⅲ and Ⅳ surgeries were found in the image fusion group comparing with the control group (4.8%(1/21) vs. 22.2%(6/27), χ²=4.214, P=0.040; 4.8%(1/21) vs. 22.2%(6/27), χ²=4.214, P=0.040).In the image fusion group, there were 3 cases of incision infection, 1 of which underwent secondary debridement.And in thecontrol group there were 7 cases of incision infection, 3 of which underwent secondary debridement. There was no significant difference in the incidence of complications between the two groups (14.2%(3/21)vs. 25.9%(7/27), χ²=0.645, P=0.422). Up to the last follow-up, 1 patient died in the image fusion group and 2 patients died in the control group, the difference was not statistically significant (χ²=1.885, P=0.220). Conclusion: Compared with the traditional operation,the image fusion technique can significantly reduce the duration of surgery,blood loss and lower the recurrence rate by achieving better surgical margins.

Using IVIM Parameters to Differentiate Prostate Cancer and Contralateral Normal Tissue through Fusion of MRI Images with Whole-Mount Pathology Specimen Images by Control Point Registration Method.

The intravoxel incoherent motion (IVIM) model may enhance the clinical value of multiparametric magnetic resonance imaging (mpMRI) in the detection of prostate cancer (PCa). However, while past IVIM modeling studies have shown promise, they have also reported inconsistent results and limitations, underscoring the need to further enhance the accuracy of IVIM modeling for PCa detection. Therefore, this study utilized the control point registration toolbox function in MATLAB to fuse T2-weighted imaging (T2WI) and diffusion-weighted imaging (DWI) MRI images with whole-mount pathology specimen images in order to eliminate potential bias in IVIM calculations. Sixteen PCa patients underwent prostate MRI scans before undergoing radical prostatectomies. The image fusion method was then applied in calculating the patients’ IVIM parameters. Furthermore, MRI scans were also performed on 22 healthy young volunteers in order to evaluate the changes in IVIM parameters with aging. Among the full study cohort, the f parameter was significantly increased with age, while the D* parameter was significantly decreased. Among the PCa patients, the D and ADC parameters could differentiate PCa tissue from contralateral normal tissue, while the f and D* parameters could not. The presented image fusion method also provided improved precision when comparing regions of interest side by side. However, further studies with more standardized methods are needed to further clarify the benefits of the presented approach and the different IVIM parameters in PCa characterization.

Assessment of 3D Lumbosacral Vascular Anatomy for OLIF51 by Non-Enhanced MRI and CT Medical Image Fusion Technique.

Study design: Prospective study. Objective: Medical image fusion can provide information from multiple modalities in a single image. The present study aimed to determine whether three-dimensional (3D) lumbosacral vascular anatomy could be adequately portrayed using a non-enhanced CT–MRI medical image fusion technique. Summary of Background Data: Lateral lumbar interbody fusion has gained popularity for the surgical treatment of adult spinal deformity (ASD). Oblique lumbar interbody fusion at L5–S1 (OLIF51) is receiving considerable attention as a method of creating good L5–S1 lordosis. Access in OLIF51 requires evaluation of the vascular anatomy in the lumbosacral region. Conventional imaging modalities need a contrast medium to describe the vascular anatomy. Methods: Participants comprised 15 patients with ASD or degenerative lumbar disease who underwent corrective surgery at our hospital between January 2020 and June 2021. A 3D vascular image with bony structures was obtained by fusing results from MRI and CT. We processed the merged image and measured the distance between left and right common iliac arteries and veins at two levels: the lower end of the L5 vertebral body (Window A) and the upper end of the S1 vertebral body (Window B). Results: The mean sizes of Window A and Window B were 29.7 ± 10.7 mm and 36.9 ± 10.3 mm, respectively. The mean distance from the bifurcation to the lower end of the L5 vertebra was 23.7 ± 10.9 mm. Coronal deviation of the bifurcation was, from center to left, 12.6 ± 12.3 mm, and the distance from the center of the L5 vertebral body to the bifurcation was 0.79 ± 7.3 mm. Only one case showed a median sacral vein (6.7%). Clinically, we performed OLIF51 in 12 of the 15 cases (80%). Conclusion: Evaluating 3D lumbosacral vascular anatomy using a non-enhanced MRI and CT medical image fusion technique is very useful for OLIF51, particularly for patients in whom the use of contrast medium is contraindicated.