Image Fusion Techniques Research Articles

Attacking the Achilles heel of cardiac amyloid nuclear scintigraphy: How to reduce equivocal and false positive studies

BackgroundPlanar and single-photon emission computed tomography (SPECT) nuclear imaging techniques with bone seeking radiotracers have been increasingly adopted for diagnosis of ATTR cardiac amyloidosis. However, inherent limitations of these techniques due to lack of anatomical landmarks have been recognized, with consequent high numbers of equivocal or false positive cases. SPECT/computed tomography (CT) fusion imaging offers a significant advantage to overcome these limitations by substantially reducing inaccurate interpretations. The authors present the results of a 3-year imaging quality improvement project that focused on reducing the high number of equivocal studies that were noted in the first two years of the amyloidosis program, comparing SPECT only to SPECT/CT fusion technique. MethodsA retrospective, systematic analysis of 176 patient records was performed to test the premise that SPECT/CT fusion imaging has the potential to reduce equivocal and false positive results. ResultsOf a total of 176 patients, 35 equivocal (19.8%), 32 (18.18%) strongly suggestive, and 109 (61.93%) not suggestive cases were identified. Recognizing that this was not consistent with the international data, the authors set out on a comprehensive quality assessment project to reduce the number of equivocal and false positive cases. In patients who initially underwent SPECT only (Group A; n = 78), the addition of SPECT/CT fusion resulted in the net reclassification of 73% of cases: 100% of equivocal cases (n = 35) were reclassified to not suggestive (n = 34) or strongly suggestive (n = 1). 73% of strongly suggestive cases (n = 30) were reclassified to not suggestive (n = 22) while 8 strongly suggestive cases were confirmed as true positives. 13 not suggestive cases remained negative after SPECT/CT fusion. In cases where SPECT/CT fusion was utilized from the beginning (Group B; n = 98), there were no reclassification of any of the cases when these cases were reprocessed as a control group. ConclusionAddition of SPECT/CT imaging reduces the false positive or equivocal studies and increases the diagnostic accuracy of the test. All false positive and equivocal studies were eliminated using the fusion technique. Utilizing the fusion imaging technique increases the spatial resolution, with the ability to localize myocardial uptake and accurately differentiate from blood pool, which is a major source of error.

DBPFNet: a dual-band polarization image fusion network based on the attention mechanism and atrous spatial pyramid pooling.

In the current image fusion techniques, typically dual-band images are fused to obtain a fused image with salient target information, or intensity and polarization images are fused to achieve an image with enhanced visual perception. However, the current lack of dual-band polarization image datasets and effective fusion methods pose significant challenges for extracting more information in a single image. To address these problems, we construct a dataset containing intensity and polarization images in the visible and near-infrared bands. Furthermore, we propose an end-to-end image fusion network using attention mechanisms and atrous spatial pyramid pooling to extract key information and multi-scale global contextual information. Moreover, we design efficient loss functions to train the network. The experiments verify that the proposed method achieves better performance than the state-of-the-art in both subjective and objective evaluations.

EVALUATION OF THE EFFECTS OF LUNGS CHEST X-RAY IMAGE FUSION WITH ITS WAVELET SCATTERING TRANSFORM COEFFICIENTS ON THE CONVENTIONAL NEURAL NETWORK CLASSIFIER ACCURACY DURING THE COVID-19 DISEASE

Background and Objective: Regarding the Coronavirus disease-2019 (COVID-19) pandemic in past years and using medical images to detect it, the image processing of the lungs and enhancement of its quality are some of the challenges in the medical image processing field. As it sounds from previous studies, the lung image processing has been raised in the other lung diseases such as lung cancer, too. Thus, the accurate classifying between normal lung image and abnormal is a challenge to aid physicians. Methods: In this paper, we have proposed an image fusion technique to increase the accuracy of classifier. In this technique, some signal preprocessing tools like discrete wavelet transform (DWT), wavelet scattering transform (WST), and image fusion by using DWT are employed to enhance ordinary convolutional neural network (CNN) classifier accuracy. Results: Unlike other studies, in this paper, different aspects of an image are fused with itself to emphasize its information which may be neglected in a total assessment of the image. We have achieved 89.8% accuracy for very simple structure of CNN classifier without using proposed fusion, and when we used proposed methods, the classifier accuracy increased to 91.8%. Conclusions: This study reveals using efficient preprocessing and presenting input images which lead to decrease the complications of deep learning classifier, and increase its accuracy overall.

A Diagnostic Accuracy Study of Targeted and Systematic Biopsies to Detect Clinically Significant Prostate Cancer, including a Model for the Partial Omission of Systematic Biopsies.

The primary objective of this study was to analyse the current accuracy of targeted and systematic prostate biopsies in detecting csPCa. A secondary objective was to determine whether there are factors predicting the finding of csPCa in targeted biopsies and, if so, to explore the utility of a predictive model for csPCa detection only in targeted biopsies. We analysed 2122 men with suspected PCa, serum PSA > 3 ng/mL, and/or a suspicious digital rectal examination (DRE), who underwent targeted and systematic biopsies between 2021 and 2022. CsPCa (grade group 2 or higher) was detected in 1026 men (48.4%). Discrepancies in csPCa detection in targeted and systematic biopsies were observed in 49.6%, with 13.9% of csPCa cases being detected only in systematic biopsies and 35.7% only in targeted biopsies. A predictive model for csPCa detection only in targeted biopsies was developed from the independent predictors age (years), prostate volume (mL), PI-RADS score (3 to 5), mpMRI Tesla (1.5 vs. 3.0), TRUS-MRI fusion image technique (cognitive vs. software), and prostate biopsy route (transrectal vs. transperineal). The csPCa discrimination ability of targeted biopsies showed an AUC of 0.741 (95% CI 0.721-0.762). The avoidance rate of systematic prostate biopsies went from 0.5% without missing csPCa to 18.3% missing 4.6% of csPCa cases. We conclude that the csPCa diagnostic accuracy of targeted biopsies is higher than that of systematic biopsies. However, a significant rate of csPCa remains detected only in systematic biopsies. A predictive model for the partial omission of systematic biopsies was developed.

Robust multi-focus image fusion using focus property detection and deep image matting

Multi-focus image fusion techniques are designed to integrate the focused regions of multiple source images to produce an all-in-focus image. Unfortunately, the fusion results generated by existing methods are prone to suffer from chromatic aberrations and blurred boundary regions. In this paper, a multi-focus image fusion framework combining focus property detection and deep image matting is proposed to achieve high-quality fusion results. Specifically, we introduce spatial frequency detection blocks, which integrate the spatial frequency information of source images at different levels to achieve effective focus property detection for the initial decision. Then, the trimap acquired from the initial decision is optimized using a deep matting model so as to further detect the focused/defocused boundaries and obtain a more precise decision map. Meanwhile, a new hybrid loss function is introduced, and a large-scale dataset is created to improve the performance of the fusion framework. Finally, the fusion result is obtained by integrating the focused regions via the decision map. Compared to 18 other advanced fusion methods, the proposed method outperforms all of them in terms of the seven objective metrics evaluated on two publicly available datasets. These experimental results provide strong evidence that the proposed fusion framework excels not only in subjective vision but also in quantitative analysis. The code of the proposed method will be available at https://github.com/govenda/DIMF.

Laplacian Pyramid Fusion Network With Hierarchical Guidance for Infrared and Visible Image Fusion

The fusion of infrared and visible images combines the information from two complementary imaging modalities for various computer vision tasks. Many existing techniques, however, fail to maintain a uniform overall style and keep salient details of individual modalities simultaneously. This paper presents an end-to-end Laplacian Pyramid Fusion Network with hierarchical guidance (HG-LPFN) that takes advantage of pixel-level saliency reservation of Laplacian Pyramid and global optimization capability of deep learning. The proposed scheme generates hierarchical saliency maps through Laplacian Pyramid decomposition and modal difference calculation. In the pyramid fusion mode, all sub-networks are connected in a bottom-up manner. The sub-network for low-frequency fusion focuses on extracting universal features to produce an opposite style while sub-networks for high-frequency fusion determine how much the details of each modality will be retained. Taking the style, details, and background into consideration, we design a set of novel loss functions to supervise both low-frequency images and full-resolution images under the guidance of saliency maps. Experimental results on public datasets demonstrate that the proposed HG-LPFN outperforms the state-of-the-art image fusion techniques.

Image Fusion Techniques and Measurement to Assess the Image Quality

Image processing techniques focus upon enhancing the quality of an image or set of images and to derive maximum information from them, producing a superior quality image from a set of available images. Image Processing is a computer imaging where application involves a human being in the visual loop. Information fusion is a naturally occurring phenomenon in the most biological system. Data from various sources are merged in order to make optimal decision. In field of remote sensing, satellite image are captured in various frequency bands with different spatial, temporal and spectral reslutions. Remote sensing data is useful in various application from monitoring growth of vegetation to detection of geographical infiltration.

Medical Diagnosis with Multimodal Image Fusion Techniques

Image Fusion is an effective approach utilized to draw out all the significant information from the source images, which supports experts in evaluation and quick decision making. Multi modal medical image fusion produces a composite fused image utilizing various sources to improve quality and extract complementary information. It is extremely challenging to gather every piece of information needed using just one imaging method. Therefore, images obtained from different modalities are fused Additional clinical information can be gleaned through the fusion of several types of medical image pairings. This study's main aim is to present a thorough review of medical image fusion techniques which also covers steps in fusion process, levels of fusion, various imaging modalities with their pros and cons, and the major scientific difficulties encountered in the area of medical image fusion. This paper also summarizes the quality assessments fusion metrics. The various approaches used by image fusion algorithms that are presently available in the literature are classified into four broad categories i) Spatial fusion methods ii) Multiscale Decomposition based methods iii) Neural Network based methods and iv) Fuzzy Logic based methods. the benefits and pitfalls of the existing literature are explored and Future insights are suggested. Moreover, this study is anticipated to create a solid platform for the development of better fusion techniques in medical applications.

The significance of image fusion in nuclear medicine and molecular imaging

Nuclear medicine molecular imaging (NMMI) typically employs radioactive isotopes to label cells or molecules and then utilizes imaging devices such as positron emission tomography and single photon emission computed tomography to generate images. However, the images produced by these devices often suffer from problems such as signal noise, low resolution, and poor soft-organ contrast. To address these limitations, image fusion technology merges images from different imaging modalities, combining multiple types of medical image information obtained through various imaging techniques. This process generates a more comprehensive and accurate image, significantly improving image quality, reducing noise, and ultimately enhancing diagnostic accuracy and treatment effectiveness. Image fusion technology has found widespread applications in NMMI, achieving significant results in various fields. This review provides an overview of the development of image fusion technology, introduces traditional image fusion techniques, explores deep learning-based image fusion methods, and finally discusses the challenges and future directions of image fusion technology in NMMI.

Comprehensive performance analysis of different medical image fusion techniques for accurate healthcare diagnosis applications

Comprehensive performance analysis of different medical image fusion techniques for accurate healthcare diagnosis applications

Implementation and Comparative Study of Pyramid-based Image Fusion Techniques for Lumbar Spine Images

Image fusion is a method of combining the necessary and relevant information from the set of source images into a single (fused) image which can be deemed to be more informative than the source. This paper discusses the implementation of various pyramid-based image fusion algorithms, such as the Laplacian pyramid, the ratio of the low-pass pyramid, the contrast pyramid, and the filter subtract decimate pyramid on multimodal CT and MR images of the lumbar spine. The fused images were evaluated using various objective evaluation quality metrics. The experimental results demonstrated that the ratio of the low pass pyramid achieved better performance compared to the other pyramids implemented, indicating that the fused image can also be used for further image fusion application or analysis purposes.

Percutaneous Microwave Ablation of Hepatocellular Carcinoma with "Double Fusion" Technique: Technical Note and Single-Center Preliminary Experience.

Percutaneous image-guided thermal ablation is included in most society guidelines for treatment of hepatocellular carcinoma (HCC). The results of this treatment in terms of efficacy depend on the ability to precisely place the device into the target tumor. Ultrasound (US) is a commonly used imaging guidance modality for its real-time feedback. However, an accurate device deployment remains challenging in some clinical scenarios, including cases of tumors that are undetectable or not clearly visible by US. To overcome this problem, fusion imaging techniques have been developed, which combine images from different modalities. The most widely known technique combines pre-procedural contrast-enhanced computed tomography (CT) or magnetic resonance imaging (MRI) with real-time US scans. Cone beam CT (CBCT) is a technology that can provide intra-procedural cross-sectional images, which can be registered to images from other modalities, including preprocedural CT/MR scans. The aim of our study is to report the preliminary experience on percutaneous microwave ablation (MWA) of patients with HCC that were treated using the "double fusion" technique, which combines the use of US fusion imaging and CBCT fusion imaging. We describe the technical details, feasibility, safety and short-term efficacy of this technique in a small series of eight patients with 11 HCCs.

A Curvelet-Transform-Based Image Fusion Method Incorporating Side-Scan Sonar Image Features

Current methods of fusing side-scan sonar images fail to tackle the issues of shadow removal, preservation of information from adjacent strip images, and maintenance of image clarity and contrast. To address these deficiencies, a novel curvelet-transform-based approach that integrates the complementary attribute of details from side-scan sonar strip images is proposed. By capitalizing on the multiple scales and orientations of the curvelet transform and its intricate hierarchical nature, myriad fusion rules were applied at the corresponding frequency levels, enabling a more-tailored image fusion technique for side-scan sonar imagery. The experimental results validated the effectiveness of this method in preserving valuable information from side-scan sonar images, reducing the presence of shadows and ensuring both clarity and contrast in the fused images. By meeting the aforementioned challenges encountered in existing methodologies, this approach demonstrated great practical significance.

A Robust and Automated Vision-Based Human Fall Detection System Using 3D Multi-Stream CNNs with an Image Fusion Technique

Unintentional human falls, particularly in older adults, can result in severe injuries and death, and negatively impact quality of life. The World Health Organization (WHO) states that falls are a significant public health issue and the primary cause of injury-related fatalities worldwide. Injuries resulting from falls, such as broken bones, trauma, and internal injuries, can have severe consequences and can lead to a loss of mobility and independence. To address this problem, there have been suggestions to develop strategies to reduce the frequency of falls, in order to decrease healthcare costs and productivity loss. Vision-based fall detection approaches have proven their effectiveness in addressing falls on time, which can help to reduce fall injuries. This paper introduces an automated vision-based system for detecting falls and issuing instant alerts upon detection. The proposed system processes live footage from a monitoring surveillance camera by utilizing a fine-tuned human segmentation model and image fusion technique as pre-processing and classifying a set of live footage with a 3D multi-stream CNN model (4S-3DCNN). The system alerts when the sequence of the Falling of the monitored human, followed by having Fallen, takes place. The effectiveness of the system was assessed using the publicly available Le2i dataset. System validation revealed an impressive result, achieving an accuracy of 99.44%, sensitivity of 99.12%, specificity of 99.12%, and precision of 99.59%. Based on the reported results, the presented system can be a valuable tool for detecting human falls, preventing fall injury complications, and reducing healthcare and productivity loss costs.

A Lightweight Multimode Medical Image Fusion Method Using Similarity Measure Between Intuitionistic Fuzzy Sets Joint Laplacian Pyramid

Medical image fusion combines the multiple features of human tissue from different source images, which will be conducive to clinical diagnosis. Because of the human visual perception and mechanisms of medical imaging, abundant fuzzy features exist in medical image fusion with remarkable influence. To address this issue, we introduce the similarity measure of fuzzy set theory into medical image fusion to abstract and measure these fuzzy features. In this study, we present a new similarity measure of intuitionistic fuzzy set theory to represent image features and present a lightweight medical image fusion technique on the basis of this new measure. First, the new similarity measure is proposed according to geometric modeling technique and verified by mathematical justification. Second, a set of high frequency sub-images and a low frequency sub-image are produced by Laplacian pyramid decomposition to separate different image features. Third, our proposed similarity measure is utilized to describe and extract the detailed image features. Finally, two fusion rules and inverse Laplacian pyramid decomposition are sequentially used to get the fused image. This research shows that the similarity measure of fuzzy set theory can get excellent performance in medical image fusion. Experiments reveal that our similarity measure and medical image fusion method both have the superior performance to most of the existing methods.

Medical image fusion based on multi-scale decomposition using hybrid deep learning network model

ABSTRACT In medical applications, effective medical diagnosis is developed by combining medical images from various morphologies using medical image fusion techniques. An accurate diagnosis cannot be made from a single modality image, and the existing approaches need better efficiency due to poor image quality and inconsistent performance. To solve this problem, this research proposes an effective artificial intelligence model based on multi-modal medical image fusion. This research develops a new method based on medical images using deep residual neural networks (ResNet-50) and a DarkNet-19. The optimised discrete wavelet transform (ODWT) is used for image decomposing. The original image’s high- and low-frequency coefficients are decomposed by using ODWT. The low-frequency coefficient is then fused using the Modified ResNet-50 model. After that, DarkNet-19 fuses the high-frequency coefficients, and the input stimulus for DarkNet-19 is a high-frequency coefficient’s modified average gradient. The inverse ODWT is employed to reconstruct the fused image finally. The proposed fusion model’s effectiveness is assessed using the various CT-MRI, CT-PET, and MRI-SPECT imaging datasets. The proposed medical image fusion approach attains a maximum performance for PSNR of 40.03 dB, Fusion Factor of 6.1025, Fusion Symmetry of 0.0869, and Visual Information Fidelity of 0.121.

An ISAR and Visible Image Fusion Algorithm Based on Adaptive Guided Multi-Layer Side Window Box Filter Decomposition

Traditional image fusion techniques generally use symmetrical methods to extract features from different sources of images. However, these conventional approaches do not resolve the information domain discrepancy from multiple sources, resulting in the incompleteness of fusion. To solve the problem, we propose an asymmetric decomposition method. Firstly, an information abundance discrimination method is used to sort images into detailed and coarse categories. Then, different decomposition methods are proposed to extract features at different scales. Next, different fusion strategies are adopted for different scale features, including sum fusion, variance-based transformation, integrated fusion, and energy-based fusion. Finally, the fusion result is obtained through summation, retaining vital features from both images. Eight fusion metrics and two datasets containing registered visible, ISAR, and infrared images were adopted to evaluate the performance of the proposed method. The experimental results demonstrate that the proposed asymmetric decomposition method could preserve more details than the symmetric one, and performed better in both objective and subjective evaluations compared with the fifteen state-of-the-art fusion methods. These findings can inspire researchers to consider a new asymmetric fusion framework that can adapt to the differences in information richness of the images, and promote the development of fusion technology.

Interval type-2 fuzzy set based block-SBU for image fusion technique

Image fusion is the process of examining one or more obscure images, combining only the critical information in those images to make them a good quality image. Due to the large gap between the high dynamic range of landscape scenes and the low quality of consumer quality cameras, a single-shot image cannot record the information of a single set. A part of image processing called image fusion is used to overcome the above problem. In this method, authors propose a new technology for image fusion with intuitionistic fuzzy sets. During the processing, the given images are converted into fuzzy images and then intuitionistic fuzzy images (IFIs). Thus, a significant change occurs between the given image and IFIs. The resulting IFIs are converted into interval type-2 fuzzy images (IT2FIs) to overcome this problem. The proposed technique is compared to other procedures such as discrete cosine harmonic wavelet transform (DCHWT), multi-resolution single value decay (MSVD), primary component analysis (PCA), standard wavelet transform (SWT), two scale image fusion (TSIF), and wavelet transform (WT). The proposed method gives the best results based on performance analysis such as Entropy, standard deviation (STD), average gradient (AG), mutual information (MIF), and blind referenceless image spatial quality evaluator (BRISQUE). Using the proposed technique, one can obtain the best results for entropy, STD, AG, MIF, and BRISQUE respectively as 7.6241, 49.6745, 69.2100, 5.7276, and 3.3091. Hence it reveals that the proposed method performs better than other methods regarding overall visual quality and performance measurements.

Assessment of long-term mangrove distribution using optimised machine learning algorithms and landscape pattern analysis.

Mangrove ecosystems provide numerous benefits, including carbon storage, coastal protection and food for marine organisms. However, mapping and monitoring of mangrove status in some regions, such as the Red Sea area, has been hindered by a lack of data, accurate and precise maps and technical expertise. In this study, an advanced machine learning algorithm was proposed to produce an accurate and precise high-resolution land use map that includes mangroves in the Al Wajh Bank habitat in northeastern Saudi Arabia. To achieve this, high-resolution multispectral images were generated using an image fusion technique, and machine learning algorithms were applied, including artificial neural networks, random forests and support vector machine algorithms. The performance of the models was evaluated using various matrices, and changes in mangrove distribution and connectivity were assessed using the landscape fragmentation model and Getis-Ord statistics. The research gap that this study aims to address is the lack of accurate and precise mapping and assessment of mangrove status in the Red Sea area, particularly in data-scarce regions. Our study produced high-resolution mobile laser scanning (MLS) imagery of 15-m length for 2014 and 2022, and trained 5, 6 and 9 models for artificial neural networks, support vector machines and random forests (RF) to predict land use and land cover maps using 15-m and 30-m resolution MLS images. The best models were identified using error matrices, and it was found that RF outperformed other models. According to the 15-m resolution map of 2022 and the best models of RF, the mangrove cover in the Al Wajh Bank is 27.6 km2, which increased to 34.99 km2 in the case of the 30-m resolution image of 2022, and was 11.94 km2 in 2014, indicating a doubling of the mangrove area. Landscape structure analysis revealed an increase in small core and hotspot areas, which were converted into medium core and very large hotspot areas in 2014. New mangrove areas were identified in the form of patches, edges, potholes and coldspots. The connectivity model showed an increase in connectivity over time, promoting biodiversity. Our study contributes to the promotion of the protection, conservation and planting of mangroves in the Red Sea area.

Image fusion technique using flat panel detector rotational angiography for transvenous embolization of intracranial dural arteriovenous fistula.

Precise evaluation of the feeders, fistulous points, and draining veins plays a key role for successful embolization of intracranial dural arteriovenous fistulas (DAVF). Digital subtraction angiography (DSA) is a gold standard diagnostic tool to assess the exact angioarchitecture of DAVFs. With the advent of new image postprocessing techniques, we lately have been able to apply image fusion techniques with two different image sets obtained with flat panel detector rotational angiography. This new technique can provide additional and better pretherapeutic information of DAVFs over the conventional 2D and 3D angiographies. In addition, it can be used during the endovascular treatment to help the accurate and precise navigation of the microcatheter and microguidwire inside the vessels and identify the proper location of microcatheter in the targeted shunting pouch. In this study, we briefly review the process of an image fusion technique and introduce our clinical application for treating DAVFs, especially focused on the transvenous embolization.