Image Modalities Research Articles

SSTar-TransGAN: Transformer-based Unsupervised Multi-modality Hyperspectral Digital Staining Network

Immunohistochemistry (IHC) plays an important role in accurate cancer screening and diagnosis, the clinical application of which has been restricted by complex operation process, high costs and demand of professional skills from pathologists. Digital staining methods based on deep learning provide the possibility for Hematoxylin & Eosin (H&E) stained images to be converted into IHC stained images, but it needs to train multiple staining networks for various modalities of IHC images from different cancers, thus weakening the versatility and convenience of digital staining in the process of practical application. At the same time, microscopic hyperspectral imaging technology can provide abundant spectral information for pathological images, which has been proved effective in digital staining tasks to transform staining modalities from one to many, but microscopic hyperspectral imaging has also been troubled by time-consuming acquisition process and huge data storage. In order to overcome the above challenges, we propose SSTar-TransGAN network for digital staining tasks. With the inhabitation of StarGAN structure, SSTar-TransGAN transfers the training burden of generators into lightweight style encoders between different modalities, while in addition to the introduction of transformer structure into the encoder, the application of spectral super-resolution Swin-Spectral Transformer U-Net (SSTU) network enables the convertion from H&E stained RGB images into hyperspectral images as well, which ensure the spatial structure and color information of IHC images under multiple staining modalities and the further conversion between different staining modalities. Qualitative and quantitative experiments prove the performance of SSTar-TransGAN on digital staining tasks superior to other state-of-the-art digital staining methods.

Medical Image Segmentation Review: The Success of U-Net.

Automatic medical image segmentation is a crucial topic in the medical domain and successively a critical counterpart in the computer-aided diagnosis paradigm. U-Net is the most widespread image segmentation architecture due to its flexibility, optimized modular design, and success in all medical image modalities. Over the years, the U-Net model has received tremendous attention from academic and industrial researchers who have extended it to address the scale and complexity created by medical tasks. These extensions are commonly related to enhancing the U-Net's backbone, bottleneck, or skip connections, or including representation learning, or combining it with a Transformer architecture, or even addressing probabilistic prediction of the segmentation map. Having a compendium of different previously proposed U-Net variants makes it easier for machine learning researchers to identify relevant research questions and understand the challenges of the biological tasks that challenge the model. In this work, we discuss the practical aspects of the U-Net model and organize each variant model into a taxonomy. Moreover, to measure the performance of these strategies in a clinical application, we propose fair evaluations of some unique and famous designs on well-known datasets. Furthermore, we provide a comprehensive implementation library with trained models. In addition, for ease of future studies, we created an online list of U-Net papers with their possible official implementation.

A Decade of Prostate Intervention Robots: Systematic Review

Abstract: The image-guided prostate intervention robot can assist surgeons in performing minimally invasive intervention procedures. It has the advantage of overcoming the disadvantages of a traditional surgeon's hand tremor and dependence on the need for an experienced surgeon, improving the intervention precision, and reducing the hazards caused by normal soft tissues due to surgical precision errors compared to traditional intervention procedures. In the design of prostate intervention robots, structural design, materials, actuators, and compatibility issues should be considered. Based on the above considerations, a representative literature of the last decade was selected for review. Patents and articles on image-guided prostate intervention robots published in the last 10 years (2013-2023)were searched in several electronic databases, focusing on keywords (structural design, puncture accuracy, image compatibility, actuator) for screening. We retrieved a total of 26 prostate intervention robots guided by different images, classified them by US, MRI, MRI-US and CT, selected representative robots guided by different image modalities for review and tabulated them for comparison, and finally boldly predicted that the future direction will be prostate robots guided by MRI-US images. Image-guided prostate intervention robots can improve the deficiencies of conventional intervention procedures. In the future how to improve the intervention accuracy will be the primary problem of all intervention robots, through structural design, materials, actuators, and image compatibility issues have been improved.

OmniSegger: A time-lapse image analysis pipeline for bacterial cells.

Time-lapse microscopy is a powerful tool for studying the cell biology of bacterial cells. The development of pipelines that facilitate the automated analysis of these datasets is a long-standing goal of the field. In this paper, we describe OmniSegger , an updated version of our SuperSegger pipeline, developed as an open-source, modular, and holistic suite of algorithms whose input is raw microscopy images and whose output is a wide range of quantitative cellular analyses, including dynamical cell cytometry data and cellular visualizations. The updated version described in this paper introduces two principal refinements: (i) robustness to cell morphologies and (ii) support for a range of common imaging modalities. To demonstrate robustness to cell morphology, we present an analysis of the proliferation dynamics of Escherichia coli treated with a drug that induces filamentation. To demonstrate extended support for new image modalities, we analyze cells imaged by five distinct modalities: phase-contrast, two brightfield modalities, and cytoplasmic and membrane fluorescence. Together, this pipeline should greatly increase the scope of tractable analyses for bacterial microscopists.

Analysis and identification of nocturnal groaning syndrome based on multimodal data

Nocturnal groaning syndrome is a common sleep disorder characterized by irregular groaning or vocalizations during nighttime sleep, representing a significant area of research in sleep disorders. Nocturnal groaning syndrome is a common sleep disorder characterized by irregular groaning or vocalizations during nighttime sleep, representing a significant area of research in sleep disorders. proposes a multimodal recognition approach based on speech, image, and text modalities. The study analyzes audio features using Mel Frequency Cepstral Coefficients (MFCC), which is the most common method for identifying nocturnal groaning syndrome. Coefficients (MFCC), extracts image features with pretrained MobileNetV2, and identifies key physiological signals from text using TF-IDF algorithm. Subsequently, Multimodal Compact Bilinear Pooling (MCB) is employed to fuse audio and image features, and a Text-Image CNN is used to combine image and text features. Support Vector Machine (SVM) is then used to classify the fused multimodal features, and decision-level fusion is performed using weighting criteria. Experimental results demonstrate an identification accuracy of 89.5% on the test set, significantly enhancing the auxiliary diagnostic effectiveness of nocturnurnal diagnosis. Experimental results demonstrate an identification accuracy of 89.5% on the test set, significantly enhancing the auxiliary diagnostic effectiveness of nocturnal groaning syndrome.

Identification of mild cognitive impairment using multimodal 3D imaging data and graph convolutional networks

Objective.Mild cognitive impairment (MCI) is a precursor stage of dementia characterized by mild cognitive decline in one or more cognitive domains, without meeting the criteria for dementia. MCI is considered a prodromal form of Alzheimer's disease (AD). Early identification of MCI is crucial for both intervention and prevention of AD. To accurately identify MCI, a novel multimodal 3D imaging data integration graph convolutional network (GCN) model is designed in this paper.Approach.The proposed model utilizes 3D-VGGNet to extract three-dimensional features from multimodal imaging data (such as structural magnetic resonance imaging and fluorodeoxyglucose positron emission tomography), which are then fused into feature vectors as the node features of a population graph. Non-imaging features of participants are combined with the multimodal imaging data to construct a population sparse graph. Additionally, in order to optimize the connectivity of the graph, we employed the pairwise attribute estimation （PAE） method to compute the edge weights based on non-imaging data, thereby enhancing the effectiveness of the graph structure. Subsequently, a population-based GCN integrates the structural and functional features of different modal images into the features of each participant for MCI classification.Main results.Experiments on the AD Neuroimaging Initiative demonstrated accuracies of 98.57%, 96.03%, and 96.83% for the normal controls (NC)-early MCI (EMCI), NC-late MCI (LMCI), and EMCI-LMCI classification tasks, respectively. The AUC, specificity, sensitivity, and F1-score are also superior to state-of-the-art models, demonstrating the effectiveness of the proposed model. Furthermore, the proposed model is applied to the ABIDE dataset for autism diagnosis, achieving an accuracy of 91.43% and outperforming the state-of-the-art models, indicating excellent generalization capabilities of the proposed model.Significance.This study demonstratesthe proposed model's ability to integrate multimodal imaging data and its excellent ability to recognize MCI. This will help achieve early warning for AD and intelligent diagnosis of other brain neurodegenerative diseases.

Intermediary-Generated Bridge Network for RGB-D Cross-Modal Re-Identification

RGB-D cross-modal person re-identification (re-id) targets at retrieving the person of interest across RGB and depth image modalities. To cope with the modal discrepancy, some existing methods generate an auxiliary mode with either inherent properties of input modes or extra deep networks. However, such useful intermediary role included in generated mode is often overlooked in these approaches, leading to insufficient exploitation of crucial bridge knowledge. By contrast, in this article, we propose a novel approach that constructs an intermediary mode through the constraints of self-supervised intermediary learning, which is freedom from modal prior knowledge and additional module parameters. We then design a bridge network to fully mine the intermediary role of generated modality through carrying out multi-modal integration and decomposition. For one thing, this network leverages a multi-modal transformer to integrate the information of three modes via fully exploiting their heterogeneous relations with the intermediary mode as the bridge. It conducts the identification consistency constraint to promote cross-modal associations. For another, it employs circle contrastive learning to decompose the cross-modal constraint process into several subprocedures, which provides the intermediate relay during pulling two original modalities closer. Experiments on two public datasets demonstrate that the proposed method exceeds the state-of-the-arts. The effectiveness of each component in this method is verified through numerous ablation studies. Additionally, we have demonstrated the generalization ability of the proposed method through experiments.

A foundation model for joint segmentation, detection and recognition of biomedical objects across nine modalities.

Biomedical image analysis is fundamental for biomedical discovery. Holistic image analysis comprises interdependent subtasks such as segmentation, detection and recognition, which are tackled separately by traditional approaches. Here, we propose BiomedParse, a biomedical foundation model that can jointly conduct segmentation, detection and recognition across nine imaging modalities. This joint learning improves the accuracy for individual tasks and enables new applications such as segmenting all relevant objects in an image through a textual description. To train BiomedParse, we created a large dataset comprising over 6 million triples of image, segmentation mask and textual description by leveraging natural language labels or descriptions accompanying existing datasets. We showed that BiomedParse outperformed existing methods on image segmentation across nine imaging modalities, with larger improvement on objects with irregular shapes. We further showed that BiomedParse can simultaneously segment and label all objects in an image. In summary, BiomedParse is an all-in-one tool for biomedical image analysis on all major image modalities, paving the path for efficient and accurate image-based biomedical discovery.

Enhanced breast cancer diagnosis through integration of computer vision with fusion based joint transfer learning using multi modality medical images

Breast cancer (BC) is a type of cancer which progresses and spreads from breast tissues and gradually exceeds the entire body; this kind of cancer originates in both sexes. Prompt recognition of this disorder is most significant in this phase, and it is measured by providing patients with the essential treatment so their efficient lifetime can be protected. Scientists and researchers in numerous studies have initiated techniques to identify tumours in early phases. Still, misperception in classifying skeptical lesions can be due to poor image excellence and dissimilar breast density. BC is a primary health concern, requiring constant initial detection and improvement in analysis. BC analysis has made major progress recently with combining multi-modal image modalities. These studies deliver an overview of the segmentation, classification, or grading of numerous cancer types, including BC, by employing conventional machine learning (ML) models over hand-engineered features. Therefore, this study uses multi-modality medical imaging to propose a Computer Vision with Fusion Joint Transfer Learning for Breast Cancer Diagnosis (CVFBJTL-BCD) technique. The presented CVFBJTL-BCD technique utilizes feature fusion and DL models to effectively detect and identify BC diagnoses. The CVFBJTL-BCD technique primarily employs the Gabor filtering (GF) technique for noise removal. Next, the CVFBJTL-BCD technique uses a fusion-based joint transfer learning (TL) process comprising three models, namely DenseNet201, InceptionV3, and MobileNetV2. The stacked autoencoders (SAE) model is implemented to classify BC diagnosis. Finally, the horse herd optimization algorithm (HHOA) model is utilized to select parameters involved in the SAE method optimally. To demonstrate the improved results of the CVFBJTL-BCD methodology, a comprehensive series of experimentations are performed on two benchmark datasets. The comparative analysis of the CVFBJTL-BCD technique portrayed a superior accuracy value of 98.18% and 99.15% over existing methods under Histopathological and Ultrasound datasets.

Automated image registration of RGB, hyperspectral and chlorophyll fluorescence imaging data

BackgroundThe early and specific detection of abiotic and biotic stresses, particularly their combinations, is a major challenge for maintaining and increasing plant productivity in sustainable agriculture under changing environmental conditions. Optical imaging techniques enable cost-efficient and non-destructive quantification of plant stress states. Monomodal detection of certain stressors is usually based on non-specific/indirect features and therefore is commonly limited in their cross-specificity to other stressors. The fusion of multi-domain sensor systems can provide more potentially discriminative features for machine learning models and potentially provide synergistic information to increase cross-specificity in plant disease detection when image data are fused at the pixel level.ResultsIn this study, we demonstrate successful multi-modal image registration of RGB, hyperspectral (HSI) and chlorophyll fluorescence (ChlF) kinetics data at the pixel level for high-throughput phenotyping of A. thaliana grown in Multi-well plates and an assay with detached leaf discs of Rosa × hybrida inoculated with the black spot disease-inducing fungus Diplocarpon rosae. Here, we showcase the effects of (i) selection of reference image selection, (ii) different registrations methods and (iii) frame selection on the performance of image registration via affine transform. In addition, we developed a combined approach for registration methods through NCC-based selection for each file, resulting in a robust and accurate approach that sacrifices computational time. Since image data encompass multiple objects, the initial coarse image registration using a global transformation matrix exhibited heterogeneity across different image regions. By employing an additional fine registration on the object-separated image data, we achieved a high overlap ratio. Specifically, for the A. thaliana test set, the overlap ratios (ORConvex) were 98.0 ± 2.3% for RGB-to-ChlF and 96.6 ± 4.2% for HSI-to-ChlF. For the Rosa × hybrida test set, the values were 98.9 ± 0.5% for RGB-to-ChlF and 98.3 ± 1.3% for HSI-to-ChlF.ConclusionThe presented multi-modal imaging pipeline enables high-throughput, high-dimensional phenotyping of different plant species with respect to various biotic or abiotic stressors. This paves the way for in-depth studies investigating the correlative relationships of the multi-domain data or the performance enhancement of machine learning models via multi modal image fusion.

Modeling inter-modal incongruous sentiment expressions for multi-modal sarcasm detection

Multi-modal sarcasm detection (MSD) presents a formidable and intricate endeavor. Despite strides made by extant models, two principal hurdles persist: Firstly, prevailing methodologies merely address superficial disparities between textual inputs and associated images, neglecting nuanced inter-modal combinations. Secondly, satirical instances frequently involve intricate emotional expressions, highlighting the imperative of leveraging emotional cues across modalities to discern satirical nuances. Accordingly, this research proposes the utilization of a deep graph convolutional network that integrates cross-modal mapping information to effectively identify significant incongruent sentiment expressions across various modalities for the purpose of multi-modal sarcasm detection. Specifically, we first design a cross-modal mapping network, which obtains the interaction information between these two modalities by mapping text feature vectors and image feature vectors two by two to compensate for the lack of multi-modal data in the fusion process. Additionally, we employ external knowledge of ANPS as a bridge to construct cross-correlation graphs through highly correlated satirical cues and their connection weights between image and text modalities. Afterward, the GCN architecture with retrieval-based attentional mechanisms will effectively capture satirical cues. The experiments conducted on two publicly available datasets demonstrate a significant enhancement in the performance of our method when compared to numerous contemporary models.

High-quality multimodal MRI with simultaneous EEG using conductive ink and polymer-thick film nets

Objective. Combining magnetic resonance imaging (MRI) and electroencephalography (EEG) provides a powerful tool for investigating brain function at varying spatial and temporal scales. Simultaneous acquisition of both modalities can provide unique information that a single modality alone cannot reveal. However, current simultaneous EEG-fMRI studies are limited to a small set of MRI sequences due to the image quality and safety limitations of commercially available MR-conditional EEG nets. We tested whether the Inknet2, a high-resistance polymer thick film based EEG net that uses conductive ink, could enable the acquisition of a variety of MR image modalities with minimal artifacts by reducing the radiofrequency-shielding caused by traditional MR-conditional nets.Approach. We first performed simulations to model the effect of the EEG nets on the magnetic field and image quality. We then performed phantom scans to test image quality with a conventional copper EEG net, with the new Inknet2, and without any EEG net. Finally, we scanned five human subjects at 3 Tesla (3 T) and three human subjects at 7 Tesla (7 T) with and without the Inknet2 to assess structural and functional MRI image quality.Main results. Across these simulations, phantom scans, and human studies, the Inknet2 induced fewer artifacts than the conventional net and produced image quality similar to scans with no net present.Significance. Our results demonstrate that high-quality structural and functional multimodal imaging across a variety of MRI pulse sequences at both 3 T and 7 T is achievable with an EEG net made with conductive ink and polymer thick film technology.

Adaptive Weighted Multiview Kernel Matrix Factorization and Its Application in Alzheimer’s Disease Analysis

Recent technology and equipment advancements have provided us with opportunities to better analyze Alzheimer’s disease (AD), where we could collect and employ the data from different image and genetic modalities that may potentially enhance the predictive performance. To perform better clustering in AD analysis, in this paper, we propose a novel model to leverage data from all different modalities/views, which can learn the weights of each view adaptively. Different from previous vanilla Non-negative matrix factorization which assumes data is linearly separable, we propose a simple yet efficient method based on kernel matrix factorization, which is not only able to deal with non-linear data structure but also can achieve better prediction accuracy. Experimental results on the ADNI dataset demonstrate the effectiveness of our proposed method, which indicates promising prospects for kernel application in AD analysis.

Field, mineralogical and petrographic features of the Miocene lava around Sağlık and Yatağan area, western Konya/Türkiye

Extensive lava domes occur west of Konya as a component of subduction-related Neogene Erenlerdağı volcanism. These light grey to grey-coloured domes commonly give rise to topographic elevations and contain some Mafic Microcrystalline Enclaves (MME) with a well-developed chilled zone, up to 4 mm thick. Petrographical and modal image analysis show that the lava samples consist predominantly of plagioclase (andesine, 8-46%, 0.11-4.3 mm), amphibole (3-17%, 0.14-1.613 mm), clinopyroxene (0-14%, 0.035-1.845 mm), biotite (3-12%, 0.09-2.30 mm), epidote (0-8%, 0.078-0.166 mm), piemontite (0-3%, 0.145-0.562 mm), quartz (0-6%, 0.4-2.0 mm), sanidine (0-5%, 0.10-0.17 mm), and opaque iron ore (3-43%), along with accessory minerals apatite and zircon, in various textures, including holocrystalline porphyritic, hypocrystalline porphyritic, glomeroporphyritic, and synneusis. The chilled zone is characterised by phenocrysts of plagioclase (25%) and amphibole (5%) in a holocrystalline porphyritic texture. The amphibole shows opacification and calcitization. The matrix includes plagioclase, amphibole (0.3-0.4 mm), epidote, opaque minerals and rare volcanic glass. The petrographical study suggests that the lava likely experienced mixing or mingling processes during the replenishment of felsic magma by mafic magma, potentially triggering a volcanic eruption. The crystallisation of skeletal and acicular microphenocrysts of plagioclase and acicular apatite indicates rapid crystallisation under undercooled conditions.

GeSeNet: A General Semantic-Guided Network With Couple Mask Ensemble for Medical Image Fusion.

At present, multimodal medical image fusion technology has become an essential means for researchers and doctors to predict diseases and study pathology. Nevertheless, how to reserve more unique features from different modal source images on the premise of ensuring time efficiency is a tricky problem. To handle this issue, we propose a flexible semantic-guided architecture with a mask-optimized framework in an end-to-end manner, termed as GeSeNet. Specifically, a region mask module is devised to deepen the learning of important information while pruning redundant computation for reducing the runtime. An edge enhancement module and a global refinement module are presented to modify the extracted features for boosting the edge textures and adjusting overall visual performance. In addition, we introduce a semantic module that is cascaded with the proposed fusion network to deliver semantic information into our generated results. Sufficient qualitative and quantitative comparative experiments (i.e., MRI-CT, MRI-PET, and MRI-SPECT) are deployed between our proposed method and ten state-of-the-art methods, which shows our generated images lead the way. Moreover, we also conduct operational efficiency comparisons and ablation experiments to prove that our proposed method can perform excellently in the field of multimodal medical image fusion. The code is available at https://github.com/lok-18/GeSeNet.

Accelerating Brain MR Imaging With Multisequence and Convolutional Neural Networks.

Magnetic resonance imaging (MRI) refers to one of the critical image modalities for diagnosis, whereas its long acquisition time limits its application. In this study, the aim was to investigate whether deep learning-based techniques are capable of using the common information in different MRI sequences to reduce the scan time of the most time-consuming sequences while maintaining the image quality. Fully sampled T1-FLAIR, T2-FLAIR, and T2WI brain MRI raw data originated from 217 patients and 105 healthy subjects. The T1-FLAIR and T2-FLAIR sequences were subsampled using Cartesian masks based on four different acceleration factors. The fully sampled T1/T2-FLAIR images were predicted from undersampled T1/T2-FLAIR images and T2WI images through deep learning-based reconstruction. They were qualitatively assessed by two senior radiologists in accordance with the diagnosis decision and a four-point scale image quality score. Furthermore, the images were quantitatively assessed based on regional signal-to-noise ratios (SNRs) and contrast-to-noise ratios (CNRs). The chi-square test was performed, where p<0.05 indicated a difference with statistical significance. The diagnosis decisions from two senior radiologists remained unchanged in accordance with the accelerated and fully sampled images. There were no significant differences in the regional SNRs and CNRs of most assessed regions (p>0.05) between the accelerated and fully sampled images. Moreover, no significant difference was identified in the image quality assessed by two senior radiologists (p>0.05). Deep learning-based image reconstruction is capable of significantly expediting the brain MR imaging process and producing acceptable image quality without affecting diagnosis decisions.

사전학습 모델 기반 발화 동영상 멀티 모달 감정 인식

Recently, as the demand for non-face-to-face counseling has rapidly increased, the need for emotion recognition technology that combines various aspects such as text, voice, and facial expressions is being emphasized. In this paper, we address issues such as the dominance of non-Korean data and the imbalance of emotion labels in existing datasets like FER-2013, CK+, and AFEW by using Korean video data. We propose methods to enhance multimodal emotion recognition performance in videos by integrating the strengths of image modality with text modality. A pre-trained model is used to overcome the limitations caused by small training data. A GPT-4-based LLM model is applied to text, and a pre-trained model based on VGG-19 architecture is fine-tuned to facial expression images. The method of extracting representative emotions by combining the emotional results of each aspect extracted using a pre-trained model is as follows. Emotion information extracted from text was combined with facial expression changes in a video. If there was a sentiment mismatch between the text and the image, we applied a threshold that prioritized the text-based sentiment if it was deemed trustworthy. Additionally, as a result of adjusting representative emotions using emotion distribution information for each frame, performance was improved by 19% based on F1-Score compared to the existing method that used average emotion values for each frame.

Advancing healthcare through multimodal data fusion: a comprehensive review of techniques and applications.

With the increasing availability of diverse healthcare data sources, such as medical images and electronic health records, there is a growing need to effectively integrate and fuse this multimodal data for comprehensive analysis and decision-making. However, despite its potential, multimodal data fusion in healthcare remains limited. This review paper provides an overview of existing literature on multimodal data fusion in healthcare, covering 69 relevant works published between 2018 and 2024. It focuses on methodologies that integrate different data types to enhance medical analysis, including techniques for integrating medical images with structured and unstructured data, combining multiple image modalities, and other features. Additionally, the paper reviews various approaches to multimodal data fusion, such as early, intermediate, and late fusion methods, and examines the challenges and limitations associated with these techniques. The potential benefits and applications of multimodal data fusion in various diseases are highlighted, illustrating specific strategies employed in healthcare artificial intelligence (AI) model development. This research synthesizes existing information to facilitate progress in using multimodal data for improved medical diagnosis and treatment planning.

Survey on Breast Cancer analysis

Breast cancer constitutes 12% of all newly diagnosed cases globally, making it the most predominant type of cancer as a whole. Histopathology images, among all the medical image modalities, maintain the cancer's path and provide richer phenotypic relevant information. The patient's immune system is progressively predictable as a critical feature in defining the suitable treatment. Tumor-infiltrating lymphocytes (TILs), immune cells found within tumors, are emerging as key biomarkers in breast cancer. The detection of lymph node metastasis affects the management of patients with primary breast cancer significantly in terms of staging, treatment, and prognosis.Tumoour infiltrating lymphocytes score and lymph node metastasis identification plays a very important role in staging of breast cancer.This review aims to provide a comprehensive overview on how deep learning-based method is used to compute the Tumour-Infiltrating Lymphocytes (TILs) score and lymph node metastasis from breast cancer histopathological images.

An ensemble deep learning model for medical image fusion with Siamese neural networks and VGG-19.

Multimodal medical image fusion methods, which combine complementary information from many multi-modality medical images, are among the most important and practical approaches in numerous clinical applications. Various conventional image fusion techniques have been developed for multimodality image fusion. Complex procedures for weight map computing, fixed fusion strategy and lack of contextual understanding remain difficult in conventional and machine learning approaches, usually resulting in artefacts that degrade the image quality. This work proposes an efficient hybrid learning model for medical image fusion using pre-trained and non-pre-trained networks i.e. VGG-19 and SNN with stacking ensemble method. The model leveraging the unique capabilities of each architecture, can effectively preserve the detailed information with high visual quality, for numerous combinations of image modalities in image fusion challenges, notably improved contrast, increased resolution, and lower artefacts. Additionally, this ensemble model can be more robust in the fusion of various combinations of source images that are publicly available from Havard-Medical-Image-Fusion Datasets, GitHub. and Kaggle. Our proposed model performance is superior in terms of visual quality and performance metrics to that of the existing fusion methods in literature like PCA+DTCWT, NSCT, DWT, DTCWT+NSCT, GADCT, CNN and VGG-19.