Multimodal Medical Research Articles

Advancing language concordant care: a multimodal medical interpretation intervention

BackgroundEnsuring language concordant care through medical interpretation services (MIS) allows for accurate information sharing and positive healthcare experiences. The COVID-19 pandemic led to a regional halt of in-person interpreters, leaving...

Multimodal Medical Image Fusion Network Based on Target Information Enhancement

Multimodal Medical Image Fusion Network Based on Target Information Enhancement

Retracted: Multimodal Medical Image Fusion of Positron Emission Tomography and Magnetic Resonance Imaging Using Generative Adversarial Networks.

[This retracts the article DOI: 10.1155/2022/6878783.].

Multimodal Medical Image Fusion Utilizing Two-scale Image Decomposition via Saliency Detection.

Modern medical imaging modalities used by clinicians have many applications in the diagnosis of complicated diseases. These imaging technologies reveal the internal anatomy and physiology of the body. The fundamental idea behind medical image fusion is to increase the image's global and local contrast, enhance the visual impact, and change its format so that it is better suited for computer processing or human viewing while preventing noise magnification and accomplishing excellent real-time performance. The top goal is to combine data from various modal images (CT/MRI and MR-T1/MR-T2) into a solitary image that, to the greatest degree possible, retains the key characteristics (prominent features) of the source images. The clinical accuracy of medical issues is compromised because innumerable classical fusion methods struggle to conserve all the prominent features of the original images. Furthermore, complex implementation, high computation time, and more memory requirements are key problems of transform domain methods. With the purpose of solving these problems, this research suggests a fusion framework for multimodal medical images that makes use of a multi-scale edge-preserving filter and visual saliency detection. The source images are decomposed using a two-scale edge-preserving filter into base and detail layers. Base layers are combined using the addition fusion rule, while detail layers are fused using weight maps constructed using the maximum symmetric surround saliency detection algorithm. The resultant image constructed by the presumed method has improved objective evaluation metrics than other classical methods, as well as unhindered edge contour, more global contrast, and no ringing effect or artifacts. The methodology offers a dominant and symbiotic arsenal of clinical symptomatic, therapeutic, and biomedical research competencies that have the prospective to considerably strengthen medical practice and biological understanding.

New Approach for Multimodal Medical Image Fusion using PDE-based Technique

Accurate diagnosis of diseases is essential for effective medical treatment, and medical image fusion plays a crucial role in achieving this goal. This article proposes a novel technique for multimodal medical image fusion based on fourth order PDE-based techniques. The proposed technique uses inputted CT and brain MRI images, and applies a PDE-based technique for noise reduction and edge preservation. Subsequently, a computed fusion rule using principal component analysis is applied to generate a fused image of the brain that presents clear and essential components for medical diagnosis and analysis. The article provides a comparative study of the proposed technique with other existing techniques, demonstrating its optimal performance. The proposed technique can significantly enhance medical image diagnosis and analysis for accurate disease detection and treatment.

F-DARTS: Foveated Differentiable Architecture Search Based Multimodal Medical Image Fusion.

Multimodal medical image fusion (MMIF) is highly significant in such fields as disease diagnosis and treatment. The traditional MMIF methods are difficult to provide satisfactory fusion accuracy and robustness due to the influence of such possible human-crafted components as image transform and fusion strategies. Existing deep learning based fusion methods are generally difficult to ensure image fusion effect due to the adoption of a human-designed network structure and a relatively simple loss function and the ignorance of human visual characteristics during weight learning. To address these issues, we have presented the foveated differentiable architecture search (F-DARTS) based unsupervised MMIF method. In this method, the foveation operator is introduced into the weight learning process to fully explore human visual characteristics for the effective image fusion. Meanwhile, a distinctive unsupervised loss function is designed for network training by integrating mutual information, sum of the correlations of differences, structural similarity and edge preservation value. Based on the presented foveation operator and loss function, an end-to-end encoder-decoder network architecture will be searched using the F-DARTS to produce the fused image. Experimental results on three multimodal medical image datasets demonstrate that the F-DARTS performs better than several traditional and deep learning based fusion methods by providing visually superior fused results and better objective evaluation metrics.

Multimodality medical image fusion analysis with multi-plane features of PET and MRI images using ONSCT

ABSTRACT The Multimodal Medical Image Fusion (MMIF) is affected by poor image quality, which leads to the extraction of inefficient features. The main intent of this work is to fuse various planes in the PET and MRI medical images efficiently using the MMIF approach. Initially, the sample images containing the axial plane of PET and MRI images are aggregated from standard datasets. Then, the collected images are employed for the decomposition process, which is accomplished via Optimal Non-Subsampled Contourlet Transform (ONSCT). The parameters in the NSCT are optimized using the Modified Water Strider Algorithm (MWSA. Once the images are decomposed, it is segmented into two sub-bands as high frequency and low-frequency sub-bands. Consequently, the high-frequency sub-bands of both PET and MRI images are fused by using the optimal weighted average fusion, in which the weight factor is obtained optimally by the MWSA algorithm. Similarly, the low-frequency sub-bands of both medical images are combined by sparse fusion technique. Finally, both the resultant fused images are subjected to Inverse Non-Subsampled Contourlet Transform (INSCT) to get desired fused images. The experimental findings suggest that the proposed model has effectively fused the images, and it also enhances the similarity score with axial planes.

Hierarchical Progressive Network for Multimodal Medical Image Fusion in Healthcare Systems

Deep learning (DL)-based multisource information processing plays an essential role in the Internet of Medical Things (IoMT). In this field, medical image fusion integrates scan results from different devices, supporting healthcare systems to make a more informed diagnosis. This study proposes a DL-based network to fuse multimodal medical images. In our method, the lattice unit (LU) is designed to improve the representation capability of the fusion network. Moreover, to acquire hierarchical features from images, the progressive module (PM) treats the network's shallow and deep layers differently. The shallow layers represent the structure of the source image; the deep layers correspond to details. Different loss functions are utilized for these two kinds of information to retain fused images' salient structures and functional information. Experiments show that the proposed algorithm performs well on visual quality and objective evaluation, which provides a reliable reference for medical diagnosis. In addition, this method is lightweight and speedy compared to existing algorithms, facilitating further placement in the IoMT's specific devices.

A Systematic Literature Review on Multimodal Medical Image Fusion

A Systematic Literature Review on Multimodal Medical Image Fusion

An Improved Multimodal Medical Image Fusion Approach Using Intuitionistic Fuzzy Set and Intuitionistic Fuzzy Cross-Correlation.

Multimodal medical image fusion (MMIF) is the process of merging different modalities of medical images into a single output image (fused image) with a significant quantity of information to improve clinical applicability. It enables a better diagnosis and makes the diagnostic process easier. In medical image fusion (MIF), an intuitionistic fuzzy set (IFS) plays a role in enhancing the quality of the image, which is useful for medical diagnosis. In this article, a new approach to intuitionistic fuzzy set-based MMIF has been proposed. Initially, the input medical images are fuzzified and then create intuitionistic fuzzy images (IFIs). Intuitionistic fuzzy entropy plays a major role in calculating the optimal value for three degrees, namely, membership, non-membership, and hesitation. After that, the IFIs are decomposed into small blocks and then perform the fusion rule. Finally, the enhanced fused image can be obtained by the defuzzification process. The proposed method is tested on various medical image datasets in terms of subjective and objective analysis. The proposed algorithm provides a better-quality fused image and is superior to other existing methods such as PCA, DWTPCA, contourlet transform (CONT), DWT with fuzzy logic, Sugeno's intuitionistic fuzzy set, Chaira's intuitionistic fuzzy set, and PC-NSCT. The assessment of the fused image is evaluated with various performance metrics such as average pixel intensity (API), standard deviation (SD), average gradient (AG), spatial frequency (SF), modified spatial frequency (MSF), cross-correlation (CC), mutual information (MI), and fusion symmetry (FS).

The Inner Complexities of Multimodal Medical Data: Bitmap-Based 3D Printing for Surgical Planning Using Dynamic Physiology.

Motivated by the need to develop more informative and data-rich patient-specific presurgical planning models, we present a high-resolution method that enables the tangible replication of multimodal medical data. By leveraging voxel-level control of multimaterial three-dimensional (3D) printing, our method allows for the digital integration of disparate medical data types, such as functional magnetic resonance imaging, tractography, and four-dimensional flow, overlaid upon traditional magnetic resonance imaging and computed tomography data. While permitting the explicit translation of multimodal medical data into physical objects, this approach also bypasses the need to process data into mesh-based boundary representations, alleviating the potential loss and remodeling of information. After evaluating the optical characteristics of test specimens generated with our correlative data-driven method, we culminate with multimodal real-world 3D-printed examples, thus highlighting current and potential applications for improved surgical planning, communication, and clinical decision-making through this approach.

Gray wolf optimization and image enhancement with NLM Algorithm for multimodal medical fusion imaging system

Multimodal Medical fusion imaging is a significant aspect of image-guided medical diagnosis. A new improved non-sub sampled shearlet transform (NSST) based Multimodal Medical Fusion imaging System based on Gray Wolf Optimization (GWO) and Image Enhancement with NLM Algorithmic computer guided medical procedures. This paper proposes an efficient medical fusion imaging system that is evolved from the concept of Non-Sub sampled Shearlet Transform and the Gray Wolf Optimization algorithm. The initial procedure in this system includes histogram matching among one image and the other to facilitate common dynamic range for both images. Decomposition of multimodal medical images is done by NSST to be fused into their coefficients. Optimum decomposition level is scaled by EGOA with optimal gain parameters. Furthermore, denoising with enhancement procedure is performed to enhance its visual quality. If the images are corrupted by noise, the these traditional fusion methods performance is greatly hampered. Therefore, this is essential to create a fusion approach that can maintain accurate information even while distorting the images.

Multimodal Medical Image Fusion: The Perspective of Deep Learning

Multimodal medical image fusion involves the integration of medical images originating from distinct modalities and captured by various sensors, with the aim to enhance image quality, minimize redundant information, and preserve specific features, ultimately leading to increased efficiency and accuracy in clinical diagnoses. In recent years, the emergence of deep learning techniques has propelled significant advancements in image fusion, addressing the limitations of conventional methods that necessitate manual design of activity level measurement and fusion rules. This paper initially presents a systematic description of the multimodal medical image fusion problem, delineating the interrelationships between different fusion modalities while summarizing their characteristics and functions. Subsequently, it reviews the theories and enhancement approaches associated with deep learning in the medical image fusion domain, striving for a comprehensive overview of the state-of-the-art developments in this field from a deep learning perspective. These developments encompass multimodal feature extraction methods based on convolutional techniques, adversarial learning-based methods, convolutional sparse representation and stacked autoencoder-based signal processing methods, and unified models. Lastly, the paper summarizes the enhancement techniques for multimodal medical image fusion methods, highlighting the pressing issues and challenges encountered by deep learning approaches in this domain.

Image-to-Image Translation for Data Augmentation on Multimodal Medical Images

Image-to-Image Translation for Data Augmentation on Multimodal Medical Images

Exploring Irish Faculty Members' Experiences of Delivering a Multi-modal Medical Education Course in a Resource-limited Environment

Introduction:In November 2022, a group of eleven Irish doctors traveled to St. Joseph’s Hospital, Nyabondo, Kenya with Global Emergency Care Skills (GECS), an Irish NGO. The group delivered a training course to healthcare staff in the hospital, in advance of the scheduled opening of a regional Major Trauma Center on site. This course incorporated didactic lectures, skills stations and simulated clinical scenarios and covered commonly encountered emergency presentations in low and middle income countries (LMICs).Method:A qualitative study was conducted using a free text questionnaire with faculty, exploring their experiences of education in a resource-limited environment. Responses were interpreted by performing thematic analysis to identify recurring themes.Results:All eleven faculty members completed the survey in full. An interrogation of the responses identified commonalities across the majority of faculty members. The main themes encountered were increased recognition of the lack of postgraduate training in LMICs, the challenge of devising material appropriate to a resource-limited setting, a growth in confidence and individual teaching ability, and a reaffirmation of the effectiveness of simulation teaching in medical education.Conclusion:This survey demonstrates the significant impact of teaching such a course on faculty members. Despite the challenges encountered, faculty members strongly felt that simulation training offered significant benefits. Survey respondents noted that moulage could be adapted to suit the needs of course participants without compromising on educational goals. In the absence of formal postgraduate education in LMICs, external agencies continue to play an important role in the delivery of structured training programs.

Multimodality Medical Image Fusion Using Clustered Dictionary Learning in Non-Subsampled Shearlet Transform.

Imaging data fusion is becoming a bottleneck in clinical applications and translational research in medical imaging. This study aims to incorporate a novel multimodality medical image fusion technique into the shearlet domain. The proposed method uses the non-subsampled shearlet transform (NSST) to extract both low- and high-frequency image components. A novel approach is proposed for fusing low-frequency components using a modified sum-modified Laplacian (MSML)-based clustered dictionary learning technique. In the NSST domain, directed contrast can be used to fuse high-frequency coefficients. Using the inverse NSST method, a multimodal medical image is obtained. Compared to state-of-the-art fusion techniques, the proposed method provides superior edge preservation. According to performance metrics, the proposed method is shown to be approximately 10% better than existing methods in terms of standard deviation, mutual information, etc. Additionally, the proposed method produces excellent visual results regarding edge preservation, texture preservation, and more information.

A Novel Bacterial Foraging Optimization Based Multimodal Medical Image Fusion Approach

Multimodal medical image fusion (MIF) is the procedure of integrating different images in single into multiple imaging modalities for increasing the image quality by preserving a certain feature. Medical image combination covered a tremendous count of hot topic areas, involving pattern recognition, image processing, artificial intelligence (AI), computer vision (CV), and machine learning (ML). In addition, MIF was more commonly applied in clinical for physicians to understand the lesion by the combination of various modalities of medicinal image. This article introduces a novel bacterial foraging optimization-based multimodal medical image fusion approach (BFO-M3IFA). The presented BFO-M3IFA technique considered two distinct patterns of the images as the input of systems and the outcome will be the fused image. Primarily, the BFO-M3IFA technique exploits Weiner filtering (WF) technique as an image pre-processing step to get rid of the noise. Besides, discrete wavelet transform (DWT) was applied for decomposing the image into distinct subbands. Afterward, the estimated coefficients of modality 1 and comprehensive coefficients of modality 2 are integrated and vice versa. At last, a fusion rule is generated to fuse the details of two image modalities and the optimal fusion rule parameter is chosen with utilize of BFO algorithm. The experimental validation of the BFO-M3IFA system was tested and outcomes ensured the improved performance of the BFO-M3IFA system on existing models.

Common Mental Disorders in Smart City Settings and Use of Multimodal Medical Sensor Fusion to Detect Them

Cities have undergone numerous permanent transformations at times of severe disruption. The Lisbon earthquake of 1755, for example, sparked the development of seismic construction rules. In 1848, when cholera spread through London, the first health law in the United Kingdom was passed. The Chicago fire of 1871 led to stricter building rules, which led to taller skyscrapers that were less likely to catch fire. Along similar lines, the COVID-19 epidemic may have a lasting effect, having pushed the global shift towards greener, more digital, and more inclusive cities. The pandemic highlighted the significance of smart/remote healthcare. Specifically, the elderly delayed seeking medical help for fear of contracting the infection. As a result, remote medical services were seen as a key way to keep healthcare services running smoothly. When it comes to both human and environmental health, cities play a critical role. By concentrating people and resources in a single location, the urban environment generates both health risks and opportunities to improve health. In this manuscript, we have identified the most common mental disorders and their prevalence rates in cities. We have also identified the factors that contribute to the development of mental health issues in urban spaces. Through careful analysis, we have found that multimodal feature fusion is the best method for measuring and analysing multiple signal types in real time. However, when utilizing multimodal signals, the most important issue is how we might combine them; this is an area of burgeoning research interest. To this end, we have highlighted ways to combine multimodal features for detecting and predicting mental issues such as anxiety, mood state recognition, suicidal tendencies, and substance abuse.

Shearlet Transform-Based Novel Method for Multimodality Medical Image Fusion Using Deep Learning

Multi-modality medical image fusion (MMIF) methods were widely used in a variety of clinical settings. For specialists, MMIF could provide an image containing anatomical and physiological information that can help develop diagnostic procedures. Different models linked to MMIF were proposed previously. However, there would be a need to enhance the functionality of prior methodologies. In this proposed model, a unique fusion model depending upon optimal thresholding and deep learning approaches are presented. An enhanced monarch butterfly optimization (EMBO) determines an optimal threshold with fusion rules as in shearlet transform. The efficiency of the fusion process mainly depends on the fusion rule and the optimization of the fusion rule can improve the efficiency of the fusion. The extraction element of the deep learning approach was then utilized to fuse high- and low-frequency sub-bands. The fusion technique was carried out using a convolutional neural network (CNN). The studies were carried out for MRI and CT images. The fusion results were attained and the proposed model was proved to offer effective performance with reduced values of error and improved values of correlation.

Multimodal Medical Image Fusion Using Two- Stage Decomposition Technique to Combine the Significant Features of Spatial Fuzzy Plane and Transformed Frequency Plane

Multimodal medical image fusion is becoming a powerful technique in various clinical applications such as disease diagnosis and treatment planning. In this work, a two-stage decomposition technique is proposed to capture the significant features from the spatial and spectral domains as well. The first stage decomposition method employs a multicluster fuzzy c-means (FCM) algorithm to produce gross- and fine-structural components. The fine-structural components containing the edge topology are again decomposed into amplitude and phase spectra in terms of the discrete cosine transformation-based discrete Fourier transform (DCT-DFT). Finally, all parts of decomposed images are integrated with three different fusion rules. A new window-based feature quality measuring (WFQM) filter is proposed for fusing the information of gross-structural components, while the singular value decomposition (SVD) method and gray-wolf optimization-based pulse-coupled neural network (GWO-PCNN) model are applied for fusing the amplitude and phase spectrum distributions of input images. As the WFQM filter and GWO-PCNN model are designed to fuse the significant information from the spatial fuzzy plane and transformed frequency plane respectively, relative contrast degradation and spectral distortion are less in the proposed fusion method. Experimental results are reported on Harvard University datasets and some qualitative as well as quantitative measures are presented for comparing the proposed method with other state-of-art-techniques. The quantitative evaluation used peak signal-to-noise ratio (PSNR), mutual information (MI), structural similarity index measurement (SSIM), visual information fidelity (VIF), and entropy (ENT) to study its superiority.