Multimodal medical image fusion and classification using deep learning techniques

Abstract

Medical image fusion plays a crucial role in accurate medical diagnostics by combining images from various modalities. To address this need, we propose an AI model for efficient medical image fusion using multiple modalities. Our approach utilizes a Siamese convolutional neural network to construct a weight map based on pixel movement information extracted from multimodality medical images. We leverage medical picture pyramids to incorporate multiscale techniques, enhancing reliability beyond human visual intuition. Additionally, we dynamically adjust the fusion mode based on local comparisons of deconstructed coefficients. Evaluation metrics including F1-score, recall, accuracy, and precision are computed to assess performance, yielding impressive results: an F1-score of 0.8551 and a mutual information (MI) value of 2.8059. Experimental results demonstrate the superiority of our method, achieving a remarkable 99.61% accuracy in targeted experiments. Moreover, the Structural Similarity Index (SSIM) of our approach is 0.8551. Compared to state-of-the-art approaches, our model excels in medical picture classification, providing accurate diagnosis through high-quality fused images. This research advances medical image fusion techniques, offering a robust solution for precise medical diagnostics across various modalities.