An efficient approach to medical image fusion based on optimization and transfer learning with VGG19

Abstract

Medical image fusion is the process of combining information from multiple medical images of the same body region acquired using different imaging modalities, such as computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET) into a single image. It is widely used in clinical applications such as oncology, neurology, and cardiology. Deep learning-based approaches have been used to solve this problem. However, some medical image fusion approaches based on pre-trained models still have limitations because these models may not be trained on a diverse set of medical images, which may lead to inefficiencies in extracting features from medical images. Therefore, it is not effective to use feature-based fusion rules extracted from a pre-trained model. In this study, we use the transfer learning technique to build a modified VGG19 model (called TL_VGG19) and use this model to extract features as well as build an efficient fusion rule for detail components. Furthermore, to ensure that the composite image is not degraded in terms of quality. We propose an adaptive fusion method for the base components based on the Equilibrium optimization algorithm (EOA). The seven latest synthesis methods were used for comparison. The experimental results clearly indicate that the methodology employed in this study outperforms existing the latest methods.