Multi-modal medical image super-resolution fusion based on detail enhancement and weighted local energy deviation

Abstract

Multi-modal medical image fusion (MMIF) integrates medical images of different modalities into an image with rich information to boost the accuracy and efficiency of clinical diagnosis and treatment. There are two main problems in medical image fusion: 1) It is difficult to balance the computational efficiency and fusion quality; 2) in the clinic, it is necessary to observe medical images in high resolution. To overcome these problems, a multi-modal medical image super-resolution (SR) fusion (MMISRF) method is proposed based on detail enhancement and weighted local energy deviation (WLED). The method has two major novelties. For the first problem, to improve the efficiency, the proposed method decomposes the SR image into one base layer and one detail layer through a two-scale decomposition method. Then, for the detail layer, a fusion rule is proposed based on detail enhancement and information refinement to enhance the detail information and retain the salient feature information. For the base layer, a WLED-based rule is designed to better preserve the energy information from the source images to the fused image. The final fused image is obtained by combining the fused detail and base layers. For the second problem, this paper introduces the bicubic interpolation-based SR into the field of MMIF for the first time. The experimental results indicate that the proposed MMISRF outperforms the state-of-the-art approaches in terms of subjective visual effect and objective evaluation. Furthermore, the proposed method is more effective for MMIF task at different resolutions.