Multi scale decomposition based medical image fusion using convolutional neural network and sparse representation

Abstract

Medical image fusion foremost focuses on discovering superior technique on merging multimodal medical images which plays an important part on clinical analysis and treatment planning. To get a merged image with improved graphical excellence associated with obvious structure information, a novel medical image fusion system based on multi scale decomposition with convolutional neural network and sparse representation is proposed. Initially, L0 smoothing filter is applied to decompose the source images into two frequency layers as Low Frequency Layers (LFLs) and High Frequency Layers (HFLs). Next, HFLs are combined by CNN fusion rule and the LFLs of different modal images are combined by using the Non-Subsampled Contourlet Transformation - sparse representation (NSCT–SR) fusion rule. Followed by combining the reconstructed LFL and HFL, the resultant fused image is attained. In the proposed work, fusion of medical image is performed between CT & MRI, CT & SPECT, MRI & SPECT, MRI/T2 & MRI/Gad, MRI & PET, MRI/T2 & MRI/PD of brain images. The experimental results depict that the proposed work yields better results than the current methodologies in terms of both visual consistency and quantitative analysis.