Fusion of registered medical images using deep learning convolutional neural network with statistics based steered image filter

Abstract

Medical image fusion technique plays an increasingly critical role in many clinical applications by deriving the complementary information from medical images with different modalities. In this, a novel MR and CT image fusion approach is proposed which utilises the deep learning convolutional neural networks (CNNs) with statistics based steered image filter (SSIF). In our method, a deep learning convolutional network is adopted to generate a weight map which integrates the pixel activity information from MR and CT images. The fusion process is conducted via SSIF fusion rule which computes the weights of obtained detail layers using image statistics. In addition, weighted average method is utilised to obtain the fused image. Further, proposed fusion algorithm is extended to applicable for RGB image fusion. Experimental results demonstrate that the proposed method can achieve promising results in terms of both visual quality and objective assessment.