ADDNS: An asymmetric dual deep network with sharing mechanism for medical image fusion of CT and MR-T2

Abstract

Medical images with different modalities have different semantic characteristics. Medical image fusion aiming to promotion of the visual quality and practical value has become important in medical diagnostics. However, the previous methods do not fully represent semantic and visual features, and the model generalization ability needs to be improved. Furthermore, the brightness-stacking phenomenon is easy to occur during the fusion process. In this paper, we propose an asymmetric dual deep network with sharing mechanism (ADDNS) for medical image fusion. In our asymmetric model-level dual framework, primal Unet part learns to fuse medical images of different modality into a fusion image, while dual Unet part learns to invert the fusion task for multi-modal image reconstruction. This asymmetry of network settings not only enables the ADDNS to fully extract semantic and visual features, but also reduces the model complexity and accelerates the convergence. Furthermore, the sharing mechanism designed according to task relevance also reduces the model complexity and improves the generalization ability of our model. In the end, we use the intermediate supervision method to minimize the difference between fusion image and source images so as to prevent the brightness-stacking problem. Experimental results show that our algorithm achieves better results on both quantitative and qualitative experiments than several state-of-the-art methods.