Chapter 3 - Image Fusion Through Deep Convolutional Neural Network

Abstract

Recent rapid deployment of imaging technologies and improvement of computational power allow us to process different data that are collected from various sensing modalities, employed in many applications of medical imaging, computer vision and remote sensing. A major gain of combining the outcomes of different modalities is to utilize the complementary information from each modality to form a better image. The method that predominantly combines images from multi-modalities in order to exalt the view of an image with upgraded complementary information is termed image fusion. With it, the multi-sensor data with complementary information about the particular region are comparatively analyzed. The new image formed by image fusion is suitable for image processing methods such as pattern or object recognition, segmentation, etc., and also for the purposes of human perception. The most essential issue in image fusion to be addressed is to define standard fusion rules for merging the multi-modal images. Current technologies aim at machine learning (ML) and deep learning (DL) for automatic image processing. The method of convolutional neural network (CNN) cannot be used directly to fuse multi-modal medical images. Various solutions have been demonstrated in the literature to make the best use of CNN for medical image fusion. This chapter presents a survey of image fusion algorithms based on deep convolutional neural network, and the results obtained by these methods are interpreted and discussed.