CNNs hard voting for multi-focus image fusion

Abstract

The main idea of image fusion is gathering the necessary features and information into one image. The multi-focus image fusion process is gathering this information from the focused areas of many images and the ideal fused image have all focused part from the input images. There are many studies of multi-focus image fusion in the spatial and transform domains. Recently the multi-focus image fusion methods based on deep learning have been emerged, and they have enhanced the decision map greatly. Nevertheless, the construction of an ideal initial decision map is still difficult and inaccessible. Therefore, the previous methods have high dependency on vast post-processing algorithms. This paper proposes a new convolution neural networks (CNNs) based on ensemble learning for multi-focus image fusion. This network uses hard voting of three branches CNNs that each branch is trained on three different datasets. It is very reasonable and reliable to use various models and datasets instead of just one and it would help to the network for improving the accuracy of classification. In addition, this paper introduces new simple arranging of the patches of the multi-focus datasets that is very useful in obtaining better classification accuracy. With this new arrangement of datasets, three types of multi-focus datasets are created with the help of gradient in the directions of vertical and horizontal. This paper illustrates that the initial segmented decision map of the proposed method is very cleaner than the others, and even it is cleaner than the other final decision maps after refined with a lot of post-processing algorithms. The conducted experimental results and analysis evidently validate that the proposed network have the cleanest initial decision map and the best quality of the output fused image compared to the other state of the art methods. These comparisons are performed with various qualitative and quantitative assessments that are assessed by several fusion metrics for demonstrating the superiority of the proposed network.