Multi-focus image fusion with convolutional neural network based on Dempster-Shafer theory

Abstract

Convolutional neural networks (CNN) have been applied to many fields including image classification. Multi-focus image fusion can be regarded as the classification of focused areas and unfocused areas. Therefore, CNN has been widely used in multi-focus image fusion. However, most methods only use information from the last convolutional layer to complete the fusion task, which leads to a suboptimal fusion result. Aiming to solve this problem, we propose a novel convolutional neural network based on the Dempster-Shafer theory (DST) for multi-focus image fusion. Firstly, as a theoretical method for dealing with uncertain issues, the DST is introduced to fuse the results from different branch layers, thus increasing the reliability of the results. Also, a gradient residual block is designed to boost the utilization of edge information by the network while reducing the dimension from feature maps in the branch layers, thereby improving the performance of the network and reducing the number of training parameters. Compared with other state-of-the-art fusion methods, the decision map of the proposed method is more precise. And objectively, the average metrics of our proposed method for the 20 images from the ``Lytro" and ``Nature" datasets perform best in terms of information entropy, mutual information, structural similarity metric, and visual perception metric.