Multi-Focus Image Fusion Using U-Shaped Networks With a Hybrid Objective

Abstract

Multi-focus image fusion (MFIF) is a fundamental task in image processing. It generates an all-in-focus image through multiple partly focused source images. There are common schemes that are based on focused region detection and activity-level measurement, and fusion rule. We found that it is difficult to directly map between source images and focus maps. In this paper, we investigate the use of U-shape networks for the end-to-end modeling of MFIF. The novelty of our framework is two-fold. First, it uses a U-shaped network as a feature extractor that captures low-frequency information through feature extraction and high-frequency texture information through high-frequency texture extraction. Compared with the common convolutional neural network, the proposed network has better representation ability so that the most visually distinctive features can be extracted, fused, and enhanced. Second, the hybrid objective with $\ell 1$ and perceptual losses enables the framework to yield fused results that are consistent with human beings’ perception. In addition, we employ a weighted strategy to merge the chrominance components in the YCbCr color space so that color distortion is almost eliminated in the color fused result. We investigate the performance through extensive experiments to verify the effectiveness of the proposed method. Through qualitative and quantitative assessment, the proposed method has performance comparable to the recent state-of-the-art methods.