Preference Learning to Multifocus Image Fusion via Generative Adversarial Network

Abstract

Multifocus image fusion (MFIF) is to produce an all-in-focus fused image by integrating a pair of multifocus images with the same scene. In this article, an effective focus detection method is proposed for MFIF, by a generative adversarial network (GAN) with preference learning (PL). Benefitting from more obvious focus characteristics from the luminance channel in HSV, we take this luminance as the input of our GAN to carry out the easier focus detection, instead of the grayscale images in existing methods. On the other hand, to train our GAN more effectively, we utilize the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${\ell }_{{2},{1}}$ </tex-math></inline-formula> norm to construct a focus fidelity loss with structural group sparseness, to regularize the generator loss, pledging a more accurate focus confidence map. More importantly, a novel learning strategy, termed PL, is further developed to enhance model training. Functionally, it assigns a larger learning weight to a sample more difficult to be learned. Extensive experiments demonstrate that our proposed method is superior to other state-of-the-art methods.