A Variational Approach for Fusion of Panchromatic and Multispectral Images Using a New Spatial–Spectral Consistency Term

Abstract

In this article, we propose a variational approach for fusion of two coregistered high-resolution panchromatic (HRP) and low-resolution multispectral (LRM) images to reach the high-resolution multispectral (HRM) one, i.e., pan-sharpening. In this fusion technique, there is a tradeoff between structural information of an HRP image and spectral information of LRM one. To reconstruct the HRM image, which benefits from the best characteristics of both images, we consider several fidelity terms. The structural fidelity term is used to transfer structural information of an HRP image to HRM one, and a spectral fidelity term is utilized to preserve spectral consistency between HRM and LRM images throughout the fusion process. To reduce the spectral distortion occurred due to the discrepancy between intensity values of HRP and LRM images, a novel spatial–spectral fidelity term is designed to keep the intensity ratio between multispectral and panchromatic pixels in the high-resolution space as the same as the low-resolution space. Moreover, the total variation (TV) regularization term is employed as a prior to promote the sparseness of gradient in HRM bands. These fidelity terms were formulated in a convex optimization problem. However, the structural and TV terms made this optimization problem nondifferentiable. Therefore, we developed an efficient majorization–minimization algorithm for solving the optimization problem. The proposed method applied to three datasets, acquired by WorldView-3, Deimos-2, and QuickBird satellites. To assess the effectiveness of the proposed method, visual analysis, as well as quantitative comparison to various pan-sharpening methods, was carried out. The experimental results suggested that the proposed method outperformed the competitors visually and quantitatively.