Pan-sharpening via intrinsic decomposition knowledge distillation

Abstract

Existing deep-learning-based pan-sharpening strategies mainly involve the fusion of panchromatic and multispectral (MS) information at both the pixel and feature levels. In this paper, we hypothesize that the MS image can be expressed as the multiplication of reflectance and illumination components, and that the reflection components of low-resolution (LR) MS and high-resolution (HR) MS images are invariant. Here, the spectral reflection component can effectively describe the spectral response of an object, while the illumination component can effectively describe its texture. Based on this hypothesis, we propose a novel and concise pan-sharpening framework called intrinsic decomposition knowledge distillation. Specifically, the teacher network decomposes the HR MS image into reflectance and illumination components, which are then combined in the student network with the reflectance component and the enhanced illumination component from LR MS to reconstruct the pan-sharpened image. To approximate the component distributions from the teacher network, we introduce a novel three-stage knowledge distillation strategy that can transfer knowledge about the relationships between components and constrain the student network. Our quantitative and qualitative comparisons demonstrate the reasonableness of our hypothesis and the effectiveness of our proposed method in significantly improving perception quality.