Optimization Algorithm Unfolding Deep Networks of Detail Injection Model for Pansharpening

Abstract

Pansharpening aims at integrating a high-spatial-resolution panchromatic (PAN) image with a low-spatial-resolution multispectral (MS) image to generate a high-resolution MS (HRMS) image. It is a fundamental and significant task in the field of remotely sensed images. Classic and convolutional neural network (CNN)-based algorithms have been developed, over the last decades, for pansharpening based on the spatial detail injection model. However, these algorithms have difficulties in extracting sufficient details or lack interpretability. In this letter, we present an algorithm unfolding pansharpening (AUP) for this task. In the proposed AUP, a two-step optimization model is first designed based on the spatial detail decomposition model. Then, the iteration processes induced by an optimization model are mapped to several detailed convolution (dc) blocks to solve the detail injection by a trainable neural network. Finally, the desired MS details are obtained in end-to-end manners through a decoder. The superiority of the proposed AUP is demonstrated by extensive experiments on datasets acquired by two different kinds of satellites. Each module of the AUP is interpretable, and its fused results are with fewer spectral and spatial distortions.