From Regression Based on Dynamic Filter Network to Pansharpening by Pixel-Dependent Spatial-Detail Injection

Xuan Liu,Ping Tang,Xing Jin,Zheng Zhang

doi:10.3390/rs14051242

Abstract

Compared with hardware upgrading, pansharpening is a low-cost way to acquire high-quality images, which usually combines multispectral images (MS) in low spatial resolution with panchromatic images (PAN) in high spatial resolution. This paper proposes a pixel-dependent spatial-detail injection network (PDSDNet). Based on a dynamic filter network, PDSDNet constructs nonlinear mapping of the simulated panchromatic band from low-resolution multispectral bands through filtering convolution regression. PDSDNet reduces the possibility of spectral distortion and enriches spatial details by improving the similarity between the simulated panchromatic band and the real panchromatic band. Moreover, PDSDNet assumes that if an ideal multispectral image that has the same resolution with the panchromatic image exists, each band of it should have the same spatial details as in the panchromatic image. Thus, the details we fill into each multispectral band are the same and they can be extracted effectively in one pass. Experimental results demonstrate that PDSDNet can generate high-quality fusion images with multispectral images and panchromatic images. Compared with BDSD, MTF-GLP-HPM-PP, and PanNet, which are widely applied on IKONOS, QuickBird, and WorldView-3 datasets, pansharpened images of the proposed method have rich spatial details and present superior visual effects without noticeable spectral and spatial distortion.

Highlights

Remote sensing sensors generate images by capturing information of electromagnetic waves reflected off the Earth’s surface
Consider the typical case when training and test data are acquired with the same sensor but come from different scenes, three state-of-the-art algorithms are employed for comparison, which are banddependent spatial-detail (BDSD) [10], modulation transfer function (MTF)-generalized Laplacian pyramid (GLP)-HPM-PP [22], and PanNet [28]
MTF-GLP-HPM-PP is one of the effective methods of multi-resolution analysis (MRA) [42], which is based on a generalized Laplacian pyramid (GLP) [43] with modulation transfer function (MTF)-matched filter [20], multiplicative injection model [44] and post-processing (MTF-GLP-HPM-PP) [22]

Summary

Introduction

Remote sensing sensors generate images by capturing information of electromagnetic waves reflected off the Earth’s surface. It is arduous to obtain images with both high spatial resolution and high spectral resolution simultaneously. The energy received by the sensor is double the integral of the electromagnetic wave in space and wavelength. Generating images with higher spatial and spectral resolution means that the energy is integrated at shorter wavelengths and in smaller areas. The energy is weaker, resulting in poorer image qualities. It is challenging to acquire high-quality images with high spectral and spatial resolution, limited by the equipment on remote sensing platforms. Compared with hardware upgrading, pansharpening is a low-cost way to sufficiently utilize data to obtain high spectral and spatial resolution images. Pansharpening combines multispectral images (MS) with low spatial resolution and panchromatic images (PAN) with high spatial resolution

Methods

Results

Discussion

Conclusion