Deep Self-Learning Network for Adaptive Pansharpening

Jie Hu,Jiemin Wu,Zhi He

doi:10.3390/rs11202395

Abstract

Deep learning (DL)-based paradigms have recently made many advances in image pansharpening. However, most of the existing methods directly downscale the multispectral (MSI) and panchromatic (PAN) images with default blur kernel to construct the training set, which will lead to the deteriorative results when the real image does not obey this degradation. In this paper, a deep self-learning (DSL) network is proposed for adaptive image pansharpening. First, rather than using the fixed blur kernel, a point spread function (PSF) estimation algorithm is proposed to obtain the blur kernel of the MSI. Second, an edge-detection-based pixel-to-pixel image registration method is designed to recover the local misalignments between MSI and PAN. Third, the original data is downscaled by the estimated PSF and the pansharpening network is trained in the down-sampled domain. The high-resolution result can be finally predicted by the trained DSL network using the original MSI and PAN. Extensive experiments on three images collected by different satellites prove the superiority of our DSL technique, compared with some state-of-the-art approaches.

Highlights

The past few years have witnessed the dramatic leap in the remotely sensed imaging
We have proposed a deep self-learning method (i.e., DSL) for image pansharpening
The main advantage of DSL is that it can explore the point spread function (PSF) from the multispectral images (MSI) and PAN and construct the training samples according to the learned degradation

Summary

Introduction

The past few years have witnessed the dramatic leap in the remotely sensed imaging. it is sometimes difficult to obtain the high-resolution satellite images due to the hardware limitations [1]. The CS method contains Principle component analysis (PCA) [7], Intensity-hue-saturation transform (IHS) [8], Gram–Schmidt transform (GS) [9], Brovey transform [10], etc Those algorithms are efficient in terms of the execution time and are able to render the spatial details of PAN with high fidelity. The MRA approach includes Decimated wavelet transform (DWT) [11], “a-trous” wavelet transform (ATWT) [12], Laplacian pyramid [13], Contourlets [14], etc This category is good at preserving the spectral information, while the fusion result suffers from the spatial misalignments. Deep learning (DL) has shown much potential in various image-processing applications such as scene classification [22], super-resolution[23,24], and face recognition [25], and has become a thriving area in the image pansharpening in the last four years

Methods

Results

Discussion

Conclusion