Shallow–Deep Convolutional Network and Spectral-Discrimination-Based Detail Injection for Multispectral Imagery Pan-Sharpening

Abstract

Pan-sharpening is a significant task in remote sensing image processing, which merges a high-resolution panchromatic (PAN) image and a low-resolution multispectral (MS) image to create a high-resolution MS image. In this article, we propose a novel deep-learning-based MS image pan-sharpening method that combines a shallow–deep convolutional network (SDCN) and a spectral discrimination-based detail injection (SDDI) model. SDCN consists of a shallow network and a deep network, which can capture mid-level and high-level spatial features from PAN images. SDDI, inspired by the “Amelioration de la Resolution Spatial par Injection de Structures” concept, is developed to merge the spatial details extracted by SDCN into MS images with minimal spectral distortion. SDCN and SDDI are collaboratively learned for achieving high-spatial-resolution MS image and preserving more spectral information. Both the visual assessment and the quantitative assessment results on IKONOS and QuickBird datasets confirmed that the proposed method outperforms several state-of-the-art pan-sharpening methods.