A Three Stages Detail Injection Network for Remote Sensing Images Pansharpening

Yuanyuan Wu,Mengxing Huang,Haijie Zhou,Siling Feng,Cong Lin

doi:10.3390/rs14051077

Abstract

Multispectral (MS) pansharpening is crucial to improve the spatial resolution of MS images. MS pansharpening has the potential to provide images with high spatial and spectral resolutions. Pansharpening technique based on deep learning is a topical issue to deal with the distortion of spatio-spectral information. To improve the preservation of spatio-spectral information, we propose a novel three-stage detail injection pansharpening network (TDPNet) for remote sensing images. First, we put forward a dual-branch multiscale feature extraction block, which extracts four scale details of panchromatic (PAN) images and the difference between duplicated PAN and MS images. Next, cascade cross-scale fusion (CCSF) employs fine-scale fusion information as prior knowledge for the coarse-scale fusion to compensate for the lost information during downsampling and retain high-frequency details. CCSF combines the fine-scale and coarse-scale fusion based on residual learning and prior information of four scales. Last, we design a multiscale detail compensation mechanism and a multiscale skip connection block to reconstruct injecting details, which strengthen spatial details and reduce parameters. Abundant experiments implemented on three satellite data sets at degraded and full resolutions confirm that TDPNet trades off the spectral information and spatial details and improves the fidelity of sharper MS images. Both the quantitative and subjective evaluation results indicate that TDPNet outperforms the compared state-of-the-art approaches in generating MS images with high spatial resolution.

Highlights

Remote sensing images (RSIs) are broadly employed in different aspects, for instance, obtaining geographic data, obtaining earth resource information, hazard prediction and analysis, urban investigation, yield estimation and others [1]
The pansharpened results of the partial replacement adaptive component substitution (PRACS), Indusion, modulation transfer function-GLP (MTF_GLP), pansharpening method (PNN), deep residual network-based pansharpening technique (DRPNN), PanNet, FusionNet, residual module-based distributed fusion network (RDFNet) and three-stage detail injection pansharpening network (TDPNet) approaches are shown in Figure 7c–k, respectively
To observe the pansharpening performance more conveniently, we show the average intensity difference map and the average spectral difference map

Summary

Introduction

Remote sensing images (RSIs) are broadly employed in different aspects, for instance, obtaining geographic data, obtaining earth resource information, hazard prediction and analysis, urban investigation, yield estimation and others [1]. In these applications, RSIs with high spatial, spectral or time resolution are usually required [2,3,4]. MS (HRMS) images or high spatial resolution HS (HRHS) images. This technique is called panchromatic sharpening (pansharpening). Traditional approaches comprise the component substitution (CS) approach, multiresolution analysis (MRA) method and variational optimization (VO) technique [2,5]

Methods

Results

Discussion

Conclusion