MDCwFB: A Multilevel Dense Connection Network with Feedback Connections for Pansharpening

Weisheng Li,Xuesong Liang,Minghao Xiang

doi:10.3390/rs13112218

Weisheng Li, Xuesong Liang + Show 1 more

Open Access

https://doi.org/10.3390/rs13112218

Copy DOI

Abstract

In most practical applications of remote sensing images, high-resolution multispectral images are needed. Pansharpening aims to generate high-resolution multispectral (MS) images from the input of high spatial resolution single-band panchromatic (PAN) images and low spatial resolution multispectral images. Inspired by the remarkable results of other researchers in pansharpening based on deep learning, we propose a multilevel dense connection network with a feedback connection. Our network consists of four parts. The first part consists of two identical subnetworks to extract features from PAN and MS images. The second part is a multilevel feature fusion and recovery network, which is used to fuse images in the feature domain and to encode and decode features at different levels so that the network can fully capture different levels of information. The third part is a continuous feedback operation, which refines shallow features by feedback. The fourth part is an image reconstruction network. High-quality images are recovered by making full use of multistage decoding features through dense connections. Experiments on different satellite datasets show that our proposed method is superior to existing methods, through subjective visual evaluation and objective evaluation indicators. Compared with the results of other models, our results achieve significant gains on the multiple objective index values used to measure the spectral quality and spatial details of the generated image, namely spectral angle mapper (SAM), relative global dimensional synthesis error (ERGAS), and structural similarity (SSIM).

Highlights

SAM [41]: The spectral angle mapper (SAM) measures the spectral distortion of the pansharpened image compared with the reference image
It is defined as the angle between the spectral vectors of the pansharpened image and the reference image in the same pixel, where x1 and x2 refer to two spectrum vectors, as follows: SAM( x1, x2 ) = arccos(
We can see that the feedback connection significantly improves the network performance and gives the network a solid early reconstruction ability

Summary

Introduction

Remote sensing satellite images are a type of image that has been widely concerned and applied at present They provide an important reference for applications in digital maps, disaster emergency, and geological observation [1,2]. Previous work has shown that increasing the depth of the network improves the performance of the network significantly, but because of the gradient explosion and gradient disappearance, deeper networks are difficult to train. He et al [27] proposed a residual learning framework to reduce the difficulty of network optimisation and to reduce degradation problems so that deeper network structures could be used. DenseNet makes comprehensive use of simple features from shallow networks through feature reuse, and achieves a better performance than ResNet, with fewer parameters and lower computational costs

Results

Discussion

Conclusion