Abstract
Pansharpening is a very debated spatio-spectral fusion problem. It refers to the fusion of a high spatial resolution panchromatic image with a lower spatial but higher spectral resolution multispectral image in order to obtain an image with high resolution in both the domains. In this article, we propose a novel variational optimization-based (VO) approach to address this issue incorporating the outcome of a deep convolutional neural network (DCNN). This solution can take advantages of both the paradigms. On one hand, higher performance can be expected introducing machine learning (ML) methods based on the training by examples philosophy into VO approaches. On other hand, the combination of VO techniques with DCNNs can aid the generalization ability of these latter. In particular, we formulate a $\ell _2$-based proximal deep injection term to evaluate the distance between the DCNN outcome, and the desired high spatial resolution multispectral image. This represents the regularization term for our VO model. Furthermore, a new data fitting term measuring the spatial fidelity is proposed. Finally, the proposed convex VO problem is efficiently solved by exploiting the framework of the alternating direction method of multipliers (ADMM), thus guaranteeing the convergence of the algorithm. Extensive experiments both on simulated, and real datasets demonstrate that the proposed approach can outperform state-of-the-art spatio-spectral fusion methods, even showing a significant generalization ability. Please find the project page at https://liangjiandeng.github.io/Projects_Res/DMPIF_2020jstars.html.
Highlights
M ULTISPECTRAL (MS) remote sensing images have become widely exploited in many fields, such as environmental monitoring, agriculture, and classification
The PAN/MS fusion is committed to integrating the spatial details contained in the PAN image and the spectral information contained in the low spatial resolution MS (LRMS) image to reconstruct a high spatial resolution MS (HRMS) image
Since the PAN image and the LRMS image are captured over the same scene, the boundary locations of the desired HRMS image should be theoretically unified with the ones of the PAN image and the same properties should be present across all the bands of the HRMS image
Summary
M ULTISPECTRAL (MS) remote sensing images have become widely exploited in many fields, such as environmental monitoring, agriculture, and classification. The fused image can be obtained by inputting the LRMS image and PAN image into the learned network They are often over-dependent on the training data [4], so that the generalization of many DCNN methods is limited by their training data, i.e., they have excellent performance only on data similar to the ones in the training set. We propose a novel VO approach for fusing MS and PAN images using the proximal deep injection (PDI), i.e., formulating the output of a DCNN as the proximal term and integrating it into the proposed variational model for further optimization. We exploit the prior knowledge provided by a DCNN into the proposed variational model through the PDI that represents a regularization term in our framework This new variational optimization problem is solved by designing an ADMM-based algorithm, which is guaranteed to efficiently converge to the global minimum.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
