Abstract
Pansharpening aims at fusing a low-resolution multiband optical (MBO) image, such as a multispectral or a hyperspectral image, with the associated high-resolution panchromatic (PAN) image to yield a high spatial resolution MBO image. Though having achieved superior performances to traditional methods, existing convolutional neural network (CNN)-based pansharpening approaches are still faced with two challenges: alleviating the phenomenon of spectral distortion and improving the interpretation abilities of pansharpening CNNs. In this work, we develop a novel spectral-aware pansharpening neural network (SA-PNN). On the one hand, SA-PNN employs a network structure composed of a detail branch and an approximation branch, which is consistent with the detail injection framework; on the other hand, SA-PNN strengthens processing along the spectral dimension by using a spectral-aware strategy, which involves spatial feature transforms (SFTs) coupling the approximation branch with the detail branch as well as 3D convolution operations in the approximation branch. Our method is evaluated with experiments on real-world multispectral and hyperspectral datasets, verifying its excellent pansharpening performance.
Highlights
Multiband optical (MBO) images, including multispectral (MS) and hyperspectral (HS) images, can provide higher spectral resolution than red, green and blue (RGB) images and panchromatic (PAN) images, which expands the difference of the target objects and increases their identifiability.Such characteristics can be used to improve the effectiveness of various image tasks such as change detection [1], classification [2], object recognition [3], and scene interpretation [4]
We carried out experiments on four dataset (three multispectral (MS) datasets acquired with the WorldView-2, IKONOS [45], and Quickbird sensors and one hyperspectral (HS) dataset acquired with the Reflective Optics System Imaging sensors (ROSIS))
In the convolutional neural network (CNN)-based methods, pansharpening CNN (PNN) gets high spatial correlation coefficient (SCC) while the other three metrics are very bad, which implies that PNN is not effective in improving spectral fidelity when used on data with a large number of bands
Summary
Multiband optical (MBO) images, including multispectral (MS) and hyperspectral (HS) images, can provide higher spectral resolution than red, green and blue (RGB) images and panchromatic (PAN) images, which expands the difference of the target objects and increases their identifiability. Different types of prior knowledge have been used to assign prior distributions for Bayesian designing [29,30,31] This type of method attempts to include the fusion process within an interpretable framework and achieves a trade-off between spatial and spectral fidelity. Compared with the aforementioned methods, the model-based optimization methods can achieve higher spatial and spectral precision in the fusion result It is relatively complicated and time-consuming due to the iterative optimization process. Most existing CNN-based methods treat pansharpening merely as a black-box process and lack clear interpretability To deal with these problems, we propose a spectral-aware pansharpening neural network (SA-PNN) in this study.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
