Abstract
Hyperspectral image (HSI) fusion can effectively improve the spatial resolution of HSIs by integrating high-resolution multispectral images (MSIs). Considering the spatial and spectral degradation relationship between a fused image and input images, a physics-based GAN is proposed to fuse HSI and MSI. A physical model estimating degradation of image is introduced in the generator and in the discriminators. For the generator, a set of recursive modules including a physical degradation model and a multiscale residual channel attention fusion module integrate the spectral-spatial difference information between input images and estimated degradation images to restore the details of the fused image. Subsequently, the residual spatial attention fusion module is used to combine the results of all recursions to obtain the final reconstructed result. As for the discriminators, three networks with the final fused image, estimated LR HSI and estimated MSI as inputs share the same architecture. Finally, the loss function that contains adversarial losses and L1 losses of the fused image and estimated degradation images is used to optimize network parameters. The experimental results demonstrate that the proposed method outperforms state-of-the-art methods.
Highlights
W ITH the development of sensor technology, many kinds of remote sensing images can be acquired
The experiments conducted on four data sets demonstrate that the proposed method outperforms state-of-the-art methods
According to the observation equation, there is a relationship between LR hyperspectral image (HSI), multispectral images (MSIs), and HR HSI, which the reconstructed image should satisfy
Summary
W ITH the development of sensor technology, many kinds of remote sensing images can be acquired. The spatial and spectral details of the image can be refined through several recursive modules It can help maintain the feature attributes of the input images, enhance the useful learned features. To further reduce the spectral distortion of the generated image and recover spatial details, three discriminator networks are proposed for the adversarial training. Three discriminator networks take the fusion image, simulated LR HSI and MSI from the generator as input. 1) In this article, the degradation subnetworks are proposed to use convolutions to simulate unknown physics degradation matrices It can effectively repair the image details by comparing the simulated and original degraded images and considering their differences in the loss function.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
More From: IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.