Abstract
Deep learning has been widely used in various computer vision tasks. As a result, researchers have begun to explore the application of deep learning for pansharpening and have achieved remarkable results. However, most current pansharpening methods focus only on the mapping relationship between images and the lack overall structure enhancement, and do not fully and completely research optimization goals and fusion rules. Therefore, for these problems, we propose a pansharpening generative adversarial network with multilevel structure enhancement and a multistream fusion architecture. This method first uses multilevel gradient operators to obtain the structural information of the high-resolution panchromatic image. Then, it combines the spectral features with multilevel gradient information and inputs them into two subnetworks of the generator for fusion training. We design a comprehensive optimization goal for the generator, which not only minimizes the gap between the fused image and the real image but also considers the adversarial loss between the generator and the discriminator and the multilevel structure loss between the fused image and the panchromatic image. It is worth mentioning that we comprehensively consider the spectral information and the multilevel structure as the input of the discriminator, which makes it easier for the discriminator to distinguish real and fake images. Experiments show that our proposed method is superior to state-of-the-art methods in both the subjective visual and objective assessments of fused images, especially in road and building areas.
Highlights
Chongqing Key Laboratory of Image Cognition, Chognqing University of Posts and Telecommunications, College of Software Engineering, Chognqing University of Posts and Telecommunications, These authors contributed to this work
We found that the structural information fo the PAN image extracted by the second-level gradient operator is still rich, so we try to use the new structure to enhance the pansharpening performance
This paper proposes a panchromatic sharpening generation confrontation network with multi-level structure enhancement and multi-stream fusion architecture
Summary
Due to the limitation of technology, a single sensor cannot simultaneously obtain remote sensing images with high resolution in both the spectral and spatial domains. To solve the problem that the MS image spectral information and PAN spatial information cannot be fully utilized, more related methods have been proposed (e.g., a hybrid algorithm that combines the IHS transform and curvelet transform algorithms [11], a variational model solved using a convex optimization difference solution framework [12], and a method based on compressed sensing with sparse prior information [13]). Researchers have begun to explore the application of deep learning in pansharpening and achieved remarkable results These methods are implemented based on a convolutional neural network (CNN), which is used to extract the spectral features from low-resolution. (2) In order to better combine the spatial information extracted by the multi-level gradient operator, we use the multi-stream fusion CNN architecture as the GAN generator.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
