Abstract
Remote sensing data have become increasingly vital in target detection, disaster monitoring, and military surveillance. Abundant pan-sharpening and super-resolution (SR) methods based on deep learning have been proposed and have achieved remarkable performance. However, pan-sharpening requires paired panchromatic (PAN) and multispectral (MS) images, and SR cannot increase the spectral resolution of PAN. Thus, we introduce a computational imaging-based method to recover or produce the incomplete data of single PAN or MS. This work also explores the integration of multiple tasks by a single neural network. We start with SR and colorization, study the feasibility of simultaneously finishing SR colorization, and use a model trained in SR colorization to finish pan-sharpening without MS. A generic neural network, remote sensing image improvement network (RSI-Net), is designed for remote sensing image SR, colorization, simultaneous SR colorization, and pan-sharpening. To verify its performance, RSI-Net is compared with the state-of-the-art SR and colorization methods. Experiments show that RSI-Net can be competitive in visual effects and evaluation indexes, and it performs well at simultaneous SR colorization, and RSI-Net finishes pan-sharpening and only needs to input PAN. Our experiments confirm the effect of integrating multiple tasks.
Highlights
R EMOTE sensing images play a crucial role in disaster monitoring, target detection, military reconnaissance, and other fields
The outputs of 4× and 8× enlargement are visualized in Fig. 6 and Fig. 7, from which we can find that peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) are higher for our method
We proposed a neural network to enhance the spatial and spectral resolution of remote sensing images
Summary
R EMOTE sensing images play a crucial role in disaster monitoring, target detection, military reconnaissance, and other fields. This work shows that a multi-task neural network can be used to recover or produce high-quality remote sensing images with incomplete data. Most researchers do not consider potential associations between them, and they combine them as an integrated image generation problem Both SR and colorization hope to give supplementary information to the input image, but they are diverse in supplying supererogatory pixels to input. We wish to design a general neural network structure to adapt to different visual tasks for remote sensing images. To solve the problems above, we propose RSINet, a neural network to super-resolve and colorize remote sensing images, which is capable of adapting SR, colorization, and simultaneous SR colorization these three visual missions for remote sensing images with limited data. Inspired by Inception modules, we propose a multiscale residual block (MRB) for feature extraction and reconstruction in our architecture
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: IEEE Transactions on Geoscience and Remote Sensing
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
