Design of CGAN Models for Multispectral Reconstruction in Remote Sensing

Francisco Argüello,Brais Rodríguez-Suárez,Pablo Quesada-Barriuso

doi:10.3390/rs14040816

Francisco Argüello, Brais Rodríguez-Suárez + Show 1 more

Open Access

https://doi.org/10.3390/rs14040816

Copy DOI

Journal: Remote Sensing	Publication Date: Feb 9, 2022
Citations: 3	License type: CC BY 4.0

Affiliation: University of Santiago de Compostela

Abstract

Multispectral imaging methods typically require cameras with dedicated sensors that make them expensive. In some cases, these sensors are not available or existing images are RGB, so the advantages of multispectral processing cannot be exploited. To solve this drawback, several techniques have been proposed to reconstruct the spectral reflectance of a scene from a single RGB image captured by a camera. Deep learning methods can already solve this problem with good spectral accuracy. Recently, a new type of deep learning network, the Conditional Generative Adversarial Network (CGAN), has been proposed. It is a deep learning architecture that simultaneously trains two networks (generator and discriminator) with the additional feature that both networks are conditioned on some sort of auxiliary information. This paper focuses the use of CGANs to achieve the reconstruction of multispectral images from RGB images. Different regression network models (convolutional neuronal networks, U-Net, and ResNet) have been adapted and integrated as generators in the CGAN, and compared in performance for multispectral reconstruction. Experiments with the BigEarthNet database show that CGAN with ResNet as a generator provides better results than other deep learning networks with a root mean square error of 316 measured over a range from 0 to 16,384.

Highlights

Multispectral images have numerous applications in remote sensing ranging from agriculture [1,2] to environmental monitoring [3,4], change detection [5,6], and geology [7].The main difference of multispectral images compared to RGB images is the incorporation of narrow bands in a specific wavelength range
The accuracy of the multispectral image reconstruction and the computational cost of the Conditional Generative Adversarial Network (CGAN) are evaluated using as generators the Convolutional Neuronal Networks (CNNs), U-Net, and residual connections (ResNet) neural networks designed in the previous section
In view of the results obtained in this work, the problem of generating multispectral images from RGB images can be achieved with sufficient quality using CGAN models

Summary

Introduction

Multispectral images have numerous applications in remote sensing ranging from agriculture [1,2] to environmental monitoring [3,4], change detection [5,6], and geology [7].The main difference of multispectral images compared to RGB images is the incorporation of narrow bands in a specific wavelength range. In the case of having a mixture of RGB and multispectral images, multispectral reconstruction allows uniform processing. This can be important for change detection applications when part of the available images are RGB, and part are multispectral. Multispectral reconstruction can be useful in applications that require multispectral image processing, but only RGB images are available. In this case, the spectral reconstruction could be considered as a preprocessing stage like that performed with filters and morphological or attribute profiles to highlight structures in the Remote Sens.

Methods

Results

Discussion

Conclusion