Abstract
Earth-observing satellites carrying multispectral sensors are widely used to monitor the physical and biological states of the atmosphere, land, and oceans. These satellites have different vantage points above the Earth and different spectral imaging bands resulting in inconsistent imagery from one to another. This presents challenges in building downstream applications. What if we could generate synthetic bands for existing satellites from the union of all domains? We tackle the problem of generating synthetic spectral imagery for multispectral sensors as an unsupervised image-to-image translation problem modeled with a variational autoencoder (VAE) and generative adversarial network (GAN) architecture. Our approach introduces a novel shared spectral reconstruction loss to constrain the high-dimensional feature space of multispectral images. Simulated experiments performed by dropping one or more spectral bands show that cross-domain reconstruction outperforms measurements obtained from a second vantage point. Our proposed approach enables the synchronization of multispectral data and provides a basis for more homogeneous remote sensing datasets.
Highlights
C LIMATE change and related environmental issues—including the loss of biodiversity and extreme weather—are listed by the World Economic Forum as the most important risks to our planet [1]
We introduce a shared spectral reconstruction loss and skip connection to effectively generate synthetic spectral bands, and the result is a 50%–80% reduction in mean absolute error (MAE)
We find that band 7, the shortwave infrared band (3.9 μm), is difficult to synthesize with RMAE, RBias, and precision significantly above that of the full reconstruction
Summary
C LIMATE change and related environmental issues—including the loss of biodiversity and extreme weather—are listed by the World Economic Forum as the most important risks to our planet [1]. Monitoring the Earth is critical to mitigating these risks, understanding the effects, and making future predictions [2]. Multispectral and hyperspectral satellite-based remote sensing enables global observation of the Earth, allowing scientists to study large-scale system dynamics and inform general circulation models [3]. In weather forecasts, satellite data initialize the atmospheric state for future predictions. Current generation GEO satellites observe 16 spectral bands over large regions every 10–15 min at a 0.5–2-km resolution. Physical and statistical models are used to convert these images into more interpreted variables, such as precipitation, cloud cover, and surface temperature [18]. Multiple GEO satellites, currently in orbit, extend the spatial ranges to actively monitoring larger regions. Differences in spectral bands and sensor uncertainties/biases present challenges to commonly used sensor-specific models, and especially, existing downstream models do not generalize well to missing spectral information
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: IEEE Transactions on Geoscience and Remote Sensing
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
