Hyperspectral Remote Sensing Image Synthesis Based on Implicit Neural Spectral Mixing Models

Abstract

Hyperspectral image (HSI) synthesis, as an emerging research topic, is of great value in overcoming sensor limitations and achieving low-cost acquisition of high-resolution remote sensing HSIs. However, the linear spectral mixing model used in recent studies oversimplifies the real-world hyperspectral imaging process, making it difficult to effectively model the imaging noise and multiple reflections of the object spectrum. As a prerequisite for hyperspectral data synthesis, accurate modeling of nonlinear spectral mixtures has long been a challenge. Considering the above difficulties, we propose a novel method for modeling nonlinear spectral mixtures based on implicit neural representations (INRs) in this article. The proposed method learns from INR and adaptively implements different mixture models for each pixel according to their spectral signature and surrounding environment. Based on the above neural mixing model, we also propose a new method for HSI synthesis. Given an RGB image as input, our method can generate an accurate and physically meaningful HSI. As a set of by-products, our method can also generate subpixel-level spectral abundance as well as the solar atmosphere signature. The whole framework is trained end-to-end in a self-supervised manner. We constructed a new dataset for HSI synthesis based on a wide range of Airborne Visible Infrared Imaging Spectrometer (AVIRIS) data. Our method achieves a mean peak signal-to-noise ratio (MPSNR) of 52.36 dB and outperforms other state-of-the-art hyperspectral synthesis methods. Finally, our method shows great benefits to downstream data-driven applications. With the HSIs and abundance directly generated from low-cost RGB images, the proposed method improves the accuracy of HSI classification tasks by a large margin, particularly for those with limited training samples.