Reversible-Prior-Based Spectral-Spatial Transformer for Efficient Hyperspectral Image Reconstruction

Abstract

The task of reconstructing a 3D cube from a 2D measurement is not well-defined in spectral imaging. Unfortunately, existing Deep Unfolding Network (DU) and End-to-End (E2E) approaches can't strike an optimal balance between computational complexity and reconstruction quality. The goal of this study is to think about ways to merge the E2E's violent mapping with DU's iterative method. Our proposed deep learning framework, the Reversible-prior-based Spectral-Spatial Transformer, combines the high-quality reconstruction capabilities of DU with the advantages of having fewer parameters and lower computing cost, similar to the E2E approach. SST-ReversibleNet uses a reversible prior to project the end-to-end mapping reconstruction results back into the measurement space, construct the residuals between the reprojection and the actual measurement, and improve reconstruction accuracy. Extensive trials show that our SST-ReversibleNet outperforms cutting-edge approaches by at least 0.8 dB and only use 34.3% Params and 44.1% giga floating-point operations per second (GFLOP).