Abstract
Multispectral image reconstruction, which aims to recover a three-dimensional (3D) spatial–spectral signal from a two-dimensional measurement in a spectral camera based on ghost imaging via sparsity constraint (GISC), has been attracting much attention recently. However, faced with abundant 3D spectral data, the reconstruction quality cannot meet the visual requirements. Based on the robust data processing capability of deep learning, a novel network called SSTU-Net3+ is constructed by improving U-Net3+ with a spatial–spectral transformer (SST). To enhance the feature representation of images during reconstruction, mixed pooling modules and new convolution processes are proposed to improve the performance of the encoder and decoder, with U-Net3+ as the backbone. To boost the quality of reconstructed images, with split and concatenate (Concat) operations, we construct SST modules by exploiting both spatial and spectral correlations of multispectral images to refine the spatial and spectral features. Furthermore, we employ the SST in the decoder to reconstruct the desired 3D cube. Given similar network parameters, experiments on GISC spectral imaging data show that, compared to convolutional neural network–based methods, the average peak signal-to-noise ratio of images reconstructed using SSTU-Net3+ is improved by 3%, the structural similarity is enhanced by 3%, and the spectral angle mapping is cut by 12%. Particularly, compared to differential ghost imaging and compressed sensing, the reconstruction quality of SSTU-Net3+ has been significantly improved. SSTU-Net3+ can process a large amount of 3D multispectral image data more efficiently and construct the target image more accurately than the abovementioned methods.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.