Design of multispectral array imaging system based on depth-guided network

Abstract

Imaging techniques using multispectral filter arrays (MSFA)have become a research hotspot with the rapid development of spectroscopic techniques. Among them, exploiting the correlation of color channels in the raw data and reconstructing raw images with high sparsity is a bottleneck and constraint in multi-band MSFA imaging systems. Therefore, this paper proposes a 4 × 4 eight-band MSFA imaging system containing a high sampling rate all-pass band. The all-pass band with a 1/2 high sampling rate contains rich color texture information to provide more features. A depth-guided reconstruction network (DGRN), including a depth-guided model (DGM) and a channel adaptive convolution model (CACM), is established to reconstruct the original spectral images. DGM extracts the color texture information of all-pass band images as the guide feature, which is combined with the initially processed eight-band shallow features to be the input of CACM to assign different guide features to different bands adaptively for learning and aggregation. The spatial correlation and spectral correlation of multiple bands are jointly learned using spectral and spatial properties to make the network flexible for MSFA imaging systems. The experimental results show that the method can effectively remove the artifacts of reconstructed images and improve the edge texture clarity.