FOV Expansion of Bioinspired Multiband Polarimetric Imagers With Convolutional Neural Networks

Abstract

Spectral and polarimetric contents of the light reflected from an object contain useful information on material type and surface characteristics of the object. Jointly exploiting spatial, spectral, and polarimetric information helps detect camouflage targets. Motivated by the vision mechanism of some known aquatic insects, we construct a bioinspired multiband polarimetric imaging system using a camera array, which simultaneously captures multiple images of different spectral bands and polarimetric angles. But the disparity between the fixed positions of each component camera leads to the loss of information in the boundary region and a reduction in the field of view (FOV). In order to overcome the limits, this paper presents a deep learning method for FOV expansion, incorporating the gradient prior of the image into a nine-dimensional convolutional neural network's framework to learn end-to-end mapping between the incomplete images and the FOV-expanded images. With FOV expansion, the proposed model recovers significant missing information. For the problem of insufficient training data, we construct the training dataset and propose the corresponding training methods to achieve good convergence of the network. We also provide some experimental results to validate its state-of-the-art performance of FOV expansion.