Color Polarization Image Super-Resolution Reconstruction via a Cross-Branch Supervised Learning Strategy

Abstract

The division-of-focal-plane polarimeter, as a real-time polarization imaging technology, suffers from missing spatial resolution. To address this challenge, we propose a novel two-branch network for color polarization image super-resolution (CPSRNet), which provides a low-cost solution. Specifically, CPSRNet utilizes the degree of linear polarization (DoLP) image and intensity image (S0) as inputs instead of a conventional intensity image. The DoLP branch, as the high-frequency signal stream, supervises cross-branch feature activation in the shallow architecture and restores the polarization characteristic. CPSRNet, which mainly consists of a cross-branch activation module (CBAM) and a related-supervised feature residual fusion module (RSRFM), tends to improve the learning of high-frequency features both locally and globally. The proposed CBAM utilizes deeper features in the DoLP branch to activate the low-level high-frequency features in the intensity branch. Meanwhile, RSRFM introduces adjusted cosine similarity (ACOS) loss to fuse two-branch feature maps in a supervised-learning manner. ACOS loss exploits the intensity stream to encourage the fused feature stream to flow in the desired direction. In addition, we build a new dataset for the color polarization image super-resolution task. All the color polarization images are captured by a division-of-focal-plane polarization camera in real scenarios. We perform extensive experiments, including polarization image super-resolution and demosaicking, to confirm the superiority of our CPSRNet. The source code is available at https://github.com/yudadabing/CPSRNet.