Real-time single-pixel video imaging based on deep learning

Abstract

Listen

Translate article

The emergence of compressed sensing (CS) theory has enabled the development of single-pixel cameras that achieve high-resolution imaging using a single photodetector. However, traditional CS reconstruction algorithms require significant computational time and face an inherent trade-off between imaging resolution and frame rate, limiting current single-pixel cameras to static scene imaging. A key challenge lies in achieving real-time single-pixel imaging with both high frame rate and high resolution. This paper proposes a real-time single-pixel imaging technology based on deep learning. We design a deep convolutional neural network architecture incorporating residual networks to simulate the measurement and reconstruction process of CS-based single-pixel imaging. The network is trained on an image dataset and subsequently deployed for single-pixel imaging. The trained network can complete image reconstruction at a low sampling rate with minimal latency, enabling real-time single-pixel video capture at 128×128 resolution with 33 frames per second (fps) at a 4\% sampling rate. Furthermore, we implement a four-channel parallel signal processing method to achieve real-time single-pixel imaging video at 256×256 resolution at 33 fps.