A Decade Review of Video Compressive Sensing: A Roadmap to Practical Applications

Abstract

It has been over a decade since the first coded aperture video compressive sensing (CS) system was reported. The underlying principle of this technology is to employ a high-frequency modulator in the optical path to modulate a recorded high-speed scene within one integration time. The superimposed image captured in this manner is modulated and compressed, since multiple modulation patterns are imposed. Following this, reconstruction algorithms are utilized to recover the desired high-speed scene. One leading advantage of video CS is that a single captured measurement can be used to reconstruct a multi-frame video, thereby enabling a low-speed camera to capture high-speed scenes. Inspired by this, a number of variants of video CS systems have been built, mainly using different modulation devices. Meanwhile, in order to obtain high-quality reconstruction videos, many algorithms have been developed, from optimization-based iterative algorithms to deep-learning-based ones. Recently, emerging deep learning methods have been dominant due to their high-speed inference and high-quality reconstruction, highlighting the possibility of deploying video CS in practical applications. Toward this end, this paper reviews the progress that has been achieved in video CS during the past decade. We further analyze the efforts that need to be made—in terms of both hardware and algorithms—to enable real applications. Research gaps are put forward and future directions are summarized to help researchers and engineers working on this topic.