Compressive imaging: exploiting multiple frames for enhanced video reconstruction

Abstract

ABSTRACT We consider a coded aperture imaging system which c ollects far fewer measurements than the underlying resolution of the scene we wish to exploit. Our sensing model con siders an imaging system which subsamples pixels in tensities with a Spatial Light Modulator (SLM) device. We present a general approach that can be applied to compressive ly sensed measurements gathered with respect to our sensing m odel, in order to improve reconstruction quality be yond a general reconstruction algorithm. The approach exploits cap turing overlapping subsequent frames in a panning c amera scene or capturing novel compressively sensed measurements o f the static camera scene by utilizing dynamic aper ture codes. We also consider the effects of projective distortions from various camera positions of subsequent frames within our approach. The result is a decrease in the effective compression rate of the system and therefore a sig nificantly improved compressively sensed reconstruction. Results are pr esented for various reconstruction algorithms on na tural, man-made, and mixed scenery of panning camera scenery as well as static camera scenery . Keywords: Coded Aperture, Compressive Imaging, Video Reconst ruction 1. INTRODUCTION The goal of Compressive Imaging is to reconstruct l arge full-scale images and video from data collecte d at a much lower resolution than the reconstructed image/video. The main benefit of such a scheme is that a much small er (and cheaper) sensor (CCD, Focal Plane Array, etc.) can be used w ithin imaging devices without noticeable loss of qu ality or frame-rate. For very expensive imaging devices, such as those used for Infrared (IR) sensors, this can amou nt to significant cost savings. While similar to traditional compres sion techniques, Compressive Imaging differs in tha t fewer data points are collected by the sensor and reconstructed at ru n-time. In contrast, current compression technique s require that all pixels be sampled by the sensor, and then compresse d. The current compression techniques do not have the advantage of smaller sensors in the way that Compressive Imaging does. Compressive Imaging reduces to the basic mod el of y = Ax that represents the compressively sensed measuremen ts “y” created by a sampling matrix “A” applied to the truth imagery/light sources “x”. The reader is referred t o [1] for further background on Compressive Imaging . When working with Compressively Sensed (CS) scene r econstructions for Compressive Imaging, there are t wo basic types of captured imagery to consider including sta tic camera imagery & panning camera imagery. With s tatic camera imagery, a camera is mounted in a fixed location at a fixed orientation as it captures data; however, with panning camera imagery, a camera is allowed to move to any positio n in free space with any orientation, such as an ai rcraft sensor that also pans as it captures the data. In [2], results for reconstructing static camera imagery with dynam ic scene content using an a priori dictionary-based approach were pr esented. In the approach presented in [2], KSVD [3] was used to create the dictionary and OMP [4] was used to recon struct the imagery. For this paper, the focus is on the topic of improving the quality of both static and panning re constructed imagery for several different reconstru ction algorithms (Least Squares, OMP, and NESTA [5]). Our approach to increasing the reconstructed qualit y is to exploit the occurrence of subsequent, overl apping frames that have substantial scene commonality. This approach i s valid for live implementation if the CS reconstru ction rate is greater than that of the sampling rate of the sense d scenes; for offline implementation, this is not a n issue as long as the sampling rate is high enough to produce image frame scene commonality/overlap. Offline implementation of this approach involves simply knowing the pattern of the imaging sensor’s CS pixels a priori such that thes e dynamic codes (which makeup the sampling matrix elements over tim e) are known. One general application is to allow l ess computationally intensive single frame reconstructi on algorithms to produce lower quality results that can then be greatly