Multi-camera Video Resolution Enhancement by Fusion of Spatial Disparity and Temporal Motion fields

Abstract

We consider the problem of spatio-temporal resolution enhancement of a dynamic video scene by synergistically combining information from multiple video sequences corresponding to the different views of the scene. While prior work has focused on spatio-temporal superresolution by primarily exploiting the blurring correlation induced by the scene imaging process, in this work we exploit another and perhaps a stronger form of correlation the intrinsic correlation present in the dynamic scene. We use the knowledge base from the areas of multi-view computer vision, video coding and signal/image processing to efficiently model the dynamic video scene by exploiting inter-camera (space) as well as intra-camera correlations (time). We present simulation results corresponding to a lowframe rate two-camera stereo video configuration where the sequences are combined to generate higher frame-rate sequences. Our proposed framework has direct application to problems such as multi-view video surveillance and relates closely to problems such as multi-view video compression.