Real-time arbitrary-view acquisition system by using random access IBR camera array

Abstract

Image-based rendering (IBR) is a powerful tool for synthesizing arbitrary-views of a real scene. So far, almost all the IBR researches are limited to static scenes because of the difficulty in simultaneously capturing a dynamic scene by a number of cameras. Although IBR needs many images of the real scene from different view positions, only a small number of pixels per each image are used for the synthesis. We propose a capturing system for real-time IBR for dynamic scenes, which uses special random access image sensors. In the proposed system, image sensors selectively outputs pixels that are required for synthesis and the selection of the pixels are dynamically changed according to the position of the virtual view. We designed and prototyped CMOS random access image sensors for the real-time IBR. By using sixteen of those sensors, we build a random access IBR camera array system. In this paper, we describe the design and the experimental results of the camera array system.