Real-time integral imaging pickup system using camera array

Abstract

Three-dimensional (3D) information acquirement in integral imaging (II) is becoming popular with the development of inexpensive digital image sensors. In this study, we present a real-time II pickup system in which multiple video streams are recorded, compressed, transmitted, and synthesized for II display. The system consists of an 8×8 network camera array, three Gigabit Ethernet switches, a single PC, and an II display. With the help of multi-thread technology, multi-view video streams from the network camera array are first captured in real-time. Then, we adopt camera array calibration and image alignment methods based on planar parallax to achieve good image quality. Finally, the aligned video streams are interweaved to synthesize the elemental image array (EIA) for the II display. We also present a remapping algorithm to accelerate the generation of the EIA. In the remapping algorithm, the mapping matrices from original parallaxes to the EIA are directly calculated. Therefore, the camera array calibration process, the parallax alignment process and the synthesizing process are simplified into a simple remapping operation. The video stream processing is fully performed on a graphics processing unit (GPU). Our proposed system achieves high-quality II video rendering at up to 37fps depending on rendering parameters. Experimental results verify the feasibility of the proposed system and high-quality EIAs synthesized from dynamic 3D scene.