Real‐time rendering system of human figures from arbitrary viewpoints using multiple cameras

Abstract

AbstractTo enable face‐to‐face communication between remote users, we developed a prototype system that projects real‐time realistic images of human figures into a virtual space. Since real‐time responsiveness, in particular, is required for this kind of use, the prototype equipment has the following special features. First, it uses a simple technique that extracts images captured by multiple cameras and projects them onto a plane located in the virtual space. Since the geometric shape of the human figure need not be obtained when using this technique, the required computing power is extremely small. The generated image error becomes large if the subject is not in the vicinity of the virtual plane. We formulated this by defining a fixed range within which the subject should exist as the “equivalent depth of field.” Also, among the video signals obtained by the multiple cameras, the implemented system selects only the ones required for image generation and enters them into the computer in terms of scan line units. By selectively discarding information at an early stage in this way, a high‐precision image can be generated without degradation even if there are few video‐capture boards. These special features enabled us to show the effectiveness of the proposed technique by successfully generating real‐time images using a consumer‐oriented, low‐cost personal computer. © 2003 Wiley Periodicals, Inc. Syst Comp Jpn, 34(11): 91–100, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.1216