An emerging medium: interactive three-dimensional digital video

Abstract

This paper describes a new visual medium called interactive 3D digital video. 3D digital video displays motion pictures of real-world events from the view of a virtual camera controlled by the viewer during playback. For 3D video to become of practical use, sophisticated data manipulation, management and processing capabilities are required. These tasks are daunting, given the amount and complexity of data involved. Furthermore, due to the hybrid nature of the 3D video data, no standardized representation and coding schemes are available. We explore these issues and present an overview of a functional system called immersive video. For real events such as basketball games, immersive video analyzes and composites recorded multiviewpoint videos to create a full 3D version of the event which is then encoded and stored for immersive playback. While replaying this 3D digital movie, viewers are able to explore the scene continuously from any perspective. We describe the steps we have taken toward using the Internet infrastructure and a client-server configuration to allow World Wide Web users to interactively view any of our experimental 3D videos including a one minute staged karate demonstration captured by six video cameras. Applications of this new medium include telepresence, interactive video and television, video-based virtual environments, and immersive feature films.