Camera tracking in virtual studio

Abstract

The virtual studio concept replaces real background sets with computer-generated synthetic scenes. In this paper, we discuss the optical landmark-based camera tracking method in virtual studio applications. Studio cameras are zoom lens imaging device, both the internal parameters and external parameters of the camera are allowed to vary from image to image. Optical tracking --uses pattern recognition--eliminates the need for painstaking calibration of the lens system and can be used with any camera mount, including hand-held cameras. Here we address the problem of accurately tracking the 3D motion and focus of a monocular camera in a known 3D environment and dynamically estimating the 3D location and focus of the camera. We utilize fully automated landmark-based camera calibration to initialize the motion estimation and employ extended Kalman filter techniques to track landmarks and to estimate the camera location and focus. The implementation of our approach has been proven to be efficient and robust and our system successfully tracks in real-time at approximately 25 Hz. This paper describes several years of work at the NUDT Multimedia laboratory to develop optical pattern recognition and tracking systems for use in virtual studio.