A self-calibrated speaker tracking system using both audio and video data

Abstract

In this paper, a self-calibrated speaker tracking system applied to both image tracking and sound source detection is proposed. The sound source estimated by microphone system is used to supply video tracking system. On the other hand, the direction of speaker detected by image tracking system is fed back to calibrate the phase distortion between microphones. Some experiments using actual devices are also done to test the performance of the proposed method. It is shown that significant improvement in source localization accuracy is possible when the camera and the microphone system are well calibrated.