Speech signals separation: a new approach exploiting the coherence of audio and visual speech

Abstract

We present a new approach to the source separation problem in the case of multiple speech signals. The method is based on the use of automatic lip reading: the objective is to extract an acoustic speech signal from other acoustic signals by exploiting its coherence with the speaker's lip movements. For this aim, a statistical model is used to quantify this coherence. The results, while very preliminary, are encouraging. They show that this method can achieve a good separation of a speech source in the case of simple 2/spl times/2 additive mixtures. Moreover, it presents some interesting complementarity with traditional pure audio techniques.