Integrating monaural and binaural analysis for localizing multiple reverberant sound sources

Abstract

Localization of simultaneous sound sources in natural environments with only two microphones is a challenging problem. Reverberation degrades performance of localization based exclusively on directional cues. We present an approach that integrates monaural and binaural analysis to improve localization of multiple speech sources in noisy and reverberant environments. Our approach incorporates pitch-based monaural processing to perform simultaneous organization of voiced speech. We propose a probabilistic framework to jointly perform localization and sequential organization using binaural cues. We evaluate our system on multi-source speech mixtures in the presence of reverberation and diffuse noise and compare it to two localization approaches that do not incorporate monaural cues. Results indicate that our system can accurately localize multiple sources in very challenging conditions.