Deep Learning-Based Dereverberation for Sound Source Localization with Beamforming

Abstract

In this work, an algorithm that combines deep-learning based dereverberation and beamforming is proposed for sound source localization in the reverberant environment. The contribution is combining deep-learning based dereverberation and beamforming together. Through deep learning, the proposed algorithm can directly achieve dereverberation of the cross-spectral matrix of microphone array measurement signals in the reverberant environment, which can overcome the challenge of requiring multiple measurements like average beamforming. In this way, the proposed algorithm can be applied to the reverberant environment that requires efficient sound source localization. At the same time, the network of deep-learning based dereverberation is trained directly using the cross-spectral matrix of signals collected by microphone arrays, which can overcome the challenge of requiring prior knowledge of reflective surfaces like empirical dereverberation beamforming. In this way, the proposed algorithm can be applied to the reverberant environment composed of walls with unknown sound reflection coefficients. Both specular reflections, diffuse reflections, and background noise are considered in the reverberant environment. The cross-spectral matrix of microphone array measurement signals in the reverberant environment is first fed into the U-net for dereverberation and then combined with beamforming for sound source localization. The results in the test set show that the proposed algorithm can eliminate the influence of reverberation on sound source localization. The applicability limitation results of the proposed algorithm show the proposed algorithm’s strong robustness to unseen sound reflection coefficient, unseen SNR, and expandability to some degree for unseen source location. However, the proposed algorithm cannot provide stable and accurate sound source localization results under unseen room geometry. The proposed algorithm has higher spatial resolution and smaller errors than the average beamforming and the empirical dereverberation beamforming.