A Multi-channel Speech Separation System for Unknown Number of Multiple Speakers

Abstract

This paper presents a multi-channel speech separation system for an unknown number of speakers. It can be applied to cases with a different number of speakers using a single model by iterative speech separation based on beam signal. It first determines the spatial directions where speakers are located (Direction of Arrival, DOA), and then the beam signals in each direction are obtained with spectral features, spatial features, and directional features by deep neural networks. Finally, the iterative speech separation is performed on the basis of the beam signals. Experimental evaluations show that the proposed method is better than the multi-channel Permutation Invariant Training (PIT) and Deep Clustering (DPCL) for an unknown number of speakers and the one-and-rest speech separation method. Besides, the system can still keep a relatively good separation performance even though the number of speakers is enlarged to 9.