Deep learning for speech separation.

Abstract

Speech source separation aims to estimate one or more individual sources from mixtures of multiple sound sources, e.g. speech, noise and music. While humans have an innate ability to separate sources in a sound mixture, this is not a trivial task for computers. In this thesis, we study the problem of speech separation, with a varying degree of complexity with respect to room reverberation, the number of speech sources and the number of microphones available for capturing the sources. We focus on the stateof- the-art deep learning techniques, and investigate the problem of separating speech sources from binaural and B-format mixtures obtained in real reverberant rooms. First, we evaluate a baseline system for binaural speech separation, where fullyconnected Deep Neural Networks (DNNs) and spatial features, such as Interaural Level Difference (ILD) and Interaural Phase Difference (IPD), are used. We further extend this baseline by using the dropout technique to mitigate the overfitting problem and adding spectral features, such as the Log-Power Spectrogram (LPS), to improve the separation performance. Second, we develop a Convolutional Neural Networks (CNNs)-based binaural speech separation system. We then study the potential of using data augmentation techniques to improve speech separation quality. In particular, we introduce contextual frames expansion, by including the information from neighbouring time frames, before and after a given time frame. Finally, we study the use of deep learning methods for B-format recordings. This allows the pressure gradient information to be exploited, in addition to the widely used acoustic pressure information, for deriving the angular features for source separation. Extensive experiments have been performed on two datasets captured in five different rooms in the University of Surrey. The proposed methods are shown to offer improved performance over the state-of-the-art, in terms of separation quality and intelligibility.