Permutation Invariant Training Based Single-Channel Multi-Talker Speech Recognition with Music Background

Abstract

This paper addresses the single-channel multi-talker speech recognition task with music background, where the speech recognition accuracy will deteriorate significantly. To improve the speech recognition accuracy with music background, we propose two approaches: 1) music-separation method to separate human speech from the music background, 2) permutation invariant training (PIT) for single-channel multi-talker, specifically two-talker, speech separation. Experimental results show that all the proposed methods can improve the speech recognition accuracy. Specifically, we use the music-separation method instead of the de-noising feature-mapping method to extract the human speech. Compared with the de-noising feature-mapping method, the music separation method achieves consistent improvement on the music-distorted speech separation and recognition tasks.