Pinyin as a Feature of Neural Machine Translation for Chinese Speech Recognition Error Correction

Abstract

Text correction after automatic speech recognition (ASR) is an important method to improve the speech recognition system. We regard the speech error correction as a translation task—from the language of bad Chinese to the language of good Chinese. We propose a speech recognition error correction algorithm based on neural machine translation (NMT) model. The algorithm is characterized by Chinese Pinyin coding, using a multilayer convolutional encoder-decoder with attention neural network. In the WeChat speech transcription data set we collected, our model substantially outperforms all prior neural approaches on this data set as well as the strong statistical machine translation-based systems. Our analysis shows the superiority of convolutional neural networks in capturing the local context via attention and thereby improving the coverage in speech transcription errors. By boosting multiple modes, using data augmentation and 3-gram language model tricks, our novel algorithm makes the error rate on the test set decreased by 26.2% on average. Our results show that using a multilayer convolutional encoder-decoder with Pinyin feature is able to achieve state-of-the-art performance in text correction after speech recognition.