Application of pronunciation knowledge on phoneme recognition by LSTM neural network

Abstract

When applied for phoneme recognition, the Connectionist Temporal Classification (CTC) objective function allows a neural network to be trained with the phoneme level transcriptions of training utterances. A limitation of the CTC is that it can not be applied directly for network training with large speech corpora, since those corpora usually only have word level transcriptions. This work extends the CTC such that a novel objective function can be evaluated even if only the word level transcriptions are available. Furthermore, various pronunciation knowledge is adopted to construct pronunciation networks which can model the pronunciations of connected speech more accurately. When combined with a bidirectional Long Short-term Memory (LSTM) network, the extended CTC achieves a phoneme error rate of 18.3% on the LibriSpeech corpus. When various pronunciation knowledge is applied, the error rate is further reduced by 18.6% relatively.