End-to-end speech recognition with reinforcement learning

Abstract

In recent years, end-to-end automatic speech recognition (ASR) based on deep neural networks become popular because of its simple pipeline and excellent performance. However, there exists a main mismatch between its training and testing that might lead to performance degradation: in the training stage, existing method use the maximum likelihood criterion which aims to maximize log-likelihood of the training data, while in the testing stage the performance is evaluated by word error rate (WER), not log-likelihood. In this paper, we propose an alternative method based on reinforcement learning to make the goals of training and testing more consistent. Viewing speech recognition as a sequence decision process, the encoder-decoder based neural network is used as the policy function. The encoder is a pre-trained speech representation model (Wav2vec2.0), which generates the environment state encoding. The decoder is trained using a policy gradient algorithm based on a mix reward function which reflects both the word error rate and language model score. Experimental results on the LibriSpeech corpus show that our proposed method achieves 4% relative improvements over the baseline with a language model in terms of WER.