Automatic Scoring of L2 English Speech Based on DNN Acoustic Models with Lattice-Free MMI

Abstract

This paper proposed improved automatic scoring methods for L2 English speaking tests based on acoustic models with lattice-free Maximum Mutual Information (MMI). Deep Neural Network (DNN) acoustic modeling with lattice-free MMI is the state-of-the-art technology in speech recognition because of its effectiveness in sequential discriminative training. Novel Goodness of Pronunciation (GOP) implementations based on lattice free MMI were proposed to improve the performance of automatic scoring for L2 English speech tests. Sequential acoustic weights during forced-alignment and posteriors based on Forward-Backward Algorithm with lattice free MMI acoustic models were used to improved GOP based automatic scoring. Experimental results show that our proposed lattice free MMI based methods outperform conventional regular DNN based automatic scoring methods.