Non-native English speech recognition using bilingual English lexicon and acoustic models

Abstract

This paper proposes an English speech recognition system which can recognize both non-native (i.e. Japanese) and native English speaker's pronunciation of English speech. The system uses a bilingual pronunciation lexicon in which each word has both English and Japanese phoneme transcriptions. The Japanese transcription is constructed considering typical Japanese pronunciation of English. Japanese and English acoustic models are used in recognizing both transcriptions, and the highest-likelihood word sequence obtained in combining with native English- and Japanese-pronounced words is the recognition results. Continuous speech recognition experiments show that the proposed system greatly improves Japanese-English speech recognition performance while maintaining the same performance levels as that of a purely native English recognition system.