Vocabulary Expansion for the Sub-word WFST-Based Automatic Speech Recognition System

Abstract

AbstractThis paper reports the improvement of the Lithuanian Automatic Speech Recognition (ASR) system focusing on “vocabulary expansion”, i.e. enabling ASR system to recognize words never seen during training. These unseen words are called out-of-vocabulary (OOV) words and involve: 1) regular Lithuanian words appearing due to different topics or domains not covered in training; 2) complicated cases, i.e., foreign names, brand names, and loanwords pronounced not according to regular Lithuanian pronunciation rules. In weighted finite-state transducer (WFST) ASR OOV problem is typically solved by applying one of the following solutions: (1) making ASR vocabulary unlimited by performing recognition on sub-word level, (2) adding words directly to the WFST decoding graph or (3) by reconstruction of OOV words from ASR result. Our baseline Lithuanian ASR system already follows the first approach, however many OOV words are still not being recognized, because of low probability of corresponding sub-word sequences. Therefore, our offered approach can be seen as a combination of the first two solutions: we boost probabilities of sequences of sub-words (corresponding to words being “added” to vocabulary) in a sub-word weighted finite-state transducer (WFST) ASR system. In such way the vocabulary of the ASR is being “expanded”. The proposed approach allowed to achieve significant improvement over the baseline: the percentage of misrecognized out-of-vocabulary words dropped by \(\sim \)7%, while F1 reached 85.6%.KeywordsSpeech recognitionOut-of-vocabularyVocabulary expansionSub-word unitsThe lithuanian language