A Multilingual ASR of Sepedi-English Code-Switched Speech for Automatic Language Identification

Abstract

This paper presents an integration of multilingual speech recognition into language identification (LID) for code-switched speech using phonotatic features as language information. A multilingual speech recognition system converts the spoken utterances into occurrences of phone sequences. The hidden Markov models (HMMs) are employed to build a multilingual acoustic models that can handle multiple languages within an utterance. We propose two phoneme clustering methods to determine the phoneme similarities among the target languages. A supervised machine learning technique is employed to learn the language transition of the phonotactic information given the phoneme sequences. The classification decision is made by support vector machines (SVM) technique which classifies language identity given the likelihood scores based on the phoneme occurrence segments. We experiments were performed using a mixed language speech corpus for Sepedi and English. We evaluate the ASR-LID system measuring the performance of the phone error rate (PER) and the LID classification accuracy portions separately. We obtained a lower PER on a system that employed data-driven phoneme clustering method which was modelled with 32-Gaussian mixtures per state. The proposed multilingual ASR-LID framework has achieved an acceptable recognition and classification accuracy on code-switched and monolingual speech respectively.