Scalable neural network based language identification from written text

Abstract

Automatic language identification is an integral part of multilingual automatic speech recognition (ASR) and synthesis systems. We propose a novel scalable method for neural network based language identification from written text. The developed algorithm is further deployed in a multilingual ASR system. The developed algorithm is particularly proposed for embedded implementation platforms with sparse memory resources. With the proposed approach, high rates of both language identification and recognition are achieved across several languages with a compact size of the language identification model. The major benefit of the approach is that the neural network based language identification model can be scaled to meet the memory requirements set by the target platform while maintaining the language identification accuracy of the baseline system. The experiments show that the suggested scalable approach can save more than 50% memory while the performance is comparable to that of the baseline system. The performance is also verified in a multilingual speech recognition task.