Residual convolutional neural network with attentive feature pooling for end-to-end language identification from short-duration speech

Abstract

The problem of language identification from speech is tackled in this work. Residual convolutional neural networks are employed to this end, aiming at exploiting the ability of such architectures to take into account large contextual segments of input data. Moreover, learnable attention mechanisms are introduced on top of the convolutional stack for data-driven feature pooling across time, enabling the computation of fixed-dimension representations given varying-length speech segments as input. Training is performed using a combination of language identification and metric learning via triplet loss minimization, aimed at enforcing class separability within the embeddings space. Evaluation is performed across different conditions, such as multi-class classification, short-duration test utterances, and confusing languages, for the closed-set case, while open-set performance is evaluated with the introduction of unseen languages. At test time, end-to-end scoring along with cosine similarity and PLDA are employed, outperforming state-of-the-art benchmark methods, such as i-vectors by improving the average cost by 30% to 40% depending on the evaluation condition.