A Comparison of Hybrid and End-to-End ASR Systems for the IberSpeech-RTVE 2020 Speech-to-Text Transcription Challenge

Juan M Perero-Codosero,Luis A Hernández-Gómez,Fernando M Espinoza-Cuadros

doi:10.3390/app12020903

Juan M Perero-Codosero, Luis A Hernández-Gómez + Show 1 more

Open Access

https://doi.org/10.3390/app12020903

Copy DOI

Journal: Applied sciences	Publication Date: Jan 17, 2022
Citations: 7	License type: CC BY 4.0

Affiliation: Universidad Politécnica de Madrid

Abstract

This paper describes a comparison between hybrid and end-to-end Automatic Speech Recognition (ASR) systems, which were evaluated on the IberSpeech-RTVE 2020 Speech-to-Text Transcription Challenge. Deep Neural Networks (DNNs) are becoming the most promising technology for ASR at present. In the last few years, traditional hybrid models have been evaluated and compared to other end-to-end ASR systems in terms of accuracy and efficiency. We contribute two different approaches: a hybrid ASR system based on a DNN-HMM and two state-of-the-art end-to-end ASR systems, based on Lattice-Free Maximum Mutual Information (LF-MMI). To address the high difficulty in the speech-to-text transcription of recordings with different speaking styles and acoustic conditions from TV studios to live recordings, data augmentation and Domain Adversarial Training (DAT) techniques were studied. Multi-condition data augmentation applied to our hybrid DNN-HMM demonstrated WER improvements in noisy scenarios (about 10% relatively). In contrast, the results obtained using an end-to-end PyChain-based ASR system were far from our expectations. Nevertheless, we found that when including DAT techniques, a relative WER improvement of 2.87% was obtained as compared to the PyChain-based system.

Highlights

The advancement of deep learning techniques has been able to improve the performance of Automatic Speech Recognition (ASR) systems
We developed both hybrid and end-to-end ASR approaches exploring some techniques to improve the performance of Speech-to-Text tasks under IberSpeechRTVE 2020 Challenge
We showed that Hybrid Deep Neural Networks (DNNs)-Hidden Markov Model (HMM) can be adapted to the TV show domain by means of multi-condition data augmentation

Summary

Introduction

The advancement of deep learning techniques has been able to improve the performance of Automatic Speech Recognition (ASR) systems. Deep Neural Networks (DNNs) became a fundamental part of conventional hybrid ASR systems [1]. Whilst hybrid systems need to use Hidden Markov Model (HMM) state probabilities to train the outputs of a DNN, end-to-end systems are trained to map an input feature sequence to a sequence of characters [6,7]. The independence of intermediate modeling (e.g., acoustic, pronunciation, and language models) makes it easier to build an ASR model. They neither require any phoneme alignment for framewise cross-entropy, nor a sophisticated beam search decoder [8]

Methods

Results

Conclusion