Monotonic Gaussian regularization of attention for robust automatic speech recognition

Abstract

The Attention-based Encoder–Decoder (AED) models are one of the most popular models for Automatic Speech Recognition (ASR). However, instability can occur in AED with problems such as incorrect insertions or word repetitions due to the violation of the inherent monotonic alignment property. To address these problems, we propose a monotonic Gaussian regularization method to guide the attention training, where the guiding map is depicted as a sequence of Gaussian distributions with monotonically moving centers. Experiments show our method reduces the insertion error rate by a relative 7% on the HKUST dataset, relative 20% and 16% on two large industrial datasets, and a relative 21% on an out-of-domain test set. The overall Character Error Rates (CERs) are all reduced at the same time, indicating that the model’s recognition ability is well maintained. Therefore, our proposed method improves model performance by enhancing monotonic alignment, and provides better robustness.