Abstract
Transformers have become popular in building end-to-end automatic speech recognition (ASR) systems. However, transformer ASR systems are usually trained to give output sequences in the left-to-right order, disregarding the right-to-left context. Currently, the existing transformer-based ASR systems that employ two decoders for bidirectional decoding are complex in terms of computation and optimization. The existing ASR transformer with a single decoder for bidirectional decoding requires extra methods (such as a self-mask) to resolve the problem of information leakage in the attention mechanism This paper explores different options for the development of a speech transformer that utilizes a single decoder equipped with bidirectional context embedding (BCE) for bidirectional decoding. The decoding direction, which is set up at the input level, enables the model to attend to different directional contexts without extra decoders and also alleviates any information leakage. The effectiveness of this method was verified with a bidirectional beam search method that generates bidirectional output sequences and determines the best hypothesis according to the output score. We achieved a word error rate (WER) of 7.65%/18.97% on the clean/other LibriSpeech test set, outperforming the left-to-right decoding style in our work by 3.17%/3.47%. The results are also close to, or better than, other state-of-the-art end-to-end models.
Highlights
Automatic speech recognition (ASR) is the process whereby an algorithm is used to generate a sequence of words from a given speech signal
We propose to explore different options and implement an improved speech transformer that relies on a single decoder equipped with bidirectional context embedding (BCE) for bidirectional decoding
We explored different options and implemented Bidirectional Context Embedding Transformer (Bi-CET), an improved later began to rise with much longer sequences
Summary
Automatic speech recognition (ASR) is the process whereby an algorithm is used to generate a sequence of words from a given speech signal. Traditional ASR systems usually consist of independent parts, such as an acoustic model, a pronunciation model, and a language model. These parts are trained separately and combined for model inference. The multi-head self-attention, which is a major component of the transformer, learns to directly connect related positions in the entire sequence. This allows the network to exploit longrange dependencies regardless of distance. This attention-based network has been found to be more parallelizable and can be trained faster than other end-to-end models, which are mostly based on recurrent neural networks (RNN)
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
