An End-to-End Model Based on Multiple Neural Networks with Data Augmentation for Keyword Spotting

Abstract

In this paper, we propose a network for small footprint keyword spotting. It includes four parts, data augmentation, Time-Delay Neural Network (TDNN) and Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), and attention mechanism. Data augmentation is Google SpecAugment with time warping and frequency mask and time mask to the spectrum on clean data set and noisy data set. TDNN and CNN model the spectrogram features from the time and space dimensions. RNN-type networks include RNN, Long Short-Term Memory network (LSTM), Gated Recurrent Unit (GRU), Bidirectional Long Short-Term Memory network (BiLSTM), and Bidirectional Gated Recurrent Unit (BiGRU). The RNN extracts hidden layer features and transforms them into high-level representations. The attention mechanism is selected to generate different weights and multiplied by the high-level representation generated by the RNN to obtain a fixed-length vector. Finally, we use a linear transformation and softmax function to generate scores. We also explored the size of attention mechanism, two attention mechanisms, rectified linear unit and hidden layer of RNN. Our model has achieved a true positive rate of 99.81% at a 5% false positive rate.