Acoustic Word Embedding Based on Multi-Head Attention Quadruplet Network

Abstract

Acoustic word embedding (AWE) has become a mainstream method in low-resource Query-by-Example keywords search. This paper proposes an AWE based on a multi-head attention quadruplet network, which can learn the attention weight sequence for all time frames of bidirectional Long Short-Term Memory by a multi-head self-attentive mechanism to pay attention to the time position information. At the same time, we construct a differences order quadruplet loss to train the AWE model to adequately consider the relative and absolute distances between the positive and negative sample pairs. In addition, attention mechanism, differences order quadruplet loss, and word label information are combined to design an objective function so that the AWE vectors have a better feature expression in the embedded space. The experimental results show that the proposed method can improve the learning ability of the network and make the AWEs more identifiable. The above two points result in better performance in the word discrimination task.