Exploring speaker-specific characteristics with deep learning

Abstract

Speech signals convey different types of information which vary from linguistic to speaker-specific and should be used in different tasks. However, it is hard to extract a special type of information such that nearly all acoustic representations of speech present all kinds of information as a whole. The use of the same representation in different tasks creates a difficulty in achieving good performance in either speech or speaker recognition. In this paper, we present a deep neural architecture to explore speaker-specific characteristics from popular Mel-frequency cepstral coefficients. For learning, we propose an objective function consisting of contrastive cost in terms of speaker similarity and dissimilarity as well as data reconstruction cost used as regularization to normalize non-speaker related information. Learning deep architecture is done by a greedy layerwise local unsupervised training for initialization and a global supervised discriminative training for extracting a speaker-specific representation. By means of two narrow-band benchmark corpora, we demonstrate that our deep architecture generates a robust overcomplete speech representation in characterizing various speakers and the use of this new representation yields a favorite performance in speaker verification.