Text-dependent Speaker Verification Research Articles

On Training Targets and Activation Functions for Deep Representation Learning in Text-Dependent Speaker Verification

Deep representation learning has gained significant momentum in advancing text-dependent speaker verification (TD-SV) systems. When designing deep neural networks (DNN) for extracting bottleneck (BN) features, the key considerations include training targets, activation functions, and loss functions. In this paper, we systematically study the impact of these choices on the performance of TD-SV. For training targets, we consider speaker identity, time-contrastive learning (TCL), and auto-regressive prediction coding, with the first being supervised and the last two being self-supervised. Furthermore, we study a range of loss functions when speaker identity is used as the training target. With regard to activation functions, we study the widely used sigmoid function, rectified linear unit (ReLU), and Gaussian error linear unit (GELU). We experimentally show that GELU is able to reduce the error rates of TD-SV significantly compared to sigmoid, irrespective of the training target. Among the three training targets, TCL performs the best. Among the various loss functions, cross-entropy, joint-softmax, and focal loss functions outperform the others. Finally, the score-level fusion of different systems is also able to reduce the error rates. To evaluate the representation learning methods, experiments are conducted on the RedDots 2016 challenge database consisting of short utterances for TD-SV systems based on classic Gaussian mixture model-universal background model (GMM-UBM) and i-vector methods.

Text-dependent speaker verification using discrete wavelet transform based on linear prediction coding

Presenting a system for verifying the identity of people so that the data could be recorded in the simplest possible way, performing identity verification with acceptable accuracy, and being robust to attacks and noises is one of some important issues in today's era. This paper presents a text-dependent speaker verification system in which features are extracted from the data using a method called discrete wavelet transform based on linear prediction coding. In addition to the conventional classification in these systems, a new method for classification in which one binary classification will perform for all speakers of the dataset can also be done, which will be introduced in this research. Then the performance of the designed system on ASVspoof 2015, ASVspoof 2017, ASVspoof 2019, AudioMNIST, and TIMIT datasets will be evaluated using accuracy, precision, sensitivity, specificity, F1-score, and equal error rate (EER) metrics. It will be shown that the EER for the case where classification is done once for each speaker is 0.14±0.83% for the ASVspoof 2017 dataset and the accuracy and EER for the case where classification is done once for all speakers is 100±0.00% and 0.00±0.00% for all datasets.

Model-Agnostic Meta-Learning for Fast Text-Dependent Speaker Embedding Adaptation

By constraining the lexical content of input speech, text-dependent speaker verification (TD-SV) offers more reliable performance than text-independent speaker verification (TI-SV) when dealing with short utterances. Because speech with constrained lexical content is harder to collect, often TD models are fine-tuned from a TI model using a small target phrase dataset. However, sometimes the target phrase dataset is too tiny for fine-tuning, which is the main obstacle for deploying TD-SV. One solution is to fine-tune the model using medium-size multi-phrase TD data and then deploy the model on the target phrase. Although this strategy does help in some cases, the performance is still sub-optimal because the model is not optimized for the target phrase. Inspired by the recent progress in meta-learning, we propose a three-stage pipeline for adapting a TI model to a TD model for the target phrase. Firstly, a TI model is trained using a large amount of speech data. Then, we use a multi-phrase TD dataset to tune the TI model via model-agnostic meta-learning. Finally, we perform fast adaptation using a small target phrase dataset. Results show that the three-stage pipeline consistently outperforms multi-phrase and target phrase fine-tuning.

Class token and knowledge distillation for multi-head self-attention speaker verification systems

This paper explores three novel approaches to improve the performance of speaker verification (SV) systems based on deep neural networks (DNN) using Multi-head Self-Attention (MSA) mechanisms and memory layers. Firstly, we propose the use of a learnable vector called Class token to replace the average global pooling mechanism to extract the embeddings. Unlike global average pooling, our proposal takes into account the temporal structure of the input what is relevant for the text-dependent SV task. The class token is concatenated to the input before the first MSA layer, and its state at the output is used to predict the classes. To gain additional robustness, we introduce two approaches. First, we have developed a new sampling estimation of the class token. In this approach, the class token is obtained by sampling from a list of several trainable vectors. This strategy introduces uncertainty that helps to generalize better compared to a single initialization as it is shown in the experiments. Second, we have added a distilled representation token for training a teacher-student pair of networks using the Knowledge Distillation (KD) philosophy, which is combined with the class token. This distillation token is trained to mimic the predictions from the teacher network, while the class token replicates the true label. All the strategies have been tested on the RSR2015-Part II and DeepMine-Part 1 databases for text-dependent SV, providing competitive results compared to the same architecture using the average pooling mechanism to extract average embeddings.

Enhancement and modification of automatic speaker verification by utilizing hidden Markov model

<div class="WordSection1"><p>The purpose of this study is to discuss the design and implementation of autonomous surface vehicle (ASV) systems. There’s a lot riding on the advancement and improvement of ASV applications, especially given the benefits they provide over other biometric approaches. Modern speaker recognition systems rely on statistical models like hidden Markov model (HMM), support vector machine (SVM), artificial neural networks (ANN), generalized method of moments (GMM), and combined models to identify speakers. Using a French dataset, this study investigates the effectiveness of prompted te xt speaker verification. At a context-free, single mixed mono phony level, this study has been constructing a continuous speech system based on HMM. After that, suitable voice data is used to build the client and world models. In order to verify speakers, the text-dependent speaker ver-ification system uses sentence HMM that have been concatenated for the key text. Normalized log-likelihood is determined from client model forced by Viterbi algorithm and world model, in the verification step as the difference between the log-likelihood. At long last, a method for figuring out the verification results is revealed.</p></div>

Pseudo-Phoneme Label Loss for Text-Independent Speaker Verification

Compared with text-independent speaker verification (TI-SV) systems, text-dependent speaker verification (TD-SV) counterparts often have better performance for their efficient utilization of speech content information. On this account, some TI-SV methods tried to boost performance by incorporating an extra automatic speech recognition (ASR) component to explore content information, such as c-vector. However, the introduced ASR component requires a large amount of annotated data and consumes high computation resources. In this paper, we propose a pseudo-phoneme label (PPL) loss for the TI-SR task by integrating content cluster loss at the frame level and speaker recognition loss at the segment level in a unified network by multitask learning, without additional data requirement and exhausting computation. By referring to HuBERT, we generate pseudo-phoneme labels to adjust a frame level feature distribution by deep cluster to ensure each cluster corresponds to an implicit pronunciation unit in the feature space. We compare the proposed loss with the softmax loss, center loss, triplet loss, log-likelihood-ratio cost loss, additive margin softmax loss and additive angular margin loss on the VoxCeleb database. Experimental results demonstrate the effectiveness of our proposed method.

Exploring single channel speech separation for short-time text-dependent speaker verification

Exploring single channel speech separation for short-time text-dependent speaker verification

A Survey on Text-Dependent and Text-Independent Speaker Verification

Speaker verification (SV) aims to detect an individual’s identity from his/her voice. SV has been successfully applied in various areas such as access control, remote service customization, financial transactions, etc. Depending on whether the text content is pre-defined or not, SV can be text-dependent or text-independent. This paper reviews recent research on text-dependent SV (TD-SV) and text-independent SV (TI-SV). Because most modern SV systems apply deep learning methods to boost performance, we focus on the studies that use deep speaker embedding, a technique representing a person’s identity via a fixed-dimensional vector encoded from a variable-length utterance. Rather than detailing every existing SV system, we make an overview of the representative SV systems that have attracted wide attention. Furthermore, an increasing number of SV systems have been devoted to addressing real-world challenges such as reverberation and noise, and this has driven a large number of studies on practical SV. Therefore, the survey compares the existing SV systems in the Far-Field Speaker Verification Challenge 2020 (FFSVC 2020) to illustrate the most effective techniques for both TD-SV and TI-SV.

Reflection of Conditional Independence Structure to Noise Variability for Noise Robust Text Dependent Speaker Verification

In the field of speaker verification (SV), the development of noise-robust systems is a challenge for their deployment in real-world environments. Noise variability compensation is a common strategy for increasing the robustness to noise variations. The performance of noise compensation depends on how well the noise variability, which is inherent in within-class variability, is estimated. However, to date, there is no information about the true noise variability that could reduce the gap between the empirical and true statistics. Most studies assume that true noise covariates are independent. This study aims to demonstrate the assumption that true noise variability has a conditional independence structure rather than an independence structure. This assumption was motivated by our previous findings, which revealed that optimal within-class variability has a conditional independence structure in text-dependent speaker verification (TD-SV) in clean environments. This indicates that the optima of all the variabilities in within-class variability, except noise variability, has a conditional independence structure; however, it is unknown whether this is also true for optimal noise variability. Our assumption was supported by the experimental results obtained under noisy TD-SV trials using systems built with graphical least absolute shrinking and selection operator-based probabilistic linear discriminant analysis, which achieved up to 10% relative equal error rate improvements.

ADCF Loss Function for Deep Metric Learning in End-to-End Text-Dependent Speaker Verification Systems

Metric learning approaches have widely expanded to the training of Speaker Verification (SV) systems based on Deep Neural Networks (DNNs), by using a loss function more consistent with the evaluation process than the traditional identification losses. However, these methods do not consider the performance measure and can involve high computational cost, for example, the need for a careful pair or triplet data selection. This paper proposes the approximated Detection Cost Function (aDCF) loss, which is a loss function based on the measure of the decision errors in SV systems, namely the False Rejection Rate (FRR) and the False Acceptance Rate (FAR). With aDCF loss as the training objective function, the end-to-end system learns how to minimize decision errors. Furthermore, we replace the typical linear layer as the last layer of DNN by a cosine distance layer, which reduces the difference between the metric in the training process and the metric during evaluation. aDCF loss function was evaluated in RSR2015-Part I and RSR2015-Part II datasets for text-dependent speaker verification. The system trained with aDCF loss outperforms all the state-of-the-art functions employed in this paper in both parts of the database.

On Training Targets and Activation Functions for Deep Representation Learning in Text-Dependent Speaker Verification

On Training Targets and Activation Functions for Deep Representation Learning in Text-Dependent Speaker Verification

HiLAM-aligned kernel discriminant analysis for text-dependent speaker verification

Probabilistic Linear Discriminant Analysis (PLDA) has been a commonly used backend classifier for many text-dependent speaker verification (TDSV) systems. Lately, PLDA projections have been integrated with the traditional Dynamic Time Warping (DTW) template matching framework, resulting in the DTW Online i-vector/PLDA system. The system is shown to achieve state-of-the-art performance for TDSV task. PLDA model serves to train a subspace that compensates for channel and session variabilities. It assumes linear separability between speaker-phrase information and other components. However, this relationship is known to be non-linear. The non-linearity is more prominent in case of short speech extracts, as in the case of the online i-vectors. This results in loss of vital speaker-phrase information at PLDA modeling. To this end, this work explores Kernel Discriminant Analysis (KDA) for TDSV task. It further proposes to use Hierarchical Multi-Layer Acoustic Model (HiLAM) to complement KDA with a more effective speaker-text class definition. The proposed system is hypothesized to benefit on three counts — non-linear modeling ability of KDA, speaker idiosyncrasy information associated with HiLAM-defined speaker-text units and modeling of the exact context of the pass-phrase, as offered by HiLAM. It shows a relative Equal Error Rate (EER) reduction of up to 50.63% on Part 1 of the RSR2015 database when compared to the baseline DTW Online i-vector/PLDA system.

Self-segmentation of pass-phrase utterances for deep feature learning in text-dependent speaker verification

In this paper, we propose a novel method to segment and label pass-phrase utterances for training deep neural network (DNN) bottleneck (BN) features for text-dependent speaker verification (TD-SV). Specifically, gender-dependent hidden Markov models (HMMs) for monophones are first trained using the pass-phrase utterances that are disjoint from evaluation. Next, the trained HMMs are speaker-adapted and then used for segmenting and labeling these training utterances at the phone level. The resulted labeled data is subsequently used for training DNN models to discriminate gender-dependent phones for the purpose of extracting phone-discriminant BN features. This is in contrast to conventional approaches that apply a general-purpose, speaker-independent automatic speech recognition (ASR) system for generating segmentation and labels. The proposed method eliminates the need for a separate ASR system, which can additionally have the disadvantage of mismatch with the pass-phrase utterances in terms languages, dialects, domains, acoustic conditions and so on. Experiments are conducted on the RedDots challenge 2016 database of TD-SV using short utterances with Gaussian mixture model-universal background model and i-vector techniques. Experimental results demonstrate that the proposed method yields lower error rates in TD-SV when compared to a set of existing methods. A thorough ablation study further confirms the effectiveness of the method. Fusion in both score and feature levels also shows the complementary nature of the proposed features.

Speaker-Phrase-Specific Adaptation of PLDA Model for Improved Performance in Text-Dependent Speaker Verification

The i-vector/probabilistic linear discriminant analysis (PLDA) framework has been popularly used in the field of speaker verification for a long time. Lately, the introduction of online i-vectors and its integration with dynamic time warping template matching technique have significantly improved the performance of text-dependent speaker verification system. The PLDA model learns to discriminate among instances of different speaker-phrase classes and also compensates for channel and session variability. However, when exposed to unseen speakers and text, the variability compensation model turns less than optimal, leading to substantial verification error. In this paper, PLDA adaptation, in order to incorporate the idea of speaker-phrase-dependent variability in the ivector/PLDA technique, has been proposed. The adapted model gets specifically tuned to particular speaker-phrase class, leading to a more optimal solution. Two adaptation techniques, namely interpolation and weighted likelihood, have been explored in this work. Experiments have been performed on Part 1 of the RSR2015 database, and relative equal error rate (EER) reductions of up to 58.22% and 45% have been observed for interpolation and weighted likelihood techniques, respectively. The use of speaker-phrase-specific mean and whitening parameters has led to further improvement, resulting in EER reduction of up to 20% relative to that of the adapted models.

Vocal Tract Length Perturbation for Text-Dependent Speaker Verification With Autoregressive Prediction Coding

In this letter, we propose a vocal tract length (VTL) perturbation method for text-dependent speaker verification (TD-SV), in which a set of TD-SV systems are trained, one for each VTL factor, and score-level fusion is applied to make a final decision. Next, we explore the bottleneck (BN) feature extracted by training deep neural networks with a self-supervised objective, autoregressive predictive coding (APC), for TD-SV and compare it with the well-studied speaker-discriminant BN feature. The proposed VTL method is then applied to APC and speaker-discriminant BN features. In the end, we combine the VTL perturbation systems trained on MFCC and the two BN features in the score domain. Experiments are performed on the RedDots challenge 2016 database of TD-SV using short utterances with Gaussian mixture model-universal background model and i-vector techniques. Results show the proposed methods significantly outperform the baselines.

Bidirectional Attention for Text-Dependent Speaker Verification.

Automatic speaker verification provides a flexible and effective way for biometric authentication. Previous deep learning-based methods have demonstrated promising results, whereas a few problems still require better solutions. In prior works examining speaker discriminative neural networks, the speaker representation of the target speaker is regarded as a fixed one when comparing with utterances from different speakers, and the joint information between enrollment and evaluation utterances is ignored. In this paper, we propose to combine CNN-based feature learning with a bidirectional attention mechanism to achieve better performance with only one enrollment utterance. The evaluation-enrollment joint information is exploited to provide interactive features through bidirectional attention. In addition, we introduce one individual cost function to identify the phonetic contents, which contributes to calculating the attention score more specifically. These interactive features are complementary to the constant ones, which are extracted from individual speakers separately and do not vary with the evaluation utterances. The proposed method archived a competitive equal error rate of 6.26% on the internal “DAN DAN NI HAO” benchmark dataset with 1250 utterances and outperformed various baseline methods, including the traditional i-vector/PLDA, d-vector, self-attention, and sequence-to-sequence attention models.

Regularized Within-Class Precision Matrix Based PLDA in Text-Dependent Speaker Verification

In the field of speaker verification, probabilistic linear discriminant analysis (PLDA) is the dominant method for back-end scoring. To estimate the PLDA model, the between-class covariance and within-class precision matrices must be estimated from samples. However, the empirical covariance/precision estimated from samples has estimation errors due to the limited number of samples available. In this paper, we propose a method to improve the conventional PLDA by estimating the PLDA model using the regularized within-class precision matrix. We use graphical least absolute shrinking and selection operator (GLASSO) for the regularization. The GLASSO regularization decreases the estimation errors in the empirical precision matrix by making the precision matrix sparse, which corresponds to the reflection of the conditional independence structure. The experimental results on text-dependent speaker verification reveal that the proposed method reduce the relative equal error rate by up to 23% compared with the conventional PLDA.

Attentional triplet neural networks for text-dependent speaker verification

Deep Neural Networks (DNNs) have been widely used in speech processing and show great performance on a range of tasks, such as speech recognition, machine translation, and speaker verification. In this paper we propose a new type of DNN model for text-dependent speaker verification. The frame-level features, being extracted by DNN, are usually equally weighted and aggregated (or averaged) to compute an utterance-level speaker representation. We combine Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) to extract frame-level features, then provide every frame with different weights produced by attention mechanism, so that utterance-level speaker representation can be generated by weighted averaging frame-level features. We explore different generating methods on the attention weights. Besides, attention mechanism can also be used to align temporal information between enrollment and evaluation utterance. Triplet loss function is used to optimize our models, requiring inputs group in triplet style. Ultimately, results of experiment on RSR2015 database show that our attention-based model outperforms various baseline models.

A Simple Distortion-Free Method to Handle Variable Length Sequences for Recurrent Neural Networks in Text Dependent Speaker Verification

Recurrent neural networks (RNNs) can model the time-dependency of time-series data. It has also been widely used in text-dependent speaker verification to extract speaker-and-phrase-discriminant embeddings. As with other neural networks, RNNs are trained in mini-batch units. In order to feed input sequences into an RNN in mini-batch units, all the sequences in each mini-batch must have the same length. However, the sequences have variable lengths and we have no knowledge of these lengths in advance. Truncation/padding are most commonly used to make all sequences the same length. However, the truncation/padding causes information distortion because some information is lost and/or unnecessary information is added, which can degrade the performance of text-dependent speaker verification. In this paper, we propose a method to handle variable length sequences for RNNs without adding information distortion by truncating the output sequence so that it has the same length as corresponding original input sequence. The experimental results for the text-dependent speaker verification task in part 2 of RSR 2015 show that our method reduces the relative equal error rate by approximately 1.3% to 27.1%, depending on the task, compared to the baselines but with an associated, small overhead in execution time.

HiLAM-state discriminative multi-task deep neural network in dynamic time warping framework for text-dependent speaker verification

This paper builds on a multi-task Deep Neural Network (DNN), which provides an utterance-level feature representation called j-vector, to implement a Text-dependent Speaker Verification (TDSV) system. This technique exploits the speaker idiosyncrasies associated with individual pass-phrases. However, speaker information is known to be characteristic of more specific speech units and, thus, it is likely that important speaker identity traits might get averaged out if it is considered as a coarse entity spread uniformly across the whole pass-phrase. This work attempts to overcome this limitation and devises a technique to leverage the finer speaker traits. It proposes to align the training data for Multi-task DNN using Hierarchical Multi-Layer Acoustic Model (HiLAM). HiLAM is an HMM-based text-dependent model that defines refined segments of a pass-phrase using Gaussian Mixture Model (GMM) states. This helps to exploit the speaker idiosyncrasies associated with finer and more specific segments of speech. Also, as HiLAM is built using the particular text in question, this alignment technique automatically takes care of the exact context of the speech units in the concerned pass-phrase. The proposed technique has been found to improve the performance of the system significantly. Integrating Dynamic Time Warping (DTW) with this technique leads to further improvement in the performance of the system. Experiments have been validated on Part 1 of RSR2015, RedDots, and NITS-TD databases. The best-performing proposed system achieves a relative Equal Error Rate (EER) reduction of up to 50.98% with respect to the baseline j-vector-based system for the overall test condition in case of RSR2015 database.