I-vector System Research Articles

Depression detection based on linear and nonlinear speech features in I-vector/SVDA framework

This study proposes depression detection systems based on the i-vector framework for classifying speakers as depressed or healthy and predicting depression levels according to the Beck Depression Inventory-II (BDI-II). Linear and non-linear speech features are investigated as front-end features to i-vectors. To take advantage of the complementary effects of features, i-vector systems based on linear and non-linear features are combined through the decision-level fusion. Variability compensation techniques, such as Linear Discriminant Analysis (LDA) and Within-Class Covariance Normalization (WCCN), are widely used to reduce unwanted variabilities. A more generalizable technique than the LDA is required when limited training data are available. We employ a support vector discriminant analysis (SVDA) technique that uses the boundary of classes to find discriminatory directions to address this problem. Experiments conducted on the 2014 Audio-Visual Emotion Challenge and Workshop (AVEC 2014) depression database indicate that the best accuracy improvement obtained using SVDA is about 15.15% compared to the uncompensated i-vectors. In all cases, experimental results confirm that the decision-level fusion of i-vector systems based on three feature sets, TEO-CB-Auto-Env+Δ, Glottal+Δ, and MFCC+Δ+ΔΔ, achieves the best results. This fusion significantly improves classifying results, yielding an accuracy of 90%. The combination of SVDA-transformed BDI-II score prediction systems based on these three feature sets achieved RMSE and MAE of 8.899 and 6.991, respectively, which means 29.18% and 30.34% improvements in RMSE and MAE, respectively, over the baseline system on the test partition. Furthermore, this proposed combination outperforms other audio-based studies available in the literature using the AVEC 2014 database.

Идентификация голоса на основе i-вектора и глубоких нейронных сетей с использованием коротких высказываний

Text-independent voice recognition of the user using short sentences is a very difficult task due to the large spread and inconsistency of the content between short sentences, in order to improve user recognition by voice, it is planned to highlight several sets of distinguishing features that contain more information related to the voice. The results show that the i-vector DNN system is superior to the GMM i-vector system for various durations. However, the characteristics of both systems deteriorate significantly as the duration of the sentences decreases. To solve this problem, we propose two new nonlinear mapping methods that train DNN models to map i-vectors extracted from short sentences to their corresponding i-vectors of long sentences.

Bionic optimization of MFCC features based on speaker fast recognition

Surrounded by low SNR, how to make the voice faster and better recognize the owner has become a heated research topic. The human auditory system can accurately acquire the characteristics of acoustic events in complex systems or low SNR noise environment, which is of significance in the research of bionic hearing of human ear. The response curve of human ear output is obtained by bionic technology, which is the best response curve for sound enhancement to modify Mel filter. The method of adaptive threshold selection is used to integrate Mel features to realize the reduction and dynamic extraction of low SNR speech features. This method not only can resist the disadvantages of poor robustness and complexity of parameter model, but also obtain dynamic and comprehensive speech information of different speakers in different scenes. Finally, the improved CNN and I-vector system are contributed to reduce the dimension of the data and to verify the recognition, so as to achieve the optimal frequency selective amplification and simplification of the acoustic signal. In the case of SNR-5db, the model is reduced by 15% and the recognition accuracy is improved by 3%.

SpeakerGAN: Speaker identification with conditional generative adversarial network

Current methods based on the traditional i-vectors and deep neural network (DNN) have shown effectiveness on the speaker identification task, especially with the corpus of large scale. However, when the size of the training dataset is small, the overfitting problem may happen and lead to performance degradation. Besides, the robust identification still remains a challenging problem even under the less strict requirements. This paper proposes a novel approach, SpeakerGAN, for speaker identification with the conditional generative adversarial network (CGAN). It allows the adversarial networks for distinguishing real/fake samples and predicting class labels simultaneously. We configure the generator and the discriminator in SpeakerGAN with the gated convolutional neural network (CNN) and the modified residual network (ResNet) to obtain generated samples of high diversity as well as increase the network capacity. The multiple loss functions are combined and optimized to encourage the correct mapping and accelerate the convergence. Experimental results show that SpeakerGAN reduces the classification error rate by 87% and 16% compared with the traditional i-vector system and the state-of-the-art DNN based method. Under the scenario of limited training data, SpeakerGAN obtains significant improvement over the baselines. In the case of taking 1.6 s of each speaker for testing, SpeakerGAN achieves the identification accuracy of 98.20%, which suggests the promise for short-utterance speaker identification.

Emotional Variability Analysis Based I-Vector for Speaker Verification in Under-Stress Conditions

Emotional conditions cause changes in the speech production system. It produces the differences in the acoustical characteristics compared to neutral conditions. The presence of emotion makes the performance of a speaker verification system degrade. In this paper, we propose a speaker modeling that accommodates the presence of emotions on the speech segments by extracting a speaker representation compactly. The speaker model is estimated by following a similar procedure to the i-vector technique, but it considerate the emotional effect as the channel variability component. We named this method as the emotional variability analysis (EVA). EVA represents the emotion subspace separately to the speaker subspace, like the joint factor analysis (JFA) model. The effectiveness of the proposed system is evaluated by comparing it with the standard i-vector system in the speaker verification task of the Speech Under Simulated and Actual Stress (SUSAS) dataset with three different scoring methods. The evaluation focus in terms of the equal error rate (EER). In addition, we also conducted an ablation study for a more comprehensive analysis of the EVA-based i-vector. Based on experiment results, the proposed system outperformed the standard i-vector system and achieved state-of-the-art results in the verification task for the under-stressed speakers.

An end-to-end deep source recording device identification system for Web media forensics

PurposeMost source recording device identification models for Web media forensics are based on a single feature to complete the identification task and often have the disadvantages of long time and poor accuracy. The purpose of this paper is to propose a new method for end-to-end network source identification of multi-feature fusion devices.Design/methodology/approachThis paper proposes an efficient multi-feature fusion source recording device identification method based on end-to-end and attention mechanism, so as to achieve efficient and convenient identification of recording devices of Web media forensics.FindingsThe authors conducted sufficient experiments to prove the effectiveness of the models that they have proposed. The experiments show that the end-to-end system is improved by 7.1% compared to the baseline i-vector system, compared to the authors’ previous system, the accuracy is improved by 0.4%, and the training time is reduced by 50%.Research limitations/implicationsWith the development of Web media forensics and internet technology, the use of Web media as evidence is increasing. Among them, it is particularly important to study the authenticity and accuracy of Web media audio.Originality/valueThis paper aims to promote the development of source recording device identification and provide effective technology for Web media forensics and judicial record evidence that need to apply device source identification technology.

Self-attention based speaker recognition using Cluster-Range Loss

Speaker recognition with short utterances is a challenging research topic in the natural language processing (NLP) field. Previous convolutional neural network (CNN) based models for speaker recognition usually utilize very deep or wide layers, resulting in many parameters and high computational cost. Besides, great training difficulty and inefficiency exist in the triplet loss, which is widely used in speaker recognition. In this work, we propose to combine the residual network (ResNet) with the self-attention mechanism to achieve better performance in text-independent speaker recognition with fewer parameters and lower computational cost. In addition, the Cluster-Range Loss based on a well-designed online exemplar mining is proposed to directly shrink the intra-class variation and to enlarge the inter-class distance. Experiments on Voxceleb dataset are conducted to verify the effectiveness of the proposed scheme. The proposed approach achieves a Top-1 accuracy of 89.1% for speaker identification by jointly training the network with the Cluster-Range Loss and softmax cross entropy loss. For speaker verification, we achieve a competitive EER of 5.5% without any heavy-tailed backend, compared with the state-of-the-art i-vector system, as well as the x-vector system.

Speaker identification using multi-modal i-vector approach for varying length speech in voice interactive systems

The development in the interface of smart devices has lead to voice interactive systems. An additional step in this direction is to enable the devices to recognize the speaker. But this is a challenging task because the interaction involves short duration speech utterances. The traditional Gaussian mixture models (GMM) based systems have achieved satisfactory results for speaker recognition only when the speech lengths are sufficiently long. The current state-of-the-art method utilizes i-vector based approach using a GMM based universal background model (GMM-UBM). It prepares an i-vector speaker model from a speaker’s enrollment data and uses it to recognize any new test speech. In this work, we propose a multi-model i-vector system for short speech lengths. We use an open database THUYG-20 for the analysis and development of short speech speaker verification and identification system. By using an optimum set of mel-frequency cepstrum coefficients (MFCC) based features we are able to achieve an equal error rate (EER) of 3.21% as compared to the previous benchmark score of EER 4.01% on the THUYG-20 database. Experiments are conducted for speech lengths as short as 0.25 s and the results are presented. The proposed method shows improvement as compared to the current i-vector based approach for shorter speech lengths. We are able to achieve improvement of around 28% even for 0.25 s speech samples. We also prepared and tested the proposed approach on our own database with 2500 speech recordings in English language consisting of actual short speech commands used in any voice interactive system.

Combining evidences from excitation source and vocal tract system features for Indian language identification using deep neural networks

In this paper, a combination of excitation source information and vocal tract system information is explored for the task of language identification (LID). The excitation source information is represented by features extracted from linear prediction (LP) residual signal called the residual cepstral coefficients (RCC). Vocal tract system information is represented by the mel frequency cepstral coefficients (MFCC). In order to incorporate additional temporal information, shifted delta cepstra (SDC) are computed. An LID system is built using SDC over both MFCC and RCC features individually and evaluated based on their equal error rate (EER). Experiments have been performed on a dataset consisting of 13 Indian languages with about 115 h for training and 30 h for testing using a deep neural network (DNN), DNN with attention (DNN-WA) and a state-of-the-art i-vector system. DNN-WA outperforms the baseline i-vector system. An EER of 9.93 and 6.25% are achieved using RCC and MFCC features respectively. By combining evidence from both features using a late fusion mechanism, an EER of 5.76% is obtained. This result indicates the complementary nature of the excitation source information to that of the widely used vocal tract system information for the task of LID.

Data selection for i-vector based automatic speaker verification anti-spoofing

State-of-the-art i-vector based automatic speaker verification (ASV) systems lead to considerably high performance and thus voice becomes one of the most important biometric modality for person authentication. However, similar to other biometrics, ASV systems are highly vulnerable to spoofing attacks. Therefore, developing countermeasures for detecting spoofing attacks plays an important role against the concerns regarding the reliability of ASV systems. Recent studies have shown that simple Gaussian mixture model (GMM) classifier outperforms i-vector countermeasures. In this study, we focus on improving the spoofing detection performance of i-vector system using cosine and probabilistic linear discriminant analysis (PLDA) scoring. Experimental results conducted on ASVspoof 2015 database reveals that the data used to train the two key elements of i-vector system, universal background model (UBM) and the i-vector extractor (T-matrix), play an important role on spoofing detection performance. In this paper, we study the effect of using different type of data (genuine/human or spoofed) to train these two elements and their performance on spoofing detection. In particular, extracting i-vectors using UBM trained with genuine (human) speech utterances and T-matrix trained from both genuine and spoofed utterances leads to 50% performance improvement on spoofing detection. With the proposed scheme, unlike the previous results, i-vector countermeasure outperforms GMM classifier. Finally, experimental results shows that recently proposed constant Q cepstral coefficients (CQCC) shows superior performance in comparison to standard Mel-frequency cepstral coefficients (MFCC).

Histogram equalization using a reduced feature set of background speakers’ utterances for speaker recognition

We propose a method for histogram equalization using supplement sets to improve the performance of speaker recognition when the training and test utterances are very short. The supplement sets are derived using outputs of selection or clustering algorithms from the background speakers’ utterances. The proposed approach is used as a feature normalization method for building histograms when there are insufficient input utterance samples. In addition, the proposed method is used as an i-vector normalization method in an i-vector-based probabilistic linear discriminant analysis (PLDA) system, which is the current state-of-the-art for speaker verification. The ranks of sample values for histogram equalization are estimated in ascending order from both the input utterances and the supplement set. New ranks are obtained by computing the sum of different kinds of ranks. Subsequently, the proposed method determines the cumulative distribution function of the test utterance using the newly defined ranks. The proposed method is compared with conventional feature normalization methods, such as cepstral mean normalization (CMN), cepstral mean and variance normalization (MVN), histogram equalization (HEQ), and the European Telecommunications Standards Institute (ETSI) advanced front-end methods. In addition, performance is compared for a case in which the greedy selection algorithm is used with fuzzy C-means and K-means algorithms. The YOHO and Electronics and Telecommunications Research Institute (ETRI) databases are used in an evaluation in the feature space. The test sets are simulated by the Opus VoIP codec. We also use the 2008 National Institute of Standards and Technology (NIST) speaker recognition evaluation (SRE) corpus for the i-vector system. The results of the experimental evaluation demonstrate that the average system performance is improved when the proposed method is used, compared to the conventional feature normalization methods.

A Pashtu speakers database using accent and dialect approach

A small scale Pashtu speakers' database with multiple accents and dialects has been developed to use in Pashtu Speaker Identification Systems (SIS) and accents and dialect identification systems. Pashtu is a major spoken language of Pakistan and Afghanistan. At present, it has become very prominent worldwide due to its regional importance. The regions of Pakistan and Afghanistan where Pashtu is spoken are mostly occupied by the extremists who use Pashtu for their communication. In order to design Pashtu voice-based systems for security and other applications, a database has been designed in which the voice data is collected from 32 native Pashtu speakers of different regions of Pakistan and Afghanistan. Finally, using a subset of the data, Multi-Layer Perceptron (MLP)-based SIS has been designed. The designed system achieved overall 87.5% identification accuracy and outperformed the recently proposed i-vector and GMM-based accent identification systems by showing 3.8% and 12.0% relative improvement respectively.

On the use of deep feedforward neural networks for automatic language identification

In this work, we present a comprehensive study on the use of deep neural networks (DNNs) for automatic language identification (LID). Motivated by the recent success of using DNNs in acoustic modeling for speech recognition, we adapt DNNs to the problem of identifying the language in a given utterance from its short-term acoustic features. We propose two different DNN-based approaches. In the first one, the DNN acts as an end-to-end LID classifier, receiving as input the speech features and providing as output the estimated probabilities of the target languages. In the second approach, the DNN is used to extract bottleneck features that are then used as inputs for a state-of-the-art i-vector system. Experiments are conducted in two different scenarios: the complete NIST Language Recognition Evaluation dataset 2009 (LRE'09) and a subset of the Voice of America (VOA) data from LRE'09, in which all languages have the same amount of training data. Results for both datasets demonstrate that the DNN-based systems significantly outperform a state-of-art i-vector system when dealing with short-duration utterances. Furthermore, the combination of the DNN-based and the classical i-vector system leads to additional performance improvements (up to 45% of relative improvement in both EER and Cavg on 3s and 10s conditions, respectively).

Improving speaker verification performance against long-term speaker variability

Speaker verification performance degrades when input speech is tested in different sessions over a long period of time chronologically. Common ways to alleviate the long-term impact on performance degradation are enrollment data augmentation, speaker model adaptation, and adapted verification thresholds. From a point of view in features of a pattern recognition system, robust features that are speaker-specific, and invariant with time and acoustic environments are preferred to deal with this long-term variability. In this paper, with a newly created speech database, CSLT-Chronos, specially collected to reflect the long-term speaker variability, we investigate the issues in the frequency domain by emphasizing higher discrimination for speaker-specific information and lower sensitivity to time-related, session-specific information. F-ratio is employed as a criterion to determine the figure of merit to judge the above two sets of information, and to find a compromise between them. Inspired by the feature extraction procedure of the traditional MFCC calculation, two emphasis strategies are explored when generating modified acoustic features, the pre-filtering frequency warping and the post-filtering filter-bank outputs weighting are used for speaker verification. Experiments show that the two proposed features outperformed the traditional MFCC on CSLT-Chronos. The proposed approach is also studied by using the NIST SRE 2008 database in a state-of-the-art, i-vector based architecture. Experimental results demonstrate the advantage of proposed features over MFCC in LDA and PLDA based i-vector systems.

Language Identification in Short Utterances Using Long Short-Term Memory (LSTM) Recurrent Neural Networks.

Long Short Term Memory (LSTM) Recurrent Neural Networks (RNNs) have recently outperformed other state-of-the-art approaches, such as i-vector and Deep Neural Networks (DNNs), in automatic Language Identification (LID), particularly when dealing with very short utterances (∼3s). In this contribution we present an open-source, end-to-end, LSTM RNN system running on limited computational resources (a single GPU) that outperforms a reference i-vector system on a subset of the NIST Language Recognition Evaluation (8 target languages, 3s task) by up to a 26%. This result is in line with previously published research using proprietary LSTM implementations and huge computational resources, which made these former results hardly reproducible. Further, we extend those previous experiments modeling unseen languages (out of set, OOS, modeling), which is crucial in real applications. Results show that a LSTM RNN with OOS modeling is able to detect these languages and generalizes robustly to unseen OOS languages. Finally, we also analyze the effect of even more limited test data (from 2.25s to 0.1s) proving that with as little as 0.5s an accuracy of over 50% can be achieved.

Unsupervised accent classification for deep data fusion of accent and language information

Automatic Dialect Identification (DID) has recently gained substantial interest in the speech processing community. Studies have shown that the variation in speech due to dialect is a factor which significantly impacts speech system performance. Dialects differ in various ways such as acoustic traits (phonetic realization of vowels and consonants, rhythmical characteristics, prosody) and content based word selection (grammar, vocabulary, phonetic distribution, lexical distribution, semantics). The traditional DID classifier is usually based on Gaussian Mixture Modeling (GMM), which is employed as baseline system. We investigate various methods of improving the DID based on acoustic and text language sub-systems to further boost the performance. For acoustic approach, we propose to use i-Vector system. For text language based dialect classification, a series of natural language processing (NLP) techniques are explored to address word selection and grammar factors, which cannot be modeled using an acoustic modeling system. These NLP techniques include: two traditional approaches, including N-Gram modeling and Latent Semantic Analysis (LSA), and a novel approach based on Term Frequency–Inverse Document Frequency (TF-IDF) and logistic regression classification. Due to the sparsity of training data, traditional text approaches do not offer superior performance. However, the proposed TF-IDF approach shows comparable performance to the i-Vector acoustic system, which when fused with the i-Vector system results in a final audio-text combined solution that is more discriminative. Compared with the GMM baseline system, the proposed audio-text DID system provides a relative improvement in dialect classification performance of +40.1% and +47.1% on the self-collected corpus (UT-Podcast) and NIST LRE-2009 data, respectively. The experiment results validate the feasibility of leveraging both acoustic and textual information in achieving improved DID performance.

Study of Senone-Based Deep Neural Network Approaches for Spoken Language Recognition

This paper compares different approaches for using deep neural networks (DNNs) trained to predict senone posteriors for the task of spoken language recognition (SLR). These approaches have recently been found to outperform various baseline systems on different datasets, but they have not yet been compared to each other or to a common baseline. Two of these approaches use the DNNs to generate feature vectors which are then processed in different ways to predict the score of each language given a test sample. The features are extracted either from a bottleneck layer in the DNN or from the output layer. In the third approach, the standard i-vector extraction procedure is modified to use the senones as classes and the DNN to predict the zeroth order statistics. We compare these three approaches and conclude that the approach based on bottleneck features followed by i-vector modeling outperform the other two approaches. We also show that score-level fusion of some of these approaches leads to gains over using a single approach for short-duration test samples. Finally, we demonstrate that fusing systems that use DNNs trained with several languages leads to improvements in performance over the best single system, and we propose an adaptation procedure for DNNs trained with languages with less available data. Overall, we show improvements between 40% and 70% relative to a state-of-the-art Gaussian mixture model (GMM) i-vector system on test durations from 3 seconds to 120 seconds on two significantly different tasks: the NIST 2009 language recognition evaluation task and the DARPA RATS language identification task.

A study on the roles of total variability space and session variability modeling in speaker recognition

Speaker verification (SV) using i-vector concept becomes state-of-the-art. In this technique, speakers are projected onto the total variability space and represented by vectors called i-vectors. During testing, the i-vectors of the test speech segment and claimant are conditioned to compensate for the session variability before scoring. So, i-vector system can be viewed as two processing blocks: one is total variability space and the other is post-processing module. Several questions arise, such as, (i) which part of the i-vector system plays a major role in speaker verification: total variability space or post-processing task; (ii) is the post-processing module intrinsic to the total variability space? The motivation of this paper is to partially answer these questions by proposing several simpler speaker characterization systems for speaker verification, where speakers are represented by their speaker characterization vectors (SCVs). The SCVs are obtained by uniform segmentation of the speakers gaussian mixture models (GMMs)- and maximum likelihood linear regression (MLLR) super-vectors. We consider two adaptation approaches for GMM super-vector: one is maximum a posteriori and other is MLLR. Similarly to the i-vector, SCVs are post-processed for session variability compensation during testing. The proposed system shows promising performance when compared to the classical i-vector system which indicates that the post-processing task plays an major role in i-vector based SV system and is not intrinsic to the total variability space. All experimental results are shown on NIST 2008 SRE core condition.

Linguistically-constrained formant-based i-vectors for automatic speaker recognition

This paper presents a large-scale study of the discriminative abilities of formant frequencies for automatic speaker recognition. Exploiting both the static and dynamic information in formant frequencies, we present linguistically-constrained formant-based i-vector systems providing well calibrated likelihood ratios per comparison of the occurrences of the same isolated linguistic units in two given utterances. As a first result, the reported analysis on the discriminative and calibration properties of the different linguistic units provide useful insights, for instance, to forensic phonetic practitioners. Furthermore, it is shown that the set of units which are more discriminative for every speaker vary from speaker to speaker. Secondly, linguistically-constrained systems are combined at score-level through average and logistic regression speaker-independent fusion rules exploiting the different speaker-distinguishing information spread among the different linguistic units. Testing on the English-only trials of the core condition of the NIST 2006 SRE (24,000 voice comparisons of 5 minutes telephone conversations from 517 speakers -219 male and 298 female-), we report equal error rates of 9.57 and 12.89% for male and female speakers respectively, using only formant frequencies as speaker discriminative information. Additionally, when the formant-based system is fused with a cepstral i-vector system, we obtain relative improvements of ∼6% in EER (from 6.54 to 6.13%) and ∼15% in minDCF (from 0.0327 to 0.0279), compared to the cepstral system alone.

Phonotactic language recognition using dynamic pronunciation and language branch discriminative information

This paper presents our study of phonotactic language recognition system using dynamic pronunciation and language branch discriminative information. The theory of language branch in linguistics is introduced to language recognition, and phonotactic language branch variability (PLBV) method based on factor analysis is proposed. In our work, phoneme variability factor containing dynamic pronunciation information is investigated firstly. By concatenating low-dimensional phoneme variability factors in the language branch spaces, phonotactic language branch variability factor is obtained. Language models are trained within and between language branches with support vector machine (SVM). The proposed method uses dynamic and discriminative pronunciation phonotactic characteristics while it doesn’t involve fallible phoneme sequences. Results on 2011 NIST Language Recognition Evaluation (LRE) 30s data set show that the proposed method outperforms parallel phoneme recognizer followed by vector space models (PPRVSM) and ivector systems, and obtains relative improvement of 28.2–72.0% in EER, minDCF and language-pair performance metrics significantly.