Speaker Verification Method Research Articles

Joint speaker encoder and neural back-end model for fully end-to-end automatic speaker verification with multiple enrollment utterances

Conventional automatic speaker verification systems can usually be decomposed into a front-end model such as time delay neural network (TDNN) for extracting speaker embeddings and a back-end model such as statistics-based probabilistic linear discriminant analysis (PLDA) or neural network-based neural PLDA (NPLDA) for similarity scoring. However, the sequential optimization of the front-end and back-end models may lead to a local minimum, which theoretically prevents the whole system from achieving the best optimization. Although some methods have been proposed for jointly optimizing the two models, such as the generalized end-to-end (GE2E) model and NPLDA E2E model, most of these methods have not fully investigated how to model the intra-relationship between multiple enrollment utterances. In this paper, we propose a new E2E joint method for speaker verification especially designed for the practical scenario of multiple enrollment utterances. To leverage the intra-relationship among multiple enrollment utterances, our model comes equipped with frame-level and utterance-level attention mechanisms. Additionally, focal loss is utilized to balance the importance of positive and negative samples within a mini-batch and focus on the difficult samples during the training process. We also utilize several data augmentation techniques, including conventional noise augmentation using MUSAN and RIRs datasets and a unique speaker embedding-level mixup strategy for better optimization.

Analysis of Speaker Verification Method Based on Hidden Markov Models for Continuous Speech System

Analysis of Speaker Verification Method Based on Hidden Markov Models for Continuous Speech System

Speaker verification using attentive multi-scale convolutional recurrent network

In this paper, we propose a speaker verification method by an Attentive Multi-scale Convolutional Recurrent Network (AMCRN). The proposed AMCRN can acquire both local spatial information and global sequential information from the input speech recordings. In the proposed method, logarithm Mel spectrum is extracted from each speech recording and then fed to the proposed AMCRN for learning speaker embedding. Afterwards, the learned speaker embedding is fed to the back-end classifier (such as cosine similarity metric) for scoring in the testing stage. The proposed method is compared with state-of-the-art methods for speaker verification. Experimental data are three public datasets that are selected from two large-scale speech corpora (VoxCeleb1 and VoxCeleb2). Experimental results show that our method exceeds baseline methods in terms of equal error rate and minimal detection cost function, and has advantages over most of baseline methods in terms of computational complexity and memory requirement. In addition, our method generalizes well across truncated speech segments with different durations, and the speaker embedding learned by the proposed AMCRN has stronger generalization ability across two back-end classifiers.

Application of Fusion of Various Spontaneous Speech Analytics Methods for Improving Far-Field Neural-Based Diarization

Recently developed methods in spontaneous speech analytics require the use of speaker separation based on audio data, referred to as diarization. It is applied to widespread use cases, such as meeting transcription based on recordings from distant microphones and the extraction of the target speaker’s voice profiles from noisy audio. However, speech recognition and analysis can be hindered by background and point-source noise, overlapping speech, and reverberation, which all affect diarization quality in conjunction with each other. To compensate for the impact of these factors, there are a variety of supportive speech analytics methods, such as quality assessments in terms of SNR and RT60 reverberation time metrics, overlapping speech detection, instant speaker number estimation, etc. The improvements in speaker verification methods have benefits in the area of speaker separation as well. This paper introduces several approaches aimed towards improving diarization system quality. The presented experimental results demonstrate the possibility of refining initial speaker labels from neural-based VAD data by means of fusion with labels from quality estimation models, overlapping speech detectors, and speaker number estimation models, which contain CNN and LSTM modules. Such fusing approaches allow us to significantly decrease DER values compared to standalone VAD methods. Cases of ideal VAD labeling are utilized to show the positive impact of ResNet-101 neural networks on diarization quality in comparison with basic x-vectors and ECAPA-TDNN architectures trained on 8 kHz data. Moreover, this paper highlights the advantage of spectral clustering over other clustering methods applied to diarization. The overall quality of diarization is improved at all stages of the pipeline, and the combination of various speech analytics methods makes a significant contribution to the improvement of diarization quality.

A speaker verification backend with robust performance across conditions

In this paper, we address the problem of speaker verification in conditions unseen or unknown during development. A standard method for speaker verification consists of extracting speaker embeddings with a deep neural network and processing them through a backend composed of probabilistic linear discriminant analysis (PLDA) and global logistic regression score calibration. This method is known to result in systems that work poorly on conditions different from those used to train the calibration model. We propose to modify the standard backend, introducing an adaptive calibrator that uses duration and other automatically extracted side-information to adapt to the conditions of the inputs. The backend is trained discriminatively to optimize binary cross-entropy. When trained on a number of diverse datasets that are labeled only with respect to speaker, the proposed backend consistently and, in some cases, dramatically improves calibration, compared to the standard PLDA approach, on a number of held-out datasets, some of which are markedly different from the training data. Discrimination performance is also consistently improved. We show that joint training of the PLDA and the adaptive calibrator is essential — the same benefits cannot be achieved when freezing PLDA and fine-tuning the calibrator. To our knowledge, the results in this paper are the first evidence in the literature that it is possible to develop a speaker verification system with robust out-of-the-box performance on a large variety of conditions.

Tune-In: Training Under Negative Environments with Interference for Attention Networks Simulating Cocktail Party Effect

We study the cocktail party problem and propose a novel attention network called Tune-In, abbreviated for training under negative environments with interference. It firstly learns two separate spaces of speaker-knowledge and speech-stimuli based on a shared feature space, where a new block structure is designed as the building block for all spaces, and then cooperatively solves different tasks. Between the two spaces, information is cast towards each other via a novel cross- and dual-attention mechanism, mimicking the bottom-up and top-down processes of a human's cocktail party effect. It turns out that substantially discriminative and generalizable speaker representations can be learnt in severely interfered conditions via our self-supervised training. The experimental results verify this seeming paradox. The learnt speaker embedding has superior discriminative power than a standard speaker verification method; meanwhile, Tune-In achieves remarkably better speech separation performances in terms of SI-SNRi and SDRi consistently in all test modes, and especially at lower memory and computational consumption, than state-of-the-art benchmark systems.

Vocal Tract Length Perturbation for Text-Dependent Speaker Verification With Autoregressive Prediction Coding

In this letter, we propose a vocal tract length (VTL) perturbation method for text-dependent speaker verification (TD-SV), in which a set of TD-SV systems are trained, one for each VTL factor, and score-level fusion is applied to make a final decision. Next, we explore the bottleneck (BN) feature extracted by training deep neural networks with a self-supervised objective, autoregressive predictive coding (APC), for TD-SV and compare it with the well-studied speaker-discriminant BN feature. The proposed VTL method is then applied to APC and speaker-discriminant BN features. In the end, we combine the VTL perturbation systems trained on MFCC and the two BN features in the score domain. Experiments are performed on the RedDots challenge 2016 database of TD-SV using short utterances with Gaussian mixture model-universal background model and i-vector techniques. Results show the proposed methods significantly outperform the baselines.

Secure mmWave-Radar-Based Speaker Verification for IoT Smart Home

Voice assistant devices function as interaction gateways in the Internet-of-Things (IoT) smart home. By using voice assistants, users are able to control smart homes via speech commands. However, voice assistants introduce potential security risks and privacy disclosures. For example, malicious actors could impersonate genuine users to send smart home speech commands. Speaker/user verification thus becomes a critical issue for smart home security. This article proposes a secure method for speaker verification in IoT smart homes using millimeter-wave (mmWave) radar. Specifically, we utilize the radar to capture both vocal cord vibration (VCV) and lip motion (LM) as multimodal biometrics for identifying speakers. Traditional voice-based speaker verification methods are vulnerable to impostor attacks, such as replay attacks and voice synthesis attacks, that use recorded or imitated voice audio to spoof the system. Our approach is able to protect IoT smart homes from these attacks by continuously detecting the liveness of the user using mmWave sensing and deep learning techniques. Extensive experiments show that the proposed approach can achieve high verification accuracy and be more robust against imposter attacks.

Dilated residual networks with multi-level attention for speaker verification

With the development of deep learning techniques, speaker verification (SV) systems based on deep neural network (DNN) achieve competitive performance compared with traditional i-vector-based works. Previous DNN-based SV methods usually employ time-delay neural network, limiting the extension of the network for an effective representation. Besides, existing attention mechanisms used in DNN-based SV systems are only applied to a single level of network architectures, leading to insufficiently extraction of important features.To address above issues, we propose an effective deep speaker embedding architecture for SV, which combines a residual connection of one-dimensional dilated convolutional layers, called dilated residual networks (DRNs), with a multi-level attention model. The DRNs can not only capture long time-frequency context information of features, but also exploit information from multiple layers of DNN. In addition, the multi-level attention model, which consists of two-dimensional convolutional block attention modules employed at the frame level and the vector-based attention utilized at the pooling layer, can emphasize important features at multiple levels of DNN. Experiments conducted on NIST SRE 2016 dataset show that the proposed architecture achieves a superior equal error rate (EER) of 7.094% and a better detection cost function (DCF16) of 0.552 compared with state-of-the-art methods. Furthermore, the ablation experiments demonstrate the effectiveness of dilated convolutions and the multi-level attention on SV tasks.

Speaker verification from mixture of speech and non-speech audio signals via using pole distribution of piecewise linear predictive coding coefficients

This paper examines a method of speaker verification from mixture of speech and non-speech audio signals via using pole distribution of speech signals. The poles of speech signal derived from the all-pole speech production model are obtained via a neural net called bagging CAN2 (competitive associative net 2) for learning efficient piecewise linear approximation of nonlinear functions. We show an analysis that poles of the mixture of two audio signals are expected to show three typical phenomena. Namely, with the change of the weight of the mixture, there will be (1) a sudden change of the number of poles, (2) a gradual change of the number of poles, and (3) novel poles not in the original signals. By means of experiments, we show that there are poles exhibiting those phenomena, and they are considered to affect the performance on speaker verification. As a performance measure of speaker verification, recall shows large decrease with the change of speech signal from unmixed to mixed, especially for the mixture with violin signal as well as with speech signal. Furthermore, precision has larger value than recall for the same dataset of mixture of audio signals. Finally, we show the usefulness of the present method in further researches.

An efficient text-independent speaker verification for short utterance data from Mobile devices

Speaker verification is the process used to recognize a speaker from his/her voice characteristics by extracting the features. Speaker verification with text-independent data is a process of verifying the speaker identity without limitation in the speech content. In the speaker verification process, long utterances are normally used but it contains lot of silences leading to complexity and more disruptions. So, we are performing speaker verification method based on short utterance data. The main objective of the research work is to extract, characterize, and recognize the information about speaker identity. Our proposed work contains four stages: 1) utterance partitioning, 2) feature extraction, 3) feature selection, and 4) classification. In our proposed model, an utterance partitioning approach is used to shorten the full-length speech into numerous short-length utterances before the pre-processing stage. In the feature extraction phase, noise removal is carried out with pre-emphasis filter in the pre-processing step. The Mel Advanced Hilbert-Huang Cepstral Coefficients (MAHCC) technique is used for extracting the features from the given input speech signal. Furthermore, the feature selection process is done with the help of a Crow Search Algorithm (CSA) by ranking the given feature set to obtain optimal features for classification. In the classification stage, the Deep Hidden Markov Model (DHMM) method is introduced to classify the features for speaker verification with discriminative pre-training process. Thus, the proposed approach provides an accurate classification and the implementation results show that the performance of the proposed method is better than the existing methods.

Limited-data automatic speaker verification algorithm using band-limitedphase-only correlation function

In this paper, a new method to deal with automatic speaker verification based on band-limited phaseonly correlation (BLPOC) is proposed. The aim of this study is to validate the use of the BLPOC function as a new limited-data automatic speaker verification technique. Although some speaker verification techniques have high accuracy, efficiency usually depends on the extraction of complex theoretical information from speech signals and the amount of the data for training the algorithms. The BLPOC function is a high-accuracy biometric technique traditionally implemented in human identification by fingerprints (through image-matching). When applying the BLPOC function in automatic speaker verification through the proposed algorithms (under limited-data conditions), a 98.24% true acceptance rate (TAR) and 87.17% true rejection rate (TRR) in a custom database (and 93.75% TAR and 67.05% TRR in the ELSDSR database) were obtained. The proposed algorithm is a theoretically simple method for automatic speaker verification whose main advantage is that it can provide identification under limited-data conditions. In this sense, the BLPOC function could be applicable in other limited-data biometric identifications by sound signals.

A Speaker Verification Method Based on TDNN–LSTMP

In speaker recognition, a robust recognition method is essential. This paper proposes a speaker verification method that is based on the time-delay neural network (TDNN) and long short-term memory with recurrent project layer (LSTMP) model for the speaker modeling problem in speaker verification. In this work, we present the application of the fusion of TDNN and LSTMP to the i-vector speaker recognition system that is based on the Gaussian mixture model-universal background model. By using a model that can establish long-term dependencies to create a universal background model that contains a larger amount of speaker information, it is possible to extract more feature parameters, which are speaker dependent, from the speech signal. We conducted experiments with this method on four corpora: two in Chinese and two in English. The equal error rate, minimum detection cost function and detection error tradeoff curve are used as criteria for system performance evaluation. The experimental results show that the TDNN–LSTMP/i-vector speaker recognition method outperforms the baseline system on both Chinese and English corpora and has better robustness.

Multitaper MFCC and normalized multitaper phase-based features for speaker verification

One trend of current research activity in speaker verification systems focuses on features that capture complementary information to the Mel-frequency cepstral coefficients (MFCCs). In conventional speaker verification methods, MFCCs are usually computed from a single-tapered DFT spectrum. These methods use the power of the Fourier transform of the time-domain speech frames and ignore the phase component due to its large amount of uncertainty. The multitaper phase information has been also ignored when the multitaper MFCC method applies. We propose a phase information extraction method that normalizes the change variation in the multitaper phase. The goal of this work is to incorporate phase information using low-variance multitaper spectrum estimation method instead of conventional single-taper window and normalizes the change variation in multitaper phase according to the frame position of the input speech to reduce the uncertainty of multitaper phase information in both the state-of-the-art Gaussian mixture model-universal background model (GMM-UBM) baseline and the i-vector speaker verification system. The experiments are conducted on the TIMIT database for both the clean and noisy conditions (SNR = 10 dB). The multitaper inverted-MFCC is also added to the overall system to produce diversity with respect to the fused subsystem. The results show relative improvement of 34% and 25% in terms of equal error rate (EER) over the MFCC baseline in clean and noisy conditions for GMM-UBM system, respectively. The relative improvement in terms of EER with i-vector back-end are about 36% and 16% in clean and noisy conditions as well.

Robust speaker verification in band-localized noise conditions

This research paper presents a robust method for speaker verification in noisy environments. The noise is assumed to contaminate certain parts of the voice’s frequency spectrum. Therefore, the verification method is based on splitting the noisy speech into subsidiary bands then using a threshold to sense the existence of noise in a specific part of the spectrum, hence activating an adaptive filter in that part to track changes in noise’s characteristics and remove it. The decomposition is achieved using low complexity quadrature mirror filters QMF in three levels thus achieving four bands in a non-uniform that resembles human hearing perceptual. Speaker recognition is based on vector quantization VQ or template matching technique. Features are extracted from speaker’s voice using the normalized power in a similar way to the Mel-frequency cepstral coefficients. The performance of the proposed system is evaluated using 60 speakers subjected to five levels of signal to noise ratio SNR using total success rate TSR, false acceptance rate FAR, false rejection rate FRR and equal error rate. The proposed method showed higher recognition accuracy than existing methods in severe noise conditions.

범주문법 속에서의 명사와 명사구 : 프랑스어 관계대명사 `qui`와 `que` 구문을 중심으로

In this paper, we propose a multimodal speaker verification method using teeth image and voice as biometric trait for personal verification in mobile terminal equipment. The proposed method obtains the biometric traits using image and sound input devices of smart-phone that is one of mobile terminal equipments, and performs verification with biometric traits. In addition, the proposed method consists the multimodal-fashion of combining two biometric authentication scores for totally performance enhancement, the fusion method is accompanied a weighted-summation method which has comparative simple structure and superior performance for considering limited resources of system. The performance evaluation of proposed multimodal speaker authentication system conducts using a database acquired in smart-phone for 40 subjects. The experimental result shows 8.59% of EER in case of teeth verification 11.73% in case of voice verification and the multimodal speaker authentication result presented the 4.05% of EER. In the experimental result, we obtain the enhanced performance more than each using teeth and voice by using the simple weight-summation method in the multimodal speaker verification system.

A Novel Arabic Text-independent Speaker Verification System based on Fuzzy Hidden Markov Model

The most important shortcoming of the current speaker verification methods (based on knowledge or possession) is that this process is not sure whether the holder of the given ID is the entitled one or an imposter. Using biometrics in the verification system is designated for minimizing this problem and decreasing the necessity of carrying the tokens. In this paper, a novel Arabic text-independent speaker verification system is presented. First of all, new speech features are proposed for speaker characterization, which denoted as Wavelet Packet Four-Directional Features (WPFDF). With the objective of speaker verification, the paper proposes a Fuzzy Hidden Markov Model, termed FHMM, where the kernel fuzzy c-means (KFCM) is extended to calculate fuzzy memberships of HMMs training samples. Thus, information loss is reduced as well as recognition rate is increased. The proposed approach reached 98.38% of recognition rate.

Sub-vector based biometric speaker verification using MLLR super-vector

In this paper, we propose a sub-vector based speaker characterization method for biometric speaker verification, where speakers are represented by uniform segmentation of their maximum likelihood linear regression (MLLR) super-vectors called m-vectors. The MLLR transformation is estimated with respect to universal background model (UBM) without any speech/phonetic information. We introduce two strategies for segmentation of MLLR super-vector: one is called disjoint and other is an overlapped window technique. During test phase, m-vectors of the test utterance are scored against the claimant speaker. Before scoring, m-vectors are post-processed to compensate the session variability. In addition, we propose a clustering algorithm for multiple-class wise MLLR transformation, where Gaussian components of the UBM are clustered into different groups using the concept of expectation maximization (EM) and maximum likelihood (ML). In this case, MLLR transformations are estimated with respect to each class using the sufficient statistics accumulated from the Gaussian components belonging to the particular class, which are then used for m-vector system. The proposed method needs only once alignment of the data with respect to the UBM for multiple MLLR transformations. We first show that the proposed multi-class m-vector system shows promising speaker verification performance when compared to the conventional i-vector based speaker verification system. Secondly, the proposed EM based clustering technique is robust to the random initialization in-contrast to the conventional K-means algorithm and yields system performance better/equal which is best obtained by the K-means. Finally, we show that the fusion of the m-vector with the i-vector further improves the performance of the speaker verification in both score as well as feature domain. The experimental results are shown on various tasks of NIST 2008 speaker recognition evaluation (SRE) core condition.

Speaker verification method for operation system of consumer electronic devices

A system is proposed that can remotely operate consumer electronic devices by voice. It uses the mobile phone as a controller. And it uses the CELP(code excited linear prediction) parameters that are used for speech coding in mobile phones. A speaker verification function protects private information and separates the user's voice from that of people nearby who are also speaking. A CELP-based speaker verification method is used to match the audio stream by comparing the trajectories of continuous phonemes. Experimental evaluation of the speaker verification method demonstrated the effectiveness of the proposed verification method.1.

Text Independent Speaker Verficiation Using Dominant State Information of HMM-UBM

본 논문에서는 Hidden Markov Model(HMM) - Universal Background Model(UBM)의 주 상태 정보 기반의 i-vector 추출 기술을 제안한다. Ergodic HMM이 UBM을 추정하는데 쓰였으며, 이를 통해 동일 화자 음성에도 다양하게 존재하는 특성을 HMM states로 분류할 수 있다. 제안한 방법을 이용하면 HMM의 state 개수에 따라 i-vector 들이 추출되는데, 주 상태 정보 방법을 통해 이들 중 하나를 선택한다. 제안한 방법을 검증하기 위해 National Institute of Standards and Technology(NIST) Speaker Recognition Evaluation(SRE) database를 이용하여 실험을 하였으며, Equal Error Rate(EER) 성능 수치에서 12 %의 성능 향상을 확인할 수 있었다. We present a speaker verification method by extracting i-vectors based on dominant state information of Hidden Markov Model (HMM) - Universal Background Model (UBM). Ergodic HMM is used for estimating UBM so that various characteristic of individual speaker can be effectively classified. Unlike Gaussian Mixture Model(GMM)-UBM based speaker verification system, the proposed system obtains i-vectors corresponding to each HMM state. Among them, the i-vector for feature is selected by extracting it from the specific state containing dominant state information. Relevant experiments are conducted for validating the proposed system performance using the National Institute of Standards and Technology (NIST) 2008 Speaker Recognition Evaluation (SRE) database. As a result, 12 % improvement is attained in terms of equal error rate.